mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
Nicolò Boschi
1d5f852809
* test(system): blackbox system-test suite driven only through the public API The api-slim suite is ~500 files, each covering one mechanism, which leaves composition bugs — consolidation wiping facts, a delta refresh missing a backdated window, a transfer dropping evidence — with nothing watching them. This adds the layer that does: tests that drive a real `hindsight-api` process over HTTP through the published Python client, with no engine access and no SQL. Determinism comes from a stub server implementing the OpenAI chat-completions/embeddings APIs and a Cohere-compatible rerank endpoint. The server under test is pointed at it with ordinary environment variables (`HINDSIGHT_API_LLM_BASE_URL`, `..._EMBEDDINGS_OPENAI_BASE_URL`, `..._RERANKER_SILICONFLOW_BASE_URL`), so no production code changes are needed and the real provider transport — the OpenAI client, JSON repair, retries, structured output — is exercised for real rather than replaced by a fake. Design decisions worth knowing: - Rules match on a named pipeline step, not the request body. Nothing on the wire identifies the caller (the json_schema name is the constant "response", and the soft json_object path sends no schema at all), so a step is a short anchor phrase from its prompt — owned by `steps.py` so a prompt edit is one line, not thirty red tests. - An unmatched call fails the test and prints the rule to paste in. `MockLLM` synthesizes plausible facts from its input when it doesn't recognize a call, which is why tests using it pass without proving anything. - Requests are validated strictly. A fake you control drifts permissive, and the provider bugs that have actually hurt here (Bedrock rejecting response_format, Azure 400ing on prompt_cache_key) are all "the provider refused our request". - Background work stays on. Consolidation runs in the worker after retain returns; tests wait for it via the operations API rather than disabling it, because that asynchronous half is where the composition bugs live. - The server runs from a scratch dir holding an empty `.env`, since a discovered `.env` deliberately overrides the ambient environment (#2961) and would otherwise silently replace the whole test configuration. CI (`test-system`) needs no provider secrets, so unlike every `test-api` job it also runs on fork PRs — and with embeddings and reranking both stubbed, nothing loads sentence-transformers, so it skips torch and the HuggingFace cache too. * test(system): pin the whole recall payload, not just a keyword With the LLM, embedder and reranker all stubbed, a recall is a pure function of its input — so assert it as one. Ranking order, the rendered fact text, the document/chunk identity composite, the temporal fields, the empty envelope sections, and each of the four scores are now pinned. The three retrieval components are reproducible to the bit across runs; only `final` drifts (~1e-9), because it folds in recency measured against wall-clock now, so it gets a tolerance and a comment saying why. The previous assertion ("Berlin appears somewhere") passed just as happily with fusion inverted, the reranker contributing nothing, or the temporal fields silently stopping being parsed. * ci: skip the two unconditional jobs for a system-tests-only change Iterating on hindsight-system-tests/ costs ~6 minutes of CI against a 36-second test. Of the 96 jobs, 91 already skip for a change confined to that package; the remainder were build-docs (2.2 min) and verify-generated-files (3.6 min), both deliberately unconditional. Both now hang off a new `outside-system-tests` filter rather than a positive one, so they still run for every other change in the repo and step aside only for a PR touching nothing but the system-test package — which no generated file is produced from and the docs site never reads. The filter is a lone negation on purpose: paths-filter builds one matcher per pattern and ORs them, so the natural-looking ['**', '!dir/**'] pair matches every file and leaves the filter permanently true. Verified against picomatch directly before committing.
5811 lines
197 KiB
YAML
5811 lines
197 KiB
YAML
name: CI
|
|
|
|
on:
|
|
pull_request:
|
|
branches: [ main ]
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
group: ci-${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
detect-changes:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
permissions:
|
|
pull-requests: read
|
|
outputs:
|
|
core: ${{ steps.filter.outputs.core }}
|
|
clients-ts: ${{ steps.filter.outputs.clients-ts }}
|
|
clients-python: ${{ steps.filter.outputs.clients-python }}
|
|
clients-rust: ${{ steps.filter.outputs.clients-rust }}
|
|
clients-go: ${{ steps.filter.outputs.clients-go }}
|
|
control-plane: ${{ steps.filter.outputs.control-plane }}
|
|
cli: ${{ steps.filter.outputs.cli }}
|
|
docker: ${{ steps.filter.outputs.docker }}
|
|
helm: ${{ steps.filter.outputs.helm }}
|
|
doc-examples: ${{ steps.filter.outputs.doc-examples }}
|
|
embed: ${{ steps.filter.outputs.embed }}
|
|
all-npm: ${{ steps.filter.outputs.all-npm }}
|
|
hindsight-all: ${{ steps.filter.outputs.hindsight-all }}
|
|
integration-tests: ${{ steps.filter.outputs.integration-tests }}
|
|
system-tests: ${{ steps.filter.outputs.system-tests }}
|
|
outside-system-tests: ${{ steps.filter.outputs.outside-system-tests }}
|
|
integrations-openclaw: ${{ steps.filter.outputs.integrations-openclaw }}
|
|
integrations-ai-sdk: ${{ steps.filter.outputs.integrations-ai-sdk }}
|
|
integrations-eliza: ${{ steps.filter.outputs.integrations-eliza }}
|
|
integrations-agent-framework: ${{ steps.filter.outputs.integrations-agent-framework }}
|
|
integrations-composio: ${{ steps.filter.outputs.integrations-composio }}
|
|
integrations-chat: ${{ steps.filter.outputs.integrations-chat }}
|
|
integrations-claude-code: ${{ steps.filter.outputs.integrations-claude-code }}
|
|
integrations-coding-agents: ${{ steps.filter.outputs.integrations-coding-agents }}
|
|
integrations-cline: ${{ steps.filter.outputs.integrations-cline }}
|
|
integrations-codex: ${{ steps.filter.outputs.integrations-codex }}
|
|
integrations-github-copilot: ${{ steps.filter.outputs.integrations-github-copilot }}
|
|
integrations-continue: ${{ steps.filter.outputs.integrations-continue }}
|
|
integrations-cursor-cli: ${{ steps.filter.outputs.integrations-cursor-cli }}
|
|
integrations-zcode: ${{ steps.filter.outputs.integrations-zcode }}
|
|
integrations-agent-plugin: ${{ steps.filter.outputs.integrations-agent-plugin }}
|
|
integrations-copilot-cli: ${{ steps.filter.outputs.integrations-copilot-cli }}
|
|
extensions-supabase-tenant: ${{ steps.filter.outputs.extensions-supabase-tenant }}
|
|
integrations-crewai: ${{ steps.filter.outputs.integrations-crewai }}
|
|
integrations-litellm: ${{ steps.filter.outputs.integrations-litellm }}
|
|
integrations-pydantic-ai: ${{ steps.filter.outputs.integrations-pydantic-ai }}
|
|
integrations-ag2: ${{ steps.filter.outputs.integrations-ag2 }}
|
|
integrations-autogen: ${{ steps.filter.outputs.integrations-autogen }}
|
|
integrations-aider: ${{ steps.filter.outputs.integrations-aider }}
|
|
integrations-langgraph: ${{ steps.filter.outputs.integrations-langgraph }}
|
|
integrations-llamaindex: ${{ steps.filter.outputs.integrations-llamaindex }}
|
|
integrations-paperclip: ${{ steps.filter.outputs.integrations-paperclip }}
|
|
integrations-opencode: ${{ steps.filter.outputs.integrations-opencode }}
|
|
integrations-eve: ${{ steps.filter.outputs.integrations-eve }}
|
|
integrations-cursor: ${{ steps.filter.outputs.integrations-cursor }}
|
|
integrations-zed: ${{ steps.filter.outputs.integrations-zed }}
|
|
integrations-n8n: ${{ steps.filter.outputs.integrations-n8n }}
|
|
integrations-zapier: ${{ steps.filter.outputs.integrations-zapier }}
|
|
integrations-cloudflare-oauth-proxy: ${{ steps.filter.outputs.integrations-cloudflare-oauth-proxy }}
|
|
integrations-superagent: ${{ steps.filter.outputs.integrations-superagent }}
|
|
integrations-lockfiles: ${{ steps.filter.outputs.integrations-lockfiles }}
|
|
integrations-openai-agents: ${{ steps.filter.outputs.integrations-openai-agents }}
|
|
integrations-openhands: ${{ steps.filter.outputs.integrations-openhands }}
|
|
integrations-devin-desktop: ${{ steps.filter.outputs.integrations-devin-desktop }}
|
|
integrations-pipecat: ${{ steps.filter.outputs.integrations-pipecat }}
|
|
integrations-agentcore: ${{ steps.filter.outputs.integrations-agentcore }}
|
|
integrations-smolagents: ${{ steps.filter.outputs.integrations-smolagents }}
|
|
integrations-claude-agent-sdk: ${{ steps.filter.outputs.integrations-claude-agent-sdk }}
|
|
integrations-dify: ${{ steps.filter.outputs.integrations-dify }}
|
|
integrations-gemini-spark: ${{ steps.filter.outputs.integrations-gemini-spark }}
|
|
integrations-vapi: ${{ steps.filter.outputs.integrations-vapi }}
|
|
integrations-flowise: ${{ steps.filter.outputs.integrations-flowise }}
|
|
integrations-google-adk: ${{ steps.filter.outputs.integrations-google-adk }}
|
|
integrations-obsidian: ${{ steps.filter.outputs.integrations-obsidian }}
|
|
integrations-omo: ${{ steps.filter.outputs.integrations-omo }}
|
|
integrations-haystack: ${{ steps.filter.outputs.integrations-haystack }}
|
|
tools-agent-sdk: ${{ steps.filter.outputs.tools-agent-sdk }}
|
|
integrations-roo-code: ${{ steps.filter.outputs.integrations-roo-code }}
|
|
dev: ${{ steps.filter.outputs.dev }}
|
|
ci: ${{ steps.filter.outputs.ci }}
|
|
# Secrets are available for internal PRs and workflow_dispatch.
|
|
# Fork PRs via pull_request event do NOT have access to secrets.
|
|
has_secrets: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }}
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- uses: dorny/paths-filter@v4
|
|
id: filter
|
|
with:
|
|
filters: |
|
|
core:
|
|
- 'hindsight-api-slim/**'
|
|
- 'hindsight-api/**'
|
|
- 'scripts/**'
|
|
- '.python-version'
|
|
- '.env.example'
|
|
# The workspace lockfile and root manifest determine which versions
|
|
# the API actually runs against, so a change to either can break it
|
|
# without touching a line of source. Both are deliberately bare
|
|
# patterns: they match only the files at the repository root. A
|
|
# glob such as '**/uv.lock' would pull every integration lockfile
|
|
# into the core suite.
|
|
- 'uv.lock'
|
|
- 'pyproject.toml'
|
|
clients-ts:
|
|
- 'hindsight-clients/typescript/**'
|
|
- 'package.json'
|
|
- 'package-lock.json'
|
|
clients-python:
|
|
- 'hindsight-clients/python/**'
|
|
clients-rust:
|
|
- 'hindsight-clients/rust/**'
|
|
clients-go:
|
|
- 'hindsight-clients/go/**'
|
|
control-plane:
|
|
- 'hindsight-control-plane/**'
|
|
- 'package.json'
|
|
- 'package-lock.json'
|
|
cli:
|
|
- 'hindsight-cli/**'
|
|
docker:
|
|
- 'docker/**'
|
|
helm:
|
|
- 'helm/**'
|
|
# The RUNNABLE samples only. This replaces a broad `docs` filter that also
|
|
# matched 'hindsight-docs/**', '*.md' and 'hindsight-integrations/**' — the
|
|
# last of those so an integration rename would be caught by the docs build's
|
|
# integrations check, except that build (build-docs) has no `if:` and runs
|
|
# unconditionally anyway. test-doc-examples was the filter's only consumer,
|
|
# and it executes every sample against a live LLM-backed server, so every
|
|
# integration and prose-only PR paid for four provider-credentialed runs that
|
|
# none of those files can affect.
|
|
doc-examples:
|
|
- 'hindsight-docs/examples/**'
|
|
- 'scripts/test-doc-examples.sh'
|
|
embed:
|
|
- 'hindsight-embed/**'
|
|
all-npm:
|
|
- 'hindsight-all-npm/**'
|
|
- 'package.json'
|
|
- 'package-lock.json'
|
|
hindsight-all:
|
|
- 'hindsight-all/**'
|
|
integration-tests:
|
|
- 'hindsight-integration-tests/**'
|
|
system-tests:
|
|
- 'hindsight-system-tests/**'
|
|
# True when the PR touches anything *other* than the system-test
|
|
# package. The two unconditional-by-design jobs (build-docs and
|
|
# verify-generated-files) hang off this rather than off a positive
|
|
# filter, so they keep running for every change in the repo and only
|
|
# step aside for a PR confined to hindsight-system-tests/ — a package
|
|
# nothing generates from and the docs site never reads. Iterating on
|
|
# that suite otherwise costs ~6 minutes of unrelated CI against a
|
|
# 36-second test.
|
|
outside-system-tests:
|
|
# A lone negation, deliberately. paths-filter builds one matcher per
|
|
# pattern and ORs them, so adding '**' alongside would make every
|
|
# file match and the filter always true — a change that looks right
|
|
# and silently does nothing.
|
|
- '!hindsight-system-tests/**'
|
|
integrations-openclaw:
|
|
- 'hindsight-integrations/openclaw/**'
|
|
integrations-ai-sdk:
|
|
- 'hindsight-integrations/ai-sdk/**'
|
|
integrations-eliza:
|
|
- 'hindsight-integrations/eliza/**'
|
|
integrations-agent-framework:
|
|
- 'hindsight-integrations/agent-framework/**'
|
|
integrations-composio:
|
|
- 'hindsight-integrations/composio/**'
|
|
integrations-chat:
|
|
- 'hindsight-integrations/chat/**'
|
|
integrations-claude-code:
|
|
- 'hindsight-integrations/claude-code/**'
|
|
integrations-coding-agents:
|
|
- 'hindsight-integrations/coding-agents/**'
|
|
integrations-cline:
|
|
- 'hindsight-integrations/cline/**'
|
|
integrations-codex:
|
|
- 'hindsight-integrations/codex/**'
|
|
integrations-github-copilot:
|
|
- 'hindsight-integrations/github-copilot/**'
|
|
integrations-continue:
|
|
- 'hindsight-integrations/continue/**'
|
|
integrations-cursor-cli:
|
|
- 'hindsight-integrations/cursor-cli/**'
|
|
integrations-copilot-cli:
|
|
- 'hindsight-integrations/copilot-cli/**'
|
|
extensions-supabase-tenant:
|
|
- 'hindsight-extensions/supabase-tenant/**'
|
|
integrations-crewai:
|
|
- 'hindsight-integrations/crewai/**'
|
|
integrations-litellm:
|
|
- 'hindsight-integrations/litellm/**'
|
|
integrations-pydantic-ai:
|
|
- 'hindsight-integrations/pydantic-ai/**'
|
|
integrations-ag2:
|
|
- 'hindsight-integrations/ag2/**'
|
|
integrations-autogen:
|
|
- 'hindsight-integrations/autogen/**'
|
|
integrations-aider:
|
|
- 'hindsight-integrations/aider/**'
|
|
integrations-langgraph:
|
|
- 'hindsight-integrations/langgraph/**'
|
|
integrations-llamaindex:
|
|
- 'hindsight-integrations/llamaindex/**'
|
|
integrations-haystack:
|
|
- 'hindsight-integrations/haystack/**'
|
|
integrations-paperclip:
|
|
- 'hindsight-integrations/paperclip/**'
|
|
integrations-opencode:
|
|
- 'hindsight-integrations/opencode/**'
|
|
integrations-eve:
|
|
- 'hindsight-integrations/eve/**'
|
|
integrations-cursor:
|
|
- 'hindsight-integrations/cursor/**'
|
|
integrations-zed:
|
|
- 'hindsight-integrations/zed/**'
|
|
integrations-zcode:
|
|
- 'hindsight-integrations/zcode/**'
|
|
integrations-agent-plugin:
|
|
- 'hindsight-integrations/agent-plugin/**'
|
|
integrations-n8n:
|
|
- 'hindsight-integrations/n8n/**'
|
|
integrations-zapier:
|
|
- 'hindsight-integrations/zapier/**'
|
|
integrations-cloudflare-oauth-proxy:
|
|
- 'hindsight-integrations/cloudflare-oauth-proxy/**'
|
|
integrations-superagent:
|
|
- 'hindsight-integrations/superagent/**'
|
|
integrations-lockfiles:
|
|
- 'hindsight-integrations/*/package-lock.json'
|
|
- 'hindsight-integrations/*/package.json'
|
|
- 'scripts/check-integration-lockfiles.sh'
|
|
integrations-openai-agents:
|
|
- 'hindsight-integrations/openai-agents/**'
|
|
integrations-openhands:
|
|
- 'hindsight-integrations/openhands/**'
|
|
integrations-devin-desktop:
|
|
- 'hindsight-integrations/devin-desktop/**'
|
|
integrations-pipecat:
|
|
- 'hindsight-integrations/pipecat/**'
|
|
integrations-agentcore:
|
|
- 'hindsight-integrations/agentcore/**'
|
|
integrations-smolagents:
|
|
- 'hindsight-integrations/smolagents/**'
|
|
integrations-claude-agent-sdk:
|
|
- 'hindsight-integrations/claude-agent-sdk/**'
|
|
integrations-dify:
|
|
- 'hindsight-integrations/dify/**'
|
|
integrations-gemini-spark:
|
|
- 'hindsight-integrations/gemini-spark/**'
|
|
integrations-vapi:
|
|
- 'hindsight-integrations/vapi/**'
|
|
integrations-flowise:
|
|
- 'hindsight-integrations/flowise/**'
|
|
integrations-google-adk:
|
|
- 'hindsight-integrations/google-adk/**'
|
|
integrations-obsidian:
|
|
- 'hindsight-integrations/obsidian/**'
|
|
integrations-omo:
|
|
- 'hindsight-integrations/omo/**'
|
|
tools-agent-sdk:
|
|
- 'hindsight-tools/hindsight-agent-sdk/**'
|
|
integrations-roo-code:
|
|
- 'hindsight-integrations/roo-code/**'
|
|
dev:
|
|
- 'hindsight-dev/**'
|
|
ci:
|
|
- '.github/**'
|
|
|
|
# Fail fast if any hindsight-integrations/*/package-lock.json was regenerated
|
|
# from the monorepo root and ended up symlinked at a workspace path instead
|
|
# of the npm registry. That bit us on the 0.6.0 openclaw release — tsc in
|
|
# the release workflow couldn't find `@vectorize-io/hindsight-client`
|
|
# because its `resolved` url pointed at a workspace dir whose `dist/` was
|
|
# gitignored and unbuilt. Catching this at PR time means the release CI
|
|
# never hits that class of failure.
|
|
check-integration-lockfiles:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-lockfiles == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Check integration lockfiles resolve from the npm registry
|
|
run: ./scripts/check-integration-lockfiles.sh
|
|
|
|
build-api-python-versions:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
strategy:
|
|
matrix:
|
|
python-version: ['3.11', '3.12', '3.13', '3.14']
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
|
|
- name: Set up Python ${{ matrix.python-version }}
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: ${{ matrix.python-version }}
|
|
|
|
- name: Build hindsight-api
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
# `uv build` only packages the source; it does not prove the dependency set
|
|
# resolves or that the code imports on this interpreter. Install into a fresh
|
|
# env and run a byte-compile + import smoke test so the matrix actually
|
|
# exercises each Python version (notably 3.14).
|
|
- name: Install and smoke-test on Python ${{ matrix.python-version }}
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv venv --python ${{ matrix.python-version }} .venv-smoke
|
|
VIRTUAL_ENV=.venv-smoke uv pip install .
|
|
.venv-smoke/bin/python -m compileall -q hindsight_api
|
|
.venv-smoke/bin/python -c "import hindsight_api, hindsight_api.main, hindsight_api.config; from hindsight_api.engine import memory_engine, llm_wrapper; print('import OK')"
|
|
|
|
build-typescript-client:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.clients-ts == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '20'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci --workspace=hindsight-clients/typescript
|
|
|
|
- name: Build TypeScript client
|
|
run: npm run build --workspace=hindsight-clients/typescript
|
|
|
|
build-hindsight-all-npm:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.all-npm == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci --workspace=hindsight-all-npm
|
|
|
|
- name: Run tests
|
|
run: npm test --workspace=hindsight-all-npm
|
|
|
|
- name: Build
|
|
run: npm run build --workspace=hindsight-all-npm
|
|
|
|
build-openclaw-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-openclaw == 'true' ||
|
|
needs.detect-changes.outputs.clients-ts == 'true' ||
|
|
needs.detect-changes.outputs.all-npm == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
# openclaw depends on two monorepo workspaces via `file:` deps:
|
|
# @vectorize-io/hindsight-client and @vectorize-io/hindsight-all. Their
|
|
# `dist/` directories are gitignored, so we must build them first.
|
|
# Otherwise vitest/tsc in openclaw fails with
|
|
# "Failed to resolve entry for package ..." on the value imports.
|
|
- name: Install root workspace dependencies
|
|
run: npm ci
|
|
|
|
- name: Build hindsight-client (openclaw dep)
|
|
run: npm run build --workspace=hindsight-clients/typescript
|
|
|
|
- name: Build hindsight-all-npm (openclaw dep)
|
|
run: npm run build --workspace=hindsight-all-npm
|
|
|
|
- name: Build hindsight-agent-sdk (openclaw dep)
|
|
run: npm run build --workspace=hindsight-tools/hindsight-agent-sdk
|
|
|
|
- name: Install openclaw dependencies
|
|
working-directory: ./hindsight-integrations/openclaw
|
|
run: npm ci
|
|
|
|
# Build must run before tests: one unit test in src/backfill.test.ts
|
|
# creates a symlink to `$cwd/dist/backfill.js` and calls realpathSync on
|
|
# it via isDirectExecution(). Without a populated dist/ the realpath call
|
|
# throws, both paths stay unresolved, and the equality assertion fails.
|
|
- name: Build
|
|
working-directory: ./hindsight-integrations/openclaw
|
|
run: npm run build
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/openclaw
|
|
run: npm test
|
|
|
|
smoke-openclaw-install:
|
|
needs: [detect-changes, build-openclaw-integration]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-openclaw == 'true' ||
|
|
needs.detect-changes.outputs.clients-ts == 'true' ||
|
|
needs.detect-changes.outputs.all-npm == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
# Install the openclaw CLI globally. The smoke test exercises the real
|
|
# `openclaw plugins install` / `openclaw config set` / `openclaw plugins
|
|
# doctor` commands — not the in-repo integration tests — so a real CLI
|
|
# must be on PATH.
|
|
- name: Install openclaw CLI
|
|
run: npm install -g openclaw
|
|
|
|
- name: Verify openclaw CLI
|
|
run: openclaw --version
|
|
|
|
# openclaw depends on the workspace packages via published version
|
|
# ranges (^0.1.0 / ^0.5.0), not file: paths, so the smoke test's
|
|
# `openclaw plugins install <tarball>` resolves them straight from the
|
|
# npm registry. These builds are just for `npm pack` / local unit
|
|
# tests, not for resolving the plugin's runtime deps.
|
|
- name: Install root workspace dependencies
|
|
run: npm ci
|
|
|
|
- name: Build hindsight-client (openclaw dep)
|
|
run: npm run build --workspace=hindsight-clients/typescript
|
|
|
|
- name: Build hindsight-all-npm (openclaw dep)
|
|
run: npm run build --workspace=hindsight-all-npm
|
|
|
|
- name: Build hindsight-agent-sdk (openclaw dep)
|
|
run: npm run build --workspace=hindsight-tools/hindsight-agent-sdk
|
|
|
|
- name: Install openclaw dependencies
|
|
working-directory: ./hindsight-integrations/openclaw
|
|
run: npm ci
|
|
|
|
# Guards the guard: the smoke test's doctor assertion once passed a plugin
|
|
# whose hooks openclaw had blocked, so its logic has its own fixtures.
|
|
- name: Self-test the smoke test's doctor assertion
|
|
working-directory: ./hindsight-integrations/openclaw
|
|
run: ./scripts/doctor-check-selftest.sh
|
|
|
|
- name: Run openclaw install smoke test
|
|
working-directory: ./hindsight-integrations/openclaw
|
|
run: ./scripts/smoke-test.sh
|
|
|
|
test-coding-agents:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-coding-agents == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 15
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
cache: npm
|
|
cache-dependency-path: hindsight-integrations/coding-agents/package-lock.json
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/coding-agents
|
|
run: npm ci
|
|
|
|
- name: Typecheck
|
|
working-directory: ./hindsight-integrations/coding-agents
|
|
run: npx tsc --noEmit
|
|
|
|
- name: Unit tests
|
|
working-directory: ./hindsight-integrations/coding-agents
|
|
run: npm test
|
|
|
|
- name: Build
|
|
working-directory: ./hindsight-integrations/coding-agents
|
|
run: npm run build
|
|
|
|
test-claude-code-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-claude-code == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: '3.11'
|
|
|
|
- name: Install pytest
|
|
run: pip install pytest
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/claude-code
|
|
run: python -m pytest tests/ -v
|
|
|
|
test-cursor-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
github.event_name != 'pull_request_review' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-cursor == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: '3.11'
|
|
|
|
- name: Install pytest
|
|
run: pip install pytest
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/cursor
|
|
run: python -m pytest tests/ -v
|
|
|
|
test-zed-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
github.event_name != 'pull_request_review' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-zed == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/zed
|
|
# Config-only integration with no dependencies — it uses Node's built-in
|
|
# test runner. The runtime MCP bridge is `npx mcp-remote` (Node), so this
|
|
# integration requires only Node.js (no Python).
|
|
run: npm test
|
|
|
|
test-omo-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-omo == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: '3.11'
|
|
|
|
- name: Install pytest
|
|
run: pip install pytest
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/omo
|
|
run: python -m pytest tests/ -v
|
|
|
|
test-cline-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-cline == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build cline integration
|
|
working-directory: ./hindsight-integrations/cline
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/cline
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/cline
|
|
run: uv run pytest tests -v
|
|
|
|
test-github-copilot-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-github-copilot == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build github-copilot integration
|
|
working-directory: ./hindsight-integrations/github-copilot
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/github-copilot
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/github-copilot
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-codex-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-codex == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: '3.11'
|
|
|
|
- name: Install pytest
|
|
run: pip install pytest
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/codex
|
|
run: python -m pytest tests/ -v
|
|
|
|
test-cursor-cli-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-cursor-cli == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build cursor-cli integration
|
|
working-directory: ./hindsight-integrations/cursor-cli
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/cursor-cli
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/cursor-cli
|
|
run: uv run pytest tests -v
|
|
|
|
test-zcode-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-zcode == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build zcode integration
|
|
working-directory: ./hindsight-integrations/zcode
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/zcode
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/zcode
|
|
run: uv run pytest tests -v
|
|
|
|
test-agent-plugin-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-agent-plugin == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 15
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Validate Agent Plugin manifests
|
|
working-directory: ./hindsight-integrations/agent-plugin
|
|
run: python3 validate.py
|
|
|
|
build-ai-sdk-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-ai-sdk == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/ai-sdk
|
|
run: npm ci
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/ai-sdk
|
|
run: npm test
|
|
|
|
- name: Build
|
|
working-directory: ./hindsight-integrations/ai-sdk
|
|
run: npm run build
|
|
|
|
test-ai-sdk-integration-deno:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-ai-sdk == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Deno
|
|
uses: denoland/setup-deno@v2
|
|
with:
|
|
deno-version: v2.x
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/ai-sdk
|
|
run: npm ci
|
|
|
|
- name: Run tests (Deno)
|
|
working-directory: ./hindsight-integrations/ai-sdk
|
|
run: npm run test:deno
|
|
|
|
build-eliza-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-eliza == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/eliza
|
|
run: npm ci
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/eliza
|
|
run: npm test
|
|
|
|
- name: Build
|
|
working-directory: ./hindsight-integrations/eliza
|
|
run: npm run build
|
|
|
|
test-opencode-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-opencode == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/opencode
|
|
run: npm ci
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/opencode
|
|
run: npm test
|
|
|
|
- name: Build
|
|
working-directory: ./hindsight-integrations/opencode
|
|
run: npm run build
|
|
|
|
test-eve-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-eve == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '24'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/eve
|
|
run: npm ci
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/eve
|
|
run: npm test
|
|
|
|
- name: Build
|
|
working-directory: ./hindsight-integrations/eve
|
|
run: npm run build
|
|
|
|
test-n8n-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-n8n == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/n8n
|
|
run: npm install --no-fund --no-audit
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/n8n
|
|
run: npm test
|
|
|
|
- name: Build
|
|
working-directory: ./hindsight-integrations/n8n
|
|
run: npm run build
|
|
|
|
test-zapier-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-zapier == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/zapier
|
|
run: npm install --no-fund --no-audit
|
|
|
|
- name: Validate app definition
|
|
working-directory: ./hindsight-integrations/zapier
|
|
run: npm run validate
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/zapier
|
|
run: npm test
|
|
|
|
test-hindsight-agent-sdk:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.tools-agent-sdk == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install root workspace dependencies
|
|
run: npm ci
|
|
|
|
- name: Build hindsight-client (agent-sdk dep)
|
|
run: npm run build --workspace=hindsight-clients/typescript
|
|
|
|
- name: Run tests
|
|
run: npm test --workspace=hindsight-tools/hindsight-agent-sdk
|
|
|
|
- name: Build
|
|
run: npm run build --workspace=hindsight-tools/hindsight-agent-sdk
|
|
|
|
test-cloudflare-oauth-proxy-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-cloudflare-oauth-proxy == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/cloudflare-oauth-proxy
|
|
run: npm ci
|
|
|
|
- name: Typecheck
|
|
working-directory: ./hindsight-integrations/cloudflare-oauth-proxy
|
|
run: npm run typecheck
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/cloudflare-oauth-proxy
|
|
run: npm test
|
|
|
|
build-chat-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-chat == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/chat
|
|
run: npm ci
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/chat
|
|
run: npm test
|
|
|
|
- name: Build
|
|
working-directory: ./hindsight-integrations/chat
|
|
run: npm run build
|
|
|
|
test-paperclip-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-paperclip == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/paperclip
|
|
run: npm ci
|
|
|
|
- name: Build
|
|
working-directory: ./hindsight-integrations/paperclip
|
|
run: npm run build
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/paperclip
|
|
run: npm test
|
|
|
|
test-pipecat-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-pipecat == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build pipecat integration
|
|
working-directory: ./hindsight-integrations/pipecat
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/pipecat
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/pipecat
|
|
run: uv run pytest tests -v
|
|
|
|
test-google-adk-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-google-adk == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build google-adk integration
|
|
working-directory: ./hindsight-integrations/google-adk
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/google-adk
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/google-adk
|
|
run: uv run pytest tests -v
|
|
|
|
test-gemini-spark-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-gemini-spark == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/gemini-spark
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/gemini-spark
|
|
run: uv run pytest tests -v
|
|
|
|
test-roo-code-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
github.event_name != 'pull_request_review' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-roo-code == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build roo-code integration
|
|
working-directory: ./hindsight-integrations/roo-code
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/roo-code
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/roo-code
|
|
run: uv run pytest tests -v
|
|
|
|
build-control-plane:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.control-plane == 'true' ||
|
|
needs.detect-changes.outputs.clients-ts == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '24'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install SDK dependencies
|
|
run: npm ci --workspace=hindsight-clients/typescript
|
|
|
|
- name: Build SDK
|
|
run: npm run build --workspace=hindsight-clients/typescript
|
|
|
|
# Install control plane deps and fix hoisted lightningcss binary
|
|
# lightningcss gets hoisted to root node_modules, so we need to reinstall it there
|
|
- name: Install Control Plane dependencies
|
|
run: |
|
|
npm install --workspace=hindsight-control-plane
|
|
rm -rf node_modules/lightningcss node_modules/@tailwindcss
|
|
npm install lightningcss @tailwindcss/postcss @tailwindcss/node
|
|
|
|
- name: Test Control Plane
|
|
run: npm test --workspace=hindsight-control-plane
|
|
|
|
- name: Check i18n locale parity and hardcoded strings
|
|
run: npm run i18n:check --workspace=hindsight-control-plane
|
|
|
|
- name: Build Control Plane
|
|
run: npm run build --workspace=hindsight-control-plane
|
|
|
|
- name: Verify standalone build
|
|
run: |
|
|
test -f hindsight-control-plane/standalone/server.js || exit 1
|
|
test -d hindsight-control-plane/standalone/node_modules || exit 1
|
|
node hindsight-control-plane/bin/cli.js --help
|
|
|
|
- name: Smoke test - verify server starts
|
|
run: |
|
|
cd hindsight-control-plane
|
|
node bin/cli.js --port 9999 &
|
|
SERVER_PID=$!
|
|
sleep 5
|
|
if curl -sf http://localhost:9999 > /dev/null 2>&1; then
|
|
echo "Server started successfully"
|
|
kill $SERVER_PID 2>/dev/null || true
|
|
exit 0
|
|
else
|
|
echo "Server failed to respond"
|
|
kill $SERVER_PID 2>/dev/null || true
|
|
exit 1
|
|
fi
|
|
|
|
build-docs:
|
|
needs: [detect-changes]
|
|
# Keep the production docs build as an unconditional PR check. OpenAPI
|
|
# generation used to build the site again inside verify-generated-files;
|
|
# running the existing job for every PR preserves that coverage without
|
|
# serializing two full Docusaurus builds in the generated-files check.
|
|
# The one exception is a PR that touches only hindsight-system-tests/,
|
|
# which the docs site cannot read from — see the filter of the same name.
|
|
if: needs.detect-changes.outputs.outside-system-tests == 'true'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
fetch-depth: 0 # fetch tags so check-released-integrations can see them
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '20'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
# Fail fast before the (slow) build: every integrations.json entry must have a
|
|
# doc page, and every released integration tag must be in integrations.json.
|
|
# Needs no npm install (pure Node) and uses the tags fetched above.
|
|
- name: Check integrations (single source of truth)
|
|
run: node hindsight-docs/scripts/check-integrations.mjs
|
|
|
|
- name: Install dependencies
|
|
run: npm ci --workspace=hindsight-docs
|
|
|
|
- name: Build docs
|
|
run: npm run build --workspace=hindsight-docs
|
|
|
|
test-rust-cli:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.cli == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install Rust
|
|
uses: dtolnay/rust-toolchain@stable
|
|
|
|
- name: Cache cargo
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: |
|
|
~/.cargo/registry
|
|
~/.cargo/git
|
|
hindsight-cli/target
|
|
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
|
|
|
|
- name: Install cargo-about
|
|
uses: taiki-e/install-action@v2
|
|
with:
|
|
tool: cargo-about@0.9.1
|
|
|
|
- name: Verify cargo-about
|
|
run: cargo about --version
|
|
|
|
# cargo-about resolves the dependency graph for every target platform, so it
|
|
# needs crates this host never builds (e.g. the Android-only
|
|
# android_system_properties). Populate the registry first: `cargo fetch`
|
|
# without --target downloads for all targets, and Cargo.lock is not checked
|
|
# in, so nothing is cached from a previous step.
|
|
- name: Fetch crate sources for the license scan
|
|
working-directory: hindsight-cli
|
|
run: cargo fetch
|
|
|
|
- name: Generate and verify license manifest
|
|
working-directory: hindsight-cli
|
|
run: |
|
|
cargo about generate --offline --manifest-path Cargo.toml --config about.toml about.hbs --output-file /tmp/THIRD_PARTY_LICENSES.txt
|
|
test -s /tmp/THIRD_PARTY_LICENSES.txt
|
|
grep -Fq "THIRD-PARTY SOFTWARE LICENSES" /tmp/THIRD_PARTY_LICENSES.txt
|
|
|
|
- name: Run unit tests
|
|
working-directory: hindsight-cli
|
|
run: cargo test
|
|
|
|
- name: Build CLI
|
|
working-directory: hindsight-cli
|
|
run: cargo build --release
|
|
|
|
- name: Upload CLI artifact
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: hindsight-cli
|
|
path: hindsight-cli/target/release/hindsight
|
|
retention-days: 1
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Create .env file
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Run CLI smoke test
|
|
run: |
|
|
HINDSIGHT_CLI=hindsight-cli/target/release/hindsight ./hindsight-cli/smoke-test.sh
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
lint-helm-chart:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.helm == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install Helm
|
|
uses: azure/setup-helm@v5
|
|
with:
|
|
version: 'latest'
|
|
|
|
- name: Lint Helm chart
|
|
run: helm lint helm/hindsight
|
|
|
|
test-standalone-start-script:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.docker == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Run standalone start script tests
|
|
run: bash docker/standalone/test-start-all.sh
|
|
|
|
build-docker-images:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.docker == 'true' ||
|
|
needs.detect-changes.outputs.control-plane == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
name: Build Docker (${{ matrix.name }})
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
strategy:
|
|
matrix:
|
|
# `smoke` decides which legs are loaded into the daemon and actually
|
|
# started. The full variants are skipped to save runner disk; cp-only
|
|
# is smoke tested despite being a full variant because the alpine
|
|
# image is small and it is the only leg that exercises the alpine Node
|
|
# runtime at all (it needs no LLM credentials - see NEEDS_LLM in
|
|
# docker/test-image.sh).
|
|
include:
|
|
- target: api-only
|
|
name: api
|
|
variant: full
|
|
smoke: false
|
|
build_args: ""
|
|
- target: api-only
|
|
name: api-slim
|
|
variant: slim
|
|
smoke: true
|
|
build_args: |
|
|
INCLUDE_LOCAL_MODELS=false
|
|
PRELOAD_ML_MODELS=false
|
|
- target: cp-only
|
|
name: control-plane
|
|
variant: full
|
|
smoke: true
|
|
build_args: ""
|
|
- target: standalone
|
|
name: standalone
|
|
variant: full
|
|
smoke: false
|
|
build_args: ""
|
|
- target: standalone
|
|
name: standalone-slim
|
|
variant: slim
|
|
smoke: true
|
|
build_args: |
|
|
INCLUDE_LOCAL_MODELS=false
|
|
PRELOAD_ML_MODELS=false
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Free Disk Space
|
|
uses: jlumbroso/free-disk-space@main
|
|
with:
|
|
tool-cache: true
|
|
android: true
|
|
dotnet: true
|
|
haskell: true
|
|
large-packages: true
|
|
docker-images: true
|
|
swap-storage: true
|
|
|
|
- name: Set up Docker Buildx
|
|
uses: docker/setup-buildx-action@v4
|
|
|
|
- name: Build ${{ matrix.name }} image (${{ matrix.variant }})
|
|
uses: docker/build-push-action@v7
|
|
with:
|
|
context: .
|
|
file: docker/standalone/Dockerfile
|
|
target: ${{ matrix.target }}
|
|
build-args: ${{ matrix.build_args }}
|
|
push: false
|
|
load: ${{ matrix.smoke }}
|
|
tags: hindsight-${{ matrix.name }}:test
|
|
# Removed GitHub Actions cache (type=gha) - it frequently returns 502 errors
|
|
# causing buildx to fail with "failed to parse error response 502"
|
|
# Build will be slower but more reliable
|
|
|
|
# Slim variants require external embedding providers; cp-only needs no
|
|
# credentials at all, so the credential step stays gated on the variant
|
|
# while the smoke test itself is gated on `smoke`.
|
|
- name: Setup GCP credentials for smoke test
|
|
if: matrix.variant == 'slim' && matrix.smoke
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Smoke test - verify container starts
|
|
if: matrix.smoke
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID: ${{ env.HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID }}
|
|
HINDSIGHT_API_EMBEDDINGS_PROVIDER: cohere
|
|
HINDSIGHT_API_RERANKER_PROVIDER: cohere
|
|
HINDSIGHT_API_COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
|
|
run: ./docker/test-image.sh "hindsight-${{ matrix.name }}:test" "${{ matrix.target }}"
|
|
|
|
test-api:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
# Sharded with pytest-split. Splits the ~19-min pytest run across 3
|
|
# parallel jobs, cutting critical-path wall time to ~7-8 min/shard. The
|
|
# .venv cache lets shards 2+ skip the ~3-min `uv sync` once shard 1
|
|
# populates the key — same key on re-runs hits cache on all three.
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
shard: [1, 2, 3]
|
|
name: test-api (${{ matrix.shard }}/3)
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
|
|
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
|
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
|
|
HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
|
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
|
AWS_REGION_NAME: ${{ secrets.AWS_REGION_NAME }}
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
# Prefer CPU-only PyTorch in CI (but keep PyPI for everything else)
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Cache hindsight-api-slim/.venv
|
|
# Keyed on the workspace lockfile, the API package's pyproject, and the
|
|
# pinned Python version — the only inputs that change the resolved env.
|
|
# `uv sync --frozen` still runs after restore but is a near-instant link
|
|
# check when the venv already matches the lock.
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: hindsight-api-slim/.venv
|
|
key: ${{ runner.os }}-venv-test-api-${{ hashFiles('uv.lock', 'hindsight-api-slim/pyproject.toml', '.python-version') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-venv-test-api-
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Run tests (shard ${{ matrix.shard }}/3)
|
|
working-directory: ./hindsight-api-slim
|
|
# `--with pytest-split` adds the plugin ad-hoc — no uv.lock churn.
|
|
# pytest-split filters at collection (before xdist takes over), so it
|
|
# composes cleanly with the `-n 8 --dist loadgroup` baked into addopts.
|
|
run: uv run --with pytest-split pytest tests -v -m "not hs_llm_mat and not hs_llm_core" --splits 3 --group ${{ matrix.shard }}
|
|
|
|
test-api-free-threaded:
|
|
needs: [detect-changes]
|
|
# Deliberately NOT gated on `has_secrets`: this job runs the mock LLM and a
|
|
# deterministic embedder, so it needs no provider credentials and therefore also
|
|
# runs on fork PRs — which skip every secret-gated test-api job entirely.
|
|
if: >-
|
|
github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 40
|
|
name: test-api (free-threaded 3.14)
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: mock
|
|
# Not a repository secret — Actions provides it to every run, forks included —
|
|
# so this keeps the job runnable on fork PRs. The github-copilot provider reads
|
|
# it to avoid looking for CLI account metadata that no runner has.
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
# strict is the default on a free-threaded build; set explicitly so the intent
|
|
# of this job is visible in the workflow rather than only in the code.
|
|
HINDSIGHT_API_FREE_THREADING: strict
|
|
# SQLAlchemy's Cython extensions do not declare free-threading support. They
|
|
# currently have no cp314t wheel so the pure-Python fallback is automatic, but
|
|
# pin it: the day a cp314t wheel ships, this job would otherwise start running
|
|
# with the GIL quietly re-enabled.
|
|
DISABLE_SQLALCHEMY_CEXT_RUNTIME: "1"
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up free-threaded Python 3.14
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: '3.14t'
|
|
|
|
- name: Create the virtualenv
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv venv --python "$(which python)" .venv-ft
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
# `uv pip install` rather than `uv sync --frozen`: uv.lock pins the resolution
|
|
# for the default interpreter, and this one resolves different wheels.
|
|
#
|
|
# No --override is needed any more: quicktok left in #4022 and orjson in #4040,
|
|
# and everything remaining publishes cp3XXt wheels.
|
|
#
|
|
# local-ml is deliberately excluded. Importing sentence_transformers re-enables
|
|
# the GIL, so a process that installs it cannot stay free-threaded; the tests
|
|
# that need local models skip themselves (see tests/conftest.py).
|
|
env:
|
|
VIRTUAL_ENV: .venv-ft
|
|
run: uv pip install -e ".[embedded-db]" --group dev
|
|
|
|
|
|
- name: Assert the interpreter is genuinely free-threaded
|
|
working-directory: ./hindsight-api-slim
|
|
# Fails the job before a single test runs if the build is not free-threaded or a
|
|
# dependency has already taken the GIL back, so that shows up as its own red step
|
|
# rather than as a confusing mass of downstream failures.
|
|
run: |
|
|
./.venv-ft/bin/python -c "
|
|
import sys, sysconfig
|
|
assert sysconfig.get_config_var('Py_GIL_DISABLED'), 'not a free-threaded build'
|
|
assert not sys._is_gil_enabled(), 'the GIL is already enabled at startup'
|
|
import hindsight_api.api.http
|
|
assert not sys._is_gil_enabled(), 'importing the API re-enabled the GIL'
|
|
print('free-threaded:', sys.version)
|
|
"
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-api-slim
|
|
# Deliberately NOT run with the GIL warning promoted to an error. The suite
|
|
# imports LiteLLM on purpose to test that provider, and LiteLLM pulls in
|
|
# fastuuid, which has no free-threaded build — so a suite-wide filter fails
|
|
# ~50 tests for doing exactly what they are meant to do.
|
|
#
|
|
# What matters is that the SERVER never takes the GIL, and that is asserted
|
|
# where it belongs: the step above imports the whole API under the fatal
|
|
# filter, tests/test_free_threading.py re-checks it in a subprocess, and the
|
|
# image build and smoke test assert it again.
|
|
run: |
|
|
./.venv-ft/bin/python -m pytest tests -v \
|
|
-m "not hs_llm_mat and not hs_llm_core"
|
|
|
|
build-freethreaded-image:
|
|
needs: [detect-changes]
|
|
# Gated on `docker` as well as `core`: this is the only job that builds
|
|
# Dockerfile.freethreaded, and it used to hang off the free-threaded pytest job,
|
|
# which does not run for a docker-only change. Two Dockerfile.freethreaded PRs
|
|
# merged without CI ever building the image before this was split out.
|
|
#
|
|
# Deliberately NOT gated on `has_secrets`, for the same reason as the pytest job
|
|
# above: the smoke test runs a mock LLM and a stub embedder, so it needs no
|
|
# provider credentials and therefore also covers fork PRs, which skip
|
|
# build-docker-images entirely.
|
|
if: >-
|
|
github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.docker == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
name: build-freethreaded-image
|
|
|
|
services:
|
|
# For the smoke test; the image runs against this rather than embedded pg0.
|
|
postgres:
|
|
image: pgvector/pgvector:pg17
|
|
env:
|
|
POSTGRES_USER: hindsight
|
|
POSTGRES_PASSWORD: hindsight
|
|
POSTGRES_DB: hindsight
|
|
ports:
|
|
- 5432:5432
|
|
options: >-
|
|
--health-cmd "pg_isready -U hindsight"
|
|
--health-interval 5s
|
|
--health-timeout 5s
|
|
--health-retries 10
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Build the free-threaded image
|
|
# Its own Dockerfile, not a target of the standalone one: different interpreter,
|
|
# different resolution, no control plane, no local ML. The build itself asserts
|
|
# the GIL is off after importing the whole API, so a mis-tagged image fails here.
|
|
run: docker build -f docker/standalone/Dockerfile.freethreaded -t hindsight-api:py3.14t .
|
|
|
|
- name: Smoke test the free-threaded image
|
|
# Starts the image against the Postgres service and a stub embedder, exercises
|
|
# retain and recall, and asserts the SERVER process still has the GIL disabled —
|
|
# the failure this tag exists to prevent is silent, so it has to be asserted.
|
|
run: ./docker/freethreaded-smoke.sh hindsight-api:py3.14t "postgresql://hindsight:hindsight@127.0.0.1:5432/hindsight"
|
|
|
|
test-api-llm-core:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
name: Core LLM tests
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
|
|
# The real-LLM delta/retraction/cache evals gate on this flag *and* a key. The
|
|
# key was already set here, so they sat skipped and the mental-model delta path —
|
|
# the one that hands a provider a schema it has to accept — had no end-to-end
|
|
# coverage anywhere. Found while landing the structured-output change (#3937).
|
|
HINDSIGHT_RUN_GEMINI_EVALS: "1"
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Run core LLM tests
|
|
working-directory: ./hindsight-api-slim
|
|
# --reruns, and ONLY on this job. Every test here asserts what a model chose to
|
|
# do — a judge verdict, whether the agent called done() rather than being forced,
|
|
# whether a summary kept a detail — so a single run's failure does not separate a
|
|
# regression from the model landing differently. It shows: this job has failed on
|
|
# a *different* test in most recent runs (5 distinct ones across 2 runs of #3982
|
|
# alone), which makes a red here carry no signal at all. Two reruns keep a real
|
|
# regression red — it reproduces — while a coin-flip settles.
|
|
#
|
|
# Deliberately NOT applied to the deterministic shards (test-api): there a rerun
|
|
# would hide exactly the races those tests exist to catch.
|
|
run: uv run pytest tests -v -m "hs_llm_core" --timeout 600 --reruns 2 --reruns-delay 5
|
|
|
|
test-api-llm-acceptance:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
((github.event_name == 'pull_request_review' && github.event.review.state == 'approved') ||
|
|
github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- provider: vertexai
|
|
model: google/gemini-2.5-flash-lite
|
|
- provider: gemini
|
|
model: gemini-2.5-flash-lite
|
|
api_key_secret: GEMINI_API_KEY
|
|
- provider: openai
|
|
model: gpt-4.1-nano
|
|
api_key_secret: OPENAI_API_KEY
|
|
- provider: groq
|
|
model: openai/gpt-oss-20b
|
|
api_key_secret: GROQ_API_KEY
|
|
- provider: bedrock
|
|
model: us.amazon.nova-2-lite-v1:0
|
|
- provider: litellmrouter
|
|
model: gpt-4.1-nano
|
|
# Single-deployment chain over OpenAI — verifies the Router-backed
|
|
# call path works end-to-end. Built from secrets in the step below.
|
|
name: LLM acceptance (${{ matrix.provider }}/${{ matrix.model }})
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: ${{ matrix.provider }}
|
|
HINDSIGHT_API_LLM_MODEL: ${{ matrix.model }}
|
|
HINDSIGHT_API_LLM_API_KEY: ${{ matrix.api_key_secret && secrets[matrix.api_key_secret] || '' }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
|
|
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
|
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
|
|
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
|
AWS_REGION_NAME: ${{ secrets.AWS_REGION_NAME }}
|
|
HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
"
|
|
|
|
- name: Build litellmrouter config
|
|
if: matrix.provider == 'litellmrouter'
|
|
env:
|
|
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
|
ROUTER_MODEL: ${{ matrix.model }}
|
|
run: |
|
|
cfg=$(jq -nc \
|
|
--arg model "openai/$ROUTER_MODEL" \
|
|
--arg key "$OPENAI_API_KEY" \
|
|
'{model_list: [{model_name: "default", litellm_params: {model: $model, api_key: $key}}]}')
|
|
echo "::add-mask::$cfg"
|
|
echo "HINDSIGHT_API_LLM_LITELLMROUTER_CONFIG=$cfg" >> "$GITHUB_ENV"
|
|
|
|
- name: Run LLM acceptance tests
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv run pytest tests -v -m "hs_llm_mat" --timeout 600
|
|
|
|
test-api-oracle:
|
|
needs: [detect-changes]
|
|
# Gated behind the "oracle-tests" PR label so it doesn't run by default.
|
|
# Add the label to any PR that needs Oracle validation.
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
contains(github.event.pull_request.labels.*.name, 'oracle-tests') &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
|
|
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
|
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
|
|
HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
|
HINDSIGHT_API_DATABASE_BACKEND: oracle
|
|
ORACLE_TEST_DSN: oracle+oracledb://hindsight_test:hindsight_test@localhost:1521/FREEPDB1
|
|
|
|
services:
|
|
oracle:
|
|
image: container-registry.oracle.com/database/free:latest
|
|
env:
|
|
ORACLE_PWD: oracle
|
|
ports:
|
|
- 1521:1521
|
|
options: >-
|
|
--health-cmd "echo 'SELECT 1 FROM DUAL;' | sqlplus -s system/oracle@localhost:1521/FREEPDB1 || exit 1"
|
|
--health-interval 30s
|
|
--health-timeout 10s
|
|
--health-retries 10
|
|
--health-start-period 120s
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
# The Oracle 23ai `free` service image is large; combined with the Python ML
|
|
# deps (torch) it exhausts the runner's ~14 GB root disk, so uv fails to
|
|
# extract a wheel with "No space left on device (os error 28)".
|
|
# Only the cheap, high-yield reclaims are enabled — the Android/.NET/Haskell
|
|
# dirs plus swap are a few `rm -rf`s worth ~16-21 GB, which is ample headroom:
|
|
# - large-packages runs apt-get remove and costs minutes for little gain;
|
|
# - tool-cache would delete the preinstalled Python that actions/setup-python
|
|
# then has to re-download, making the job slower, not faster;
|
|
# - docker-images is pointless here (the Oracle service container is already
|
|
# running, so its image is in use and cannot be pruned anyway).
|
|
- name: Free Disk Space
|
|
uses: jlumbroso/free-disk-space@main
|
|
with:
|
|
tool-cache: false
|
|
android: true
|
|
dotnet: true
|
|
haskell: true
|
|
large-packages: false
|
|
docker-images: false
|
|
swap-storage: true
|
|
|
|
- name: Setup Oracle test user
|
|
# The SYSTEM tablespace uses manual segment space management which
|
|
# doesn't support VECTOR types. Create an ASSM tablespace and a
|
|
# dedicated test user so VECTOR columns work correctly.
|
|
run: |
|
|
pip install oracledb
|
|
python3 -c "
|
|
import oracledb
|
|
conn = oracledb.connect(user='system', password='oracle', dsn='localhost:1521/FREEPDB1')
|
|
cursor = conn.cursor()
|
|
cursor.execute(\"\"\"
|
|
CREATE BIGFILE TABLESPACE hindsight_ts
|
|
DATAFILE 'hindsight_ts.dbf' SIZE 2G AUTOEXTEND ON NEXT 500M MAXSIZE UNLIMITED
|
|
EXTENT MANAGEMENT LOCAL
|
|
SEGMENT SPACE MANAGEMENT AUTO
|
|
\"\"\")
|
|
cursor.execute(\"\"\"
|
|
CREATE USER hindsight_test IDENTIFIED BY hindsight_test
|
|
DEFAULT TABLESPACE hindsight_ts
|
|
TEMPORARY TABLESPACE temp
|
|
QUOTA UNLIMITED ON hindsight_ts
|
|
\"\"\")
|
|
cursor.execute('GRANT CONNECT, RESOURCE, CREATE TABLE, CREATE SEQUENCE, CREATE VIEW, CREATE PROCEDURE TO hindsight_test')
|
|
cursor.execute('GRANT CTXAPP TO hindsight_test')
|
|
conn.commit()
|
|
conn.close()
|
|
print('Oracle test user created successfully')
|
|
"
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Run Oracle tests
|
|
working-directory: ./hindsight-api-slim
|
|
# -n0: run sequentially to avoid ORA-00060 deadlocks from concurrent
|
|
# test transactions against the same Oracle Free container.
|
|
run: uv run pytest tests -v -m oracle -n0
|
|
|
|
test-python-client:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.clients-python == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
# Prefer CPU-only PyTorch in CI (but keep PyPI for everything else)
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Build Python client
|
|
working-directory: ./hindsight-clients/python
|
|
run: uv build
|
|
|
|
- name: Install client test dependencies
|
|
working-directory: ./hindsight-clients/python
|
|
run: uv sync --frozen --extra test --index-strategy unsafe-best-match
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Create .env file
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Run Python client tests
|
|
working-directory: ./hindsight-clients/python
|
|
run: uv run pytest tests -v
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
test-typescript-client:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.clients-ts == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
# Prefer CPU-only PyTorch in CI (but keep PyPI for everything else)
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '20'
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Install TypeScript client dependencies
|
|
working-directory: ./hindsight-clients/typescript
|
|
run: npm ci
|
|
|
|
- name: Build TypeScript client
|
|
working-directory: ./hindsight-clients/typescript
|
|
run: npm run build
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Create .env file
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Run TypeScript client tests
|
|
working-directory: ./hindsight-clients/typescript
|
|
run: npm test
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
# -----------------------------------------------------------------
|
|
# Client integration tests against Oracle 23ai.
|
|
#
|
|
# Same shape as test-python-client / test-typescript-client but with the
|
|
# API server pointed at an Oracle 23ai service container. Catches API
|
|
# changes that work on PG but break on Oracle (or vice versa) before they
|
|
# ship — the unit tests use the abstraction layer, but only the client
|
|
# tests exercise full HTTP round-trips with real serialized payloads.
|
|
#
|
|
# Runs on every API/clients change (no label needed). The "oracle-tests"
|
|
# label is reserved for test-api-oracle — the full unit suite is too
|
|
# heavy to run on every PR.
|
|
# -----------------------------------------------------------------
|
|
test-python-client-oracle:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.clients-python == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
HINDSIGHT_API_DATABASE_BACKEND: oracle
|
|
HINDSIGHT_API_DATABASE_URL: oracle+oracledb://hindsight_test:hindsight_test@localhost:1521/FREEPDB1
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
services:
|
|
oracle:
|
|
image: container-registry.oracle.com/database/free:latest
|
|
env:
|
|
ORACLE_PWD: oracle
|
|
ports:
|
|
- 1521:1521
|
|
options: >-
|
|
--health-cmd "echo 'SELECT 1 FROM DUAL;' | sqlplus -s system/oracle@localhost:1521/FREEPDB1 || exit 1"
|
|
--health-interval 30s
|
|
--health-timeout 10s
|
|
--health-retries 10
|
|
--health-start-period 120s
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
# The Oracle 23ai `free` service image is large; combined with the Python ML
|
|
# deps (torch) it exhausts the runner's ~14 GB root disk, so uv fails to
|
|
# extract a wheel with "No space left on device (os error 28)".
|
|
# Only the cheap, high-yield reclaims are enabled — the Android/.NET/Haskell
|
|
# dirs plus swap are a few `rm -rf`s worth ~16-21 GB, which is ample headroom:
|
|
# - large-packages runs apt-get remove and costs minutes for little gain;
|
|
# - tool-cache would delete the preinstalled Python that actions/setup-python
|
|
# then has to re-download, making the job slower, not faster;
|
|
# - docker-images is pointless here (the Oracle service container is already
|
|
# running, so its image is in use and cannot be pruned anyway).
|
|
- name: Free Disk Space
|
|
uses: jlumbroso/free-disk-space@main
|
|
with:
|
|
tool-cache: false
|
|
android: true
|
|
dotnet: true
|
|
haskell: true
|
|
large-packages: false
|
|
docker-images: false
|
|
swap-storage: true
|
|
|
|
- name: Setup Oracle test user
|
|
run: |
|
|
pip install oracledb
|
|
python3 -c "
|
|
import oracledb
|
|
conn = oracledb.connect(user='system', password='oracle', dsn='localhost:1521/FREEPDB1')
|
|
cursor = conn.cursor()
|
|
cursor.execute(\"\"\"
|
|
CREATE BIGFILE TABLESPACE hindsight_ts
|
|
DATAFILE 'hindsight_ts.dbf' SIZE 2G AUTOEXTEND ON NEXT 500M MAXSIZE UNLIMITED
|
|
EXTENT MANAGEMENT LOCAL
|
|
SEGMENT SPACE MANAGEMENT AUTO
|
|
\"\"\")
|
|
cursor.execute(\"\"\"
|
|
CREATE USER hindsight_test IDENTIFIED BY hindsight_test
|
|
DEFAULT TABLESPACE hindsight_ts
|
|
TEMPORARY TABLESPACE temp
|
|
QUOTA UNLIMITED ON hindsight_ts
|
|
\"\"\")
|
|
cursor.execute('GRANT CONNECT, RESOURCE, CREATE TABLE, CREATE SEQUENCE, CREATE VIEW, CREATE PROCEDURE TO hindsight_test')
|
|
cursor.execute('GRANT CTXAPP TO hindsight_test')
|
|
conn.commit()
|
|
conn.close()
|
|
print('Oracle test user created successfully')
|
|
"
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Build Python client
|
|
working-directory: ./hindsight-clients/python
|
|
run: uv build
|
|
|
|
- name: Install client test dependencies
|
|
working-directory: ./hindsight-clients/python
|
|
run: uv sync --frozen --extra test --index-strategy unsafe-best-match
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Create .env file
|
|
# The Oracle DB env vars need to be in .env for start-api.sh to pick them
|
|
# up — it source's the file before exec'ing the API.
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
HINDSIGHT_API_DATABASE_BACKEND=${{ env.HINDSIGHT_API_DATABASE_BACKEND }}
|
|
HINDSIGHT_API_DATABASE_URL=${{ env.HINDSIGHT_API_DATABASE_URL }}
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Run Python client tests (Oracle, sequential)
|
|
working-directory: ./hindsight-clients/python
|
|
# -n0 overrides the "-n auto" default in pyproject.toml. Oracle's
|
|
# row-level locking deadlocks (ORA-00060) under concurrent retain
|
|
# cleanup — same reason test-api-oracle runs sequentially.
|
|
run: uv run pytest tests -v -n0
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
test-typescript-client-oracle:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.clients-ts == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
HINDSIGHT_API_DATABASE_BACKEND: oracle
|
|
HINDSIGHT_API_DATABASE_URL: oracle+oracledb://hindsight_test:hindsight_test@localhost:1521/FREEPDB1
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
services:
|
|
oracle:
|
|
image: container-registry.oracle.com/database/free:latest
|
|
env:
|
|
ORACLE_PWD: oracle
|
|
ports:
|
|
- 1521:1521
|
|
options: >-
|
|
--health-cmd "echo 'SELECT 1 FROM DUAL;' | sqlplus -s system/oracle@localhost:1521/FREEPDB1 || exit 1"
|
|
--health-interval 30s
|
|
--health-timeout 10s
|
|
--health-retries 10
|
|
--health-start-period 120s
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
# The Oracle 23ai `free` service image is large; combined with the Python ML
|
|
# deps (torch) it exhausts the runner's ~14 GB root disk, so uv fails to
|
|
# extract a wheel with "No space left on device (os error 28)".
|
|
# Only the cheap, high-yield reclaims are enabled — the Android/.NET/Haskell
|
|
# dirs plus swap are a few `rm -rf`s worth ~16-21 GB, which is ample headroom:
|
|
# - large-packages runs apt-get remove and costs minutes for little gain;
|
|
# - tool-cache would delete the preinstalled Python that actions/setup-python
|
|
# then has to re-download, making the job slower, not faster;
|
|
# - docker-images is pointless here (the Oracle service container is already
|
|
# running, so its image is in use and cannot be pruned anyway).
|
|
- name: Free Disk Space
|
|
uses: jlumbroso/free-disk-space@main
|
|
with:
|
|
tool-cache: false
|
|
android: true
|
|
dotnet: true
|
|
haskell: true
|
|
large-packages: false
|
|
docker-images: false
|
|
swap-storage: true
|
|
|
|
- name: Setup Oracle test user
|
|
run: |
|
|
pip install oracledb
|
|
python3 -c "
|
|
import oracledb
|
|
conn = oracledb.connect(user='system', password='oracle', dsn='localhost:1521/FREEPDB1')
|
|
cursor = conn.cursor()
|
|
cursor.execute(\"\"\"
|
|
CREATE BIGFILE TABLESPACE hindsight_ts
|
|
DATAFILE 'hindsight_ts.dbf' SIZE 2G AUTOEXTEND ON NEXT 500M MAXSIZE UNLIMITED
|
|
EXTENT MANAGEMENT LOCAL
|
|
SEGMENT SPACE MANAGEMENT AUTO
|
|
\"\"\")
|
|
cursor.execute(\"\"\"
|
|
CREATE USER hindsight_test IDENTIFIED BY hindsight_test
|
|
DEFAULT TABLESPACE hindsight_ts
|
|
TEMPORARY TABLESPACE temp
|
|
QUOTA UNLIMITED ON hindsight_ts
|
|
\"\"\")
|
|
cursor.execute('GRANT CONNECT, RESOURCE, CREATE TABLE, CREATE SEQUENCE, CREATE VIEW, CREATE PROCEDURE TO hindsight_test')
|
|
cursor.execute('GRANT CTXAPP TO hindsight_test')
|
|
conn.commit()
|
|
conn.close()
|
|
print('Oracle test user created successfully')
|
|
"
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '20'
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Install TypeScript client dependencies
|
|
working-directory: ./hindsight-clients/typescript
|
|
run: npm ci
|
|
|
|
- name: Build TypeScript client
|
|
working-directory: ./hindsight-clients/typescript
|
|
run: npm run build
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Create .env file
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
HINDSIGHT_API_DATABASE_BACKEND=${{ env.HINDSIGHT_API_DATABASE_BACKEND }}
|
|
HINDSIGHT_API_DATABASE_URL=${{ env.HINDSIGHT_API_DATABASE_URL }}
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Run TypeScript client tests
|
|
working-directory: ./hindsight-clients/typescript
|
|
run: npm test
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
test-typescript-client-deno:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.clients-ts == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '20'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Set up Deno
|
|
uses: denoland/setup-deno@v2
|
|
with:
|
|
deno-version: v2.x
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Install TypeScript client dependencies
|
|
working-directory: ./hindsight-clients/typescript
|
|
run: npm ci
|
|
|
|
- name: Build TypeScript client
|
|
working-directory: ./hindsight-clients/typescript
|
|
run: npm run build
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Create .env file
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Run TypeScript client tests (Deno)
|
|
working-directory: ./hindsight-clients/typescript
|
|
run: npm run test:deno
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
build-rust-cli-arm64:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.cli == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-24.04-arm
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install Rust
|
|
uses: dtolnay/rust-toolchain@stable
|
|
with:
|
|
targets: aarch64-unknown-linux-gnu
|
|
|
|
- name: Cache cargo
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: |
|
|
~/.cargo/registry
|
|
~/.cargo/git
|
|
hindsight-cli/target
|
|
key: linux-arm64-cargo-${{ hashFiles('**/Cargo.lock') }}
|
|
|
|
- name: Build CLI
|
|
working-directory: hindsight-cli
|
|
run: cargo build --release --target aarch64-unknown-linux-gnu
|
|
|
|
test-rust-client:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.clients-rust == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
# Prefer CPU-only PyTorch in CI (but keep PyPI for everything else)
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install Rust
|
|
uses: dtolnay/rust-toolchain@stable
|
|
|
|
- name: Cache cargo
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: |
|
|
~/.cargo/registry
|
|
~/.cargo/git
|
|
hindsight-clients/rust/target
|
|
key: ${{ runner.os }}-cargo-client-${{ hashFiles('hindsight-clients/rust/Cargo.lock') }}
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Create .env file
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Run Rust client tests
|
|
working-directory: ./hindsight-clients/rust
|
|
run: cargo test --lib
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
test-go-client:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.clients-go == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
# Prefer CPU-only PyTorch in CI (but keep PyPI for everything else)
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version: '1.23'
|
|
cache-dependency-path: hindsight-clients/go/go.sum
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Create .env file
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Build Go client
|
|
working-directory: ./hindsight-clients/go
|
|
run: go build ./...
|
|
|
|
- name: Run Go client tests
|
|
working-directory: ./hindsight-clients/go
|
|
run: go test -v -tags=integration
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
test-openclaw-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.integrations-openclaw == 'true' ||
|
|
needs.detect-changes.outputs.embed == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
HINDSIGHT_EMBED_PACKAGE_PATH: ${{ github.workspace }}/hindsight-embed
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Install embed dependencies
|
|
working-directory: ./hindsight-embed
|
|
run: uv sync --frozen --index-strategy unsafe-best-match
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
# openclaw depends on @vectorize-io/hindsight-client and
|
|
# @vectorize-io/hindsight-all via `file:` — their `dist/` directories are
|
|
# gitignored and must be built before openclaw's npm ci copies them.
|
|
- name: Install root workspace dependencies
|
|
run: npm ci
|
|
|
|
- name: Build hindsight-client (openclaw dep)
|
|
run: npm run build --workspace=hindsight-clients/typescript
|
|
|
|
- name: Build hindsight-all-npm (openclaw dep)
|
|
run: npm run build --workspace=hindsight-all-npm
|
|
|
|
- name: Build hindsight-agent-sdk (openclaw dep)
|
|
run: npm run build --workspace=hindsight-tools/hindsight-agent-sdk
|
|
|
|
- name: Install openclaw integration dependencies
|
|
working-directory: ./hindsight-integrations/openclaw
|
|
run: npm ci
|
|
|
|
- name: Create .env file
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Run openclaw integration tests
|
|
working-directory: ./hindsight-integrations/openclaw
|
|
run: npm run test:integration
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
test-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.integration-tests == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build API
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv build
|
|
|
|
- name: Install integration test dependencies
|
|
working-directory: ./hindsight-integration-tests
|
|
run: uv sync --frozen
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Create .env file
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Run integration tests
|
|
working-directory: ./hindsight-integration-tests
|
|
run: uv run pytest tests/ -v
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
test-system:
|
|
needs: [detect-changes]
|
|
# Deliberately NOT gated on `has_secrets`. The whole suite runs against a stub
|
|
# that implements the OpenAI chat-completions/embeddings APIs and a
|
|
# Cohere-compatible rerank endpoint, so it needs no provider credentials — and
|
|
# therefore runs on fork PRs, which skip every secret-gated test-api job.
|
|
if: >-
|
|
github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.system-tests == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
name: test-system (blackbox)
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install API dependencies
|
|
working-directory: ./hindsight-api-slim
|
|
# `embedded-db` (pg0) is the only extra needed: with embeddings and
|
|
# reranking both pointed at the stub, nothing loads sentence-transformers,
|
|
# so this job skips torch and the HuggingFace model cache entirely.
|
|
run: uv sync --frozen --extra embedded-db --index-strategy unsafe-best-match
|
|
|
|
- name: Install system-test dependencies
|
|
working-directory: ./hindsight-system-tests
|
|
run: uv sync --frozen
|
|
|
|
- name: Run system tests
|
|
working-directory: ./hindsight-system-tests
|
|
run: uv run pytest tests -v
|
|
|
|
test-ag2-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-ag2 == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build ag2 integration
|
|
working-directory: ./hindsight-integrations/ag2
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/ag2
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/ag2
|
|
run: uv run pytest tests -v
|
|
|
|
test-aider-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-aider == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build aider integration
|
|
working-directory: ./hindsight-integrations/aider
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/aider
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/aider
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-autogen-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-autogen == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build autogen integration
|
|
working-directory: ./hindsight-integrations/autogen
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/autogen
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/autogen
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-composio-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-composio == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build composio integration
|
|
working-directory: ./hindsight-integrations/composio
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/composio
|
|
run: uv sync --frozen
|
|
|
|
- name: Lint
|
|
working-directory: ./hindsight-integrations/composio
|
|
run: uv run ruff check .
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/composio
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-continue-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-continue == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build continue integration
|
|
working-directory: ./hindsight-integrations/continue
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/continue
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/continue
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-smolagents-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-smolagents == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build smolagents integration
|
|
working-directory: ./hindsight-integrations/smolagents
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/smolagents
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/smolagents
|
|
run: uv run pytest tests -v
|
|
|
|
test-dify-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
github.event_name != 'pull_request_review' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-dify == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
# Dify plugin runtime targets Python 3.12 (see manifest.yaml)
|
|
python-version: '3.12'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/dify
|
|
run: uv pip install --system -e . pytest pytest-mock
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/dify
|
|
run: pytest tests -v
|
|
|
|
test-flowise-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
github.event_name != 'pull_request_review' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-flowise == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/flowise
|
|
run: npm install --no-audit --no-fund
|
|
|
|
- name: Type check
|
|
working-directory: ./hindsight-integrations/flowise
|
|
run: npx tsc --noEmit
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/flowise
|
|
run: npm test
|
|
|
|
test-obsidian-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
github.event_name != 'pull_request_review' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-obsidian == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/obsidian
|
|
run: npm install --no-audit --no-fund
|
|
|
|
- name: Type check
|
|
working-directory: ./hindsight-integrations/obsidian
|
|
run: npx tsc --noEmit
|
|
|
|
- name: Build
|
|
working-directory: ./hindsight-integrations/obsidian
|
|
run: npm run build
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/obsidian
|
|
run: npm test
|
|
|
|
test-agent-framework-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-agent-framework == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build agent-framework integration
|
|
working-directory: ./hindsight-integrations/agent-framework
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/agent-framework
|
|
run: uv sync --frozen
|
|
|
|
- name: Lint
|
|
working-directory: ./hindsight-integrations/agent-framework
|
|
run: uv run ruff check .
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/agent-framework
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-copilot-cli-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-copilot-cli == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build copilot-cli integration
|
|
working-directory: ./hindsight-integrations/copilot-cli
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/copilot-cli
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/copilot-cli
|
|
run: uv run pytest tests -v
|
|
|
|
test-extension-supabase-tenant:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.extensions-supabase-tenant == 'true' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
# No --frozen: the lock is not checked in, because tool.uv.sources resolves
|
|
# hindsight-api-slim from the local checkout and its dependency tree moves
|
|
# with core.
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-extensions/supabase-tenant
|
|
run: uv sync
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-extensions/supabase-tenant
|
|
run: uv run pytest tests -v
|
|
|
|
# The Dockerfile IS the distribution mechanism — its final `import` step is
|
|
# the only thing proving the extension is reachable from the interpreter the
|
|
# server runs. Build it from the repo root, where the sources are in context.
|
|
- name: Build the extension image
|
|
run: |
|
|
docker build \
|
|
-f hindsight-extensions/supabase-tenant/Dockerfile \
|
|
--build-arg HINDSIGHT_IMAGE=ghcr.io/vectorize-io/hindsight:latest-slim \
|
|
-t hindsight-with-supabase .
|
|
|
|
test-crewai-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-crewai == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build crewai integration
|
|
working-directory: ./hindsight-integrations/crewai
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/crewai
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/crewai
|
|
run: uv run pytest tests -v
|
|
|
|
test-vapi-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
github.event_name != 'pull_request_review' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-vapi == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build vapi integration
|
|
working-directory: ./hindsight-integrations/vapi
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/vapi
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/vapi
|
|
run: uv run pytest tests -v
|
|
|
|
test-superagent-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
github.event_name != 'pull_request_review' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-superagent == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build superagent integration
|
|
working-directory: ./hindsight-integrations/superagent
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/superagent
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/superagent
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs live Hindsight + provider keys and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-litellm-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-litellm == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build litellm integration
|
|
working-directory: ./hindsight-integrations/litellm
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/litellm
|
|
run: uv sync --frozen --extra dev
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/litellm
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs live Hindsight + provider keys and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-pydantic-ai-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-pydantic-ai == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build pydantic-ai integration
|
|
working-directory: ./hindsight-integrations/pydantic-ai
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/pydantic-ai
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/pydantic-ai
|
|
run: uv run pytest tests -v
|
|
|
|
test-langgraph-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-langgraph == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build langgraph integration
|
|
working-directory: ./hindsight-integrations/langgraph
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/langgraph
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/langgraph
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-llamaindex-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-llamaindex == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build llamaindex integration
|
|
working-directory: ./hindsight-integrations/llamaindex
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/llamaindex
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/llamaindex
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-haystack-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
github.event_name != 'pull_request_review' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-haystack == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build haystack integration
|
|
working-directory: ./hindsight-integrations/haystack
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/haystack
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/haystack
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-openai-agents-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-openai-agents == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build openai-agents integration
|
|
working-directory: ./hindsight-integrations/openai-agents
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/openai-agents
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/openai-agents
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-openhands-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-openhands == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build openhands integration
|
|
working-directory: ./hindsight-integrations/openhands
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/openhands
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/openhands
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-devin-desktop-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-devin-desktop == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build devin-desktop integration
|
|
working-directory: ./hindsight-integrations/devin-desktop
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/devin-desktop
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/devin-desktop
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-claude-agent-sdk-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-claude-agent-sdk == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build claude-agent-sdk integration
|
|
working-directory: ./hindsight-integrations/claude-agent-sdk
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/claude-agent-sdk
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/claude-agent-sdk
|
|
# PR CI runs only the deterministic bucket; the real-LLM E2E bucket
|
|
# (requires_real_llm) needs a live Hindsight server and runs separately.
|
|
run: uv run pytest tests -v -m "not requires_real_llm"
|
|
|
|
test-agentcore-integration:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.integrations-agentcore == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Build agentcore integration
|
|
working-directory: ./hindsight-integrations/agentcore
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-integrations/agentcore
|
|
run: uv sync --frozen
|
|
|
|
- name: Run tests
|
|
working-directory: ./hindsight-integrations/agentcore
|
|
run: uv run pytest tests -v
|
|
|
|
test-pip-slim:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_EMBEDDINGS_PROVIDER: cohere
|
|
HINDSIGHT_API_RERANKER_PROVIDER: cohere
|
|
HINDSIGHT_API_COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install hindsight-api-slim (embedded-db only, no local ML)
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --extra embedded-db --index-strategy unsafe-best-match
|
|
|
|
- name: Start API server
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run hindsight-api --port 8888 > /tmp/slim-api-server.log 2>&1 &
|
|
for i in $(seq 1 60); do
|
|
if curl -s http://localhost:8888/health | grep -q "healthy"; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 60 ]; then
|
|
echo "API server failed to start after 60s"
|
|
cat /tmp/slim-api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Smoke test - retain and recall
|
|
run: ./scripts/smoke-test-slim.sh http://localhost:8888
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/slim-api-server.log 2>/dev/null || true
|
|
|
|
verify-embed-control-center-bundle:
|
|
# The control center UI (Preact + Tailwind) is built with Vite and its static
|
|
# output is committed (served as-is by the embed's Python http.server, no Node
|
|
# at runtime). We can't byte-diff the committed bundle against a fresh build —
|
|
# Vite's content-hashed asset filenames aren't reproducible across the CI
|
|
# runner's OS/arch vs the committer's. So instead verify: (1) the committed
|
|
# bundle is a real, wired Vite build (index.html references JS/CSS that exist),
|
|
# and (2) the source still builds cleanly.
|
|
needs: [detect-changes]
|
|
if: >-
|
|
github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.embed == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup Node
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '22'
|
|
|
|
# Check the committed bundle BEFORE building (the build overwrites static/).
|
|
- name: Verify the committed bundle is wired
|
|
working-directory: ./hindsight-embed/hindsight_embed/control_center
|
|
run: |
|
|
test -f static/index.html || { echo "::error::static/index.html missing — run 'npm run build' in control_center/ui and commit static/"; exit 1; }
|
|
js=$(grep -oE 'assets/[A-Za-z0-9_.-]+\.js' static/index.html | head -1)
|
|
css=$(grep -oE 'assets/[A-Za-z0-9_.-]+\.css' static/index.html | head -1)
|
|
{ [ -n "$js" ] && [ -f "static/$js" ]; } || { echo "::error::index.html does not reference a committed JS bundle — rebuild the UI and commit static/"; exit 1; }
|
|
{ [ -n "$css" ] && [ -f "static/$css" ]; } || { echo "::error::index.html does not reference a committed CSS bundle — rebuild the UI and commit static/"; exit 1; }
|
|
echo "committed bundle is wired ✓"
|
|
|
|
- name: Verify the source builds cleanly
|
|
working-directory: ./hindsight-embed/hindsight_embed/control_center/ui
|
|
run: |
|
|
npm ci
|
|
npm run build
|
|
echo "control center UI builds ✓"
|
|
|
|
test-embed:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.embed == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
# Prefer CPU-only PyTorch in CI
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-embed
|
|
run: uv sync --frozen --index-strategy unsafe-best-match
|
|
|
|
- name: Install API dependencies (with local-ml and embedded-db for smoke test)
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-embed-${{ hashFiles('hindsight-embed/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-embed-
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Run unit and integration tests
|
|
working-directory: ./hindsight-embed
|
|
run: uv run pytest tests/ -v
|
|
|
|
- name: Run smoke test
|
|
working-directory: ./hindsight-embed
|
|
run: ./test.sh
|
|
|
|
test-embed-windows:
|
|
# Windows coverage for hindsight-embed. Runs the same unit tests + smoke
|
|
# test as the Linux `test-embed` job, plus a `uv pip install --target`
|
|
# sanity check that validates the sibling-binary resolution used by
|
|
# users who install via `uv pip install hindsight-all` on Windows
|
|
# (closes #1240).
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.embed == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: windows-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: ${{ github.workspace }}/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
# Force UTF-8 I/O so the CLI's ✓/box-drawing output doesn't crash the
|
|
# default Windows cp1252 codec. Also applied at runtime via
|
|
# sys.stdout.reconfigure in cli.py; this belt-and-suspenders covers
|
|
# subprocesses the daemon spawns.
|
|
PYTHONIOENCODING: utf-8
|
|
PYTHONUTF8: "1"
|
|
# pg0-embedded unpacks Postgres on first boot — noticeably slower on a
|
|
# cold Windows runner than POSIX. Double the embed startup budget.
|
|
HINDSIGHT_EMBED_DAEMON_STARTUP_TIMEOUT: "360"
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
shell: bash
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> "$GITHUB_ENV"
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install embed dependencies
|
|
working-directory: ./hindsight-embed
|
|
run: uv sync --frozen --index-strategy unsafe-best-match
|
|
|
|
- name: Install API dependencies (with local-ml and embedded-db for smoke test)
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-embed-${{ hashFiles('hindsight-embed/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-embed-
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Run unit and integration tests
|
|
working-directory: ./hindsight-embed
|
|
run: uv run pytest tests/ -v
|
|
|
|
# Smoke test's retain/recall commands delegate to the Rust hindsight CLI.
|
|
# On POSIX, hindsight-embed auto-installs the CLI via curl|bash; on
|
|
# Windows that installer isn't available (and `bash` on windows-latest
|
|
# routes to WSL which isn't provisioned). Build the CLI from source and
|
|
# drop it into ~/.local/bin where find_cli_binary() looks first.
|
|
- name: Install Rust
|
|
uses: dtolnay/rust-toolchain@stable
|
|
|
|
- name: Cache cargo build
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: |
|
|
~/.cargo/registry
|
|
~/.cargo/git
|
|
hindsight-cli/target
|
|
key: ${{ runner.os }}-cargo-embed-smoke-${{ hashFiles('**/Cargo.lock') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-cargo-embed-smoke-
|
|
${{ runner.os }}-cargo-
|
|
|
|
- name: Build hindsight CLI
|
|
working-directory: ./hindsight-cli
|
|
run: cargo build --release
|
|
|
|
- name: Stage hindsight CLI where find_cli_binary expects it
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
install_dir="$HOME/.local/bin"
|
|
mkdir -p "$install_dir"
|
|
cp hindsight-cli/target/release/hindsight.exe "$install_dir/hindsight.exe"
|
|
"$install_dir/hindsight.exe" --version
|
|
|
|
- name: Run smoke test
|
|
shell: bash
|
|
working-directory: ./hindsight-embed
|
|
run: ./test.sh
|
|
|
|
# Real-world install test for issue #1240: drop both packages into a
|
|
# --target directory (the layout you get from `uv pip install hindsight-all`
|
|
# or NixOS) and verify the sibling binary is discovered (not the uvx
|
|
# fallback). Exercises a different code path than the smoke test, which
|
|
# uses `uv run --project` via the monorepo branch of _find_api_command.
|
|
#
|
|
# IMPORTANT: install outside the repo checkout. `_find_api_command` first
|
|
# probes `<pkg>/../../hindsight-api-slim` for dev mode; if the target dir
|
|
# lives inside the monorepo, that branch matches and we never exercise
|
|
# the sibling-binary path we actually want to test.
|
|
- name: Install hindsight-embed and hindsight-api into --target directory
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
target="$RUNNER_TEMP/install-test"
|
|
rm -rf "$target"
|
|
mkdir -p "$target"
|
|
uv pip install --target "$target" ./hindsight-embed ./hindsight-api-slim
|
|
|
|
- name: Verify sibling hindsight-api.exe is present
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
target="$RUNNER_TEMP/install-test"
|
|
if [ -f "$target/Scripts/hindsight-api.exe" ]; then
|
|
echo "Found $target/Scripts/hindsight-api.exe"
|
|
elif [ -f "$target/bin/hindsight-api.exe" ]; then
|
|
echo "Found $target/bin/hindsight-api.exe"
|
|
else
|
|
echo "::error::hindsight-api.exe not found in install target"
|
|
ls "$target/"
|
|
exit 1
|
|
fi
|
|
|
|
- name: Verify _find_api_command resolves the sibling binary (not uvx)
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
target="$RUNNER_TEMP/install-test"
|
|
PYTHONPATH="$target" python -c "
|
|
from hindsight_embed.daemon_embed_manager import DaemonEmbedManager
|
|
# api_version is only used for the uvx fallback; the binary branch ignores it.
|
|
cmd = DaemonEmbedManager()._find_api_command('0.0.0')
|
|
print('Resolved command:', cmd)
|
|
assert len(cmd) == 1 and cmd[0].endswith('hindsight-api.exe'), (
|
|
f'Expected sibling hindsight-api.exe, got {cmd!r}. '
|
|
'Falling back to uvx on --target installs reintroduces issue #1240.'
|
|
)
|
|
"
|
|
|
|
- name: Smoke-check installed hindsight-embed binary runs
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
target="$RUNNER_TEMP/install-test"
|
|
export PYTHONPATH="$target"
|
|
if [ -f "$target/Scripts/hindsight-embed.exe" ]; then
|
|
"$target/Scripts/hindsight-embed.exe" --help
|
|
else
|
|
"$target/bin/hindsight-embed.exe" --help
|
|
fi
|
|
|
|
- name: Collect daemon logs on failure
|
|
if: failure()
|
|
shell: bash
|
|
run: |
|
|
for f in ~/.hindsight/daemon.log ~/.hindsight/profiles/*.log ~/.hindsight/profiles/*.stderr.log; do
|
|
if [ -f "$f" ]; then
|
|
echo "=== $f ==="
|
|
cat "$f"
|
|
fi
|
|
done || true
|
|
|
|
# Compatibility gate against hermes-agent's *main* branch.
|
|
#
|
|
# `hermes memory setup` installs `hindsight-all` into Hermes' own venv to run
|
|
# memory in local_embedded mode, and Hermes exact-pins every direct dependency
|
|
# (`==X.Y.Z`) as a deliberate supply-chain policy — they will not loosen a pin
|
|
# for us. So any version range Hindsight declares that excludes one of their
|
|
# pins makes the two impossible to co-install for every Hermes user on
|
|
# embedded memory. That was #3251: our `cryptography>=48.0.1` / `pillow>=12.3.0`
|
|
# against their `==46.0.7` / `==12.2.0`, which left `pip check` permanently
|
|
# broken. Both sides bump on their own schedule, so this needs a standing gate
|
|
# rather than a one-off fix; tracking main surfaces the next collision while
|
|
# it is still cheap to fix on either side.
|
|
#
|
|
# Deliberately not gated on has_secrets — the resolution, wiring, runtime and
|
|
# daemon-boot checks need no credentials, so this runs on fork PRs too. Only
|
|
# retain/recall need an LLM and the script skips them when no key is present.
|
|
test-hermes-compat:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.embed == 'true' ||
|
|
needs.detect-changes.outputs.hindsight-all == 'true' ||
|
|
needs.detect-changes.outputs.clients-python == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 45
|
|
permissions:
|
|
contents: read
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
# hindsight-all pulls the local-ml extra, so the embedded daemon can load
|
|
# sentence-transformers models. Cache them like the test-embed job does.
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-hermes-compat-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-hermes-compat-
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Run Hermes compatibility test
|
|
env:
|
|
GITHUB_TOKEN: ${{ github.token }}
|
|
run: ./scripts/test-hermes-compat.sh
|
|
|
|
- name: Collect embedded daemon logs on failure
|
|
if: failure()
|
|
run: |
|
|
for f in ~/.hindsight/profiles/hermes-ci*.log ~/.hindsight/profiles/hermes-ci*.stderr.log; do
|
|
if [ -f "$f" ]; then
|
|
echo "=== $f ==="
|
|
cat "$f"
|
|
fi
|
|
done || true
|
|
|
|
test-hindsight-all:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.hindsight-all == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
# For test_server_integration.py compatibility
|
|
HINDSIGHT_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
# Prefer CPU-only PyTorch in CI
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '20'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install SDK dependencies
|
|
run: npm ci --workspace=hindsight-clients/typescript
|
|
|
|
- name: Build SDK
|
|
run: npm run build --workspace=hindsight-clients/typescript
|
|
|
|
- name: Install Control Plane dependencies
|
|
run: |
|
|
npm install --workspace=hindsight-control-plane
|
|
rm -rf node_modules/lightningcss node_modules/@tailwindcss
|
|
npm install lightningcss @tailwindcss/postcss @tailwindcss/node
|
|
|
|
- name: Test Control Plane
|
|
run: npm test --workspace=hindsight-control-plane
|
|
|
|
- name: Check i18n locale parity and hardcoded strings
|
|
run: npm run i18n:check --workspace=hindsight-control-plane
|
|
|
|
- name: Build Control Plane
|
|
run: npm run build --workspace=hindsight-control-plane
|
|
|
|
- name: Build hindsight-all
|
|
working-directory: ./hindsight-all
|
|
run: uv build
|
|
|
|
- name: Install dependencies
|
|
working-directory: ./hindsight-all
|
|
run: uv sync --frozen --extra test --index-strategy unsafe-best-match
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-all-${{ hashFiles('hindsight-all/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-all-
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Run unit tests
|
|
working-directory: ./hindsight-all
|
|
run: uv run pytest tests/ -v
|
|
|
|
test-doc-examples:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.clients-ts == 'true' ||
|
|
needs.detect-changes.outputs.clients-python == 'true' ||
|
|
needs.detect-changes.outputs.clients-go == 'true' ||
|
|
needs.detect-changes.outputs.cli == 'true' ||
|
|
needs.detect-changes.outputs.doc-examples == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
language: [python, node, cli, go]
|
|
name: test-doc-examples (${{ matrix.language }})
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
HINDSIGHT_API_URL: http://localhost:8888
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Install Rust
|
|
if: matrix.language == 'cli'
|
|
uses: dtolnay/rust-toolchain@stable
|
|
|
|
- name: Cache cargo
|
|
if: matrix.language == 'cli'
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: |
|
|
~/.cargo/registry
|
|
~/.cargo/git
|
|
hindsight-cli/target
|
|
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
|
|
|
|
- name: Build CLI
|
|
if: matrix.language == 'cli'
|
|
working-directory: hindsight-cli
|
|
run: |
|
|
cargo build --release
|
|
cp target/release/hindsight /usr/local/bin/hindsight
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Set up Node.js
|
|
if: matrix.language == 'node'
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '20'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install Python client dependencies
|
|
if: matrix.language == 'python'
|
|
working-directory: ./hindsight-clients/python
|
|
run: uv sync --frozen --extra test --index-strategy unsafe-best-match
|
|
|
|
- name: Build and install API
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv build
|
|
uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Install TypeScript client
|
|
if: matrix.language == 'node'
|
|
run: |
|
|
npm ci --workspace=hindsight-clients/typescript
|
|
npm run build --workspace=hindsight-clients/typescript
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading reranker model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Create .env file
|
|
run: |
|
|
cat > .env << EOF
|
|
HINDSIGHT_API_LLM_PROVIDER=${{ env.HINDSIGHT_API_LLM_PROVIDER }}
|
|
HINDSIGHT_API_LLM_MODEL=${{ env.HINDSIGHT_API_LLM_MODEL }}
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
|
|
EOF
|
|
|
|
- name: Start API server
|
|
run: |
|
|
./scripts/dev/start-api.sh > /tmp/api-server.log 2>&1 &
|
|
echo "Waiting for API server to be ready..."
|
|
for i in {1..120}; do
|
|
if curl -sf http://localhost:8888/health > /dev/null 2>&1; then
|
|
echo "API server is ready after ${i}s"
|
|
break
|
|
fi
|
|
if [ $i -eq 120 ]; then
|
|
echo "API server failed to start after 120s"
|
|
cat /tmp/api-server.log
|
|
exit 1
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
- name: Configure CLI
|
|
if: matrix.language == 'cli'
|
|
run: hindsight configure --api-url http://localhost:8888
|
|
|
|
- name: Run doc examples (${{ matrix.language }})
|
|
run: ./scripts/test-doc-examples.sh --lang ${{ matrix.language }}
|
|
|
|
- name: Show API server logs
|
|
if: always()
|
|
run: |
|
|
echo "=== API Server Logs ==="
|
|
cat /tmp/api-server.log || echo "No API server log found"
|
|
|
|
test-upgrade:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
needs.detect-changes.outputs.has_secrets == 'true' &&
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.dev == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
HINDSIGHT_API_LLM_PROVIDER: vertexai
|
|
HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json
|
|
HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
fetch-depth: 0 # Full history needed for git clone of tags
|
|
|
|
- name: Setup GCP credentials
|
|
run: |
|
|
printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json
|
|
PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json)
|
|
echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV
|
|
|
|
- name: Fetch tags
|
|
run: git fetch --tags
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
prune-cache: false
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Cache HuggingFace models
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: ~/.cache/huggingface
|
|
key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }}
|
|
restore-keys: |
|
|
${{ runner.os }}-huggingface-
|
|
|
|
- name: Install hindsight-dev dependencies
|
|
working-directory: ./hindsight-dev
|
|
run: uv sync --frozen --extra test --index-strategy unsafe-best-match
|
|
|
|
- name: Install current hindsight-api
|
|
working-directory: ./hindsight-api-slim
|
|
run: uv sync --frozen --all-extras --index-strategy unsafe-best-match
|
|
|
|
- name: Pre-download models
|
|
working-directory: ./hindsight-api-slim
|
|
run: |
|
|
uv run python -c "
|
|
from sentence_transformers import SentenceTransformer, CrossEncoder
|
|
print('Downloading embedding model...')
|
|
SentenceTransformer('BAAI/bge-small-en-v1.5')
|
|
print('Downloading cross-encoder model...')
|
|
CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
|
|
print('Models downloaded successfully')
|
|
"
|
|
|
|
- name: Run upgrade tests
|
|
working-directory: ./hindsight-dev
|
|
run: uv run pytest upgrade_tests/ -v --tb=short
|
|
|
|
- name: Show upgrade test logs
|
|
if: always()
|
|
run: |
|
|
echo "=== Upgrade Test Server Logs ==="
|
|
for log in /tmp/upgrade-test-*.log; do
|
|
if [ -f "$log" ]; then
|
|
echo ""
|
|
echo "--- $log ---"
|
|
tail -500 "$log"
|
|
fi
|
|
done
|
|
|
|
# Dead-code detection beyond what ruff's F401/F841 catch (those are already
|
|
# BLOCKING via the ruff config + the verify-generated-files job).
|
|
#
|
|
# - knip (control plane): BLOCKING on unused files / dependencies / unlisted
|
|
# dependencies. These are unambiguous — an orphaned file or a dead
|
|
# package.json entry — so they fail the build.
|
|
# - vulture (Python) + knip unused *exports*: ADVISORY only. vulture's
|
|
# function/argument heuristics false-positive on FastAPI/SQLAlchemy/Pydantic
|
|
# patterns, and the control plane intentionally keeps an unused shadcn/ui
|
|
# component surface, so these are surfaced in the step summary, not gated.
|
|
check-unused-code:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.control-plane == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 15
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '20'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install Control Plane dependencies
|
|
run: npm install --workspace=hindsight-control-plane
|
|
|
|
- name: knip — unused files / dependencies (blocking)
|
|
working-directory: hindsight-control-plane
|
|
run: npx --yes knip@5 --no-progress --include files,dependencies,unlisted
|
|
|
|
- name: Advisory scan — vulture + knip exports
|
|
continue-on-error: true
|
|
run: |
|
|
{
|
|
echo '## Dead-code scan (advisory)'
|
|
echo ''
|
|
echo '```'
|
|
./scripts/hooks/check-unused.sh 2>&1 | sed 's/\x1b\[[0-9;]*m//g'
|
|
echo '```'
|
|
} | tee -a "$GITHUB_STEP_SUMMARY"
|
|
|
|
verify-generated-files:
|
|
needs: [detect-changes]
|
|
# Unconditional for every change except one confined to
|
|
# hindsight-system-tests/ — no generated file in the repo is produced from
|
|
# that package, so it cannot drift them. See the outside-system-tests filter.
|
|
if: needs.detect-changes.outputs.outside-system-tests == 'true'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
env:
|
|
UV_FROZEN: "1"
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: '20'
|
|
cache: 'npm'
|
|
cache-dependency-path: package-lock.json
|
|
|
|
- name: Install Rust
|
|
uses: dtolnay/rust-toolchain@stable
|
|
|
|
- name: Cache cargo
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: |
|
|
~/.cargo/registry
|
|
~/.cargo/git
|
|
key: ${{ runner.os }}-cargo-gen-${{ hashFiles('**/Cargo.lock') }}
|
|
|
|
- name: Install Node dependencies
|
|
run: npm ci
|
|
|
|
- name: Install Python dependencies
|
|
run: |
|
|
cd hindsight-dev && uv sync --frozen --index-strategy unsafe-best-match
|
|
cd ../hindsight-api && uv sync --frozen --index-strategy unsafe-best-match
|
|
cd ../hindsight-embed && uv sync --frozen --index-strategy unsafe-best-match
|
|
|
|
- name: Run generate-openapi
|
|
working-directory: hindsight-dev
|
|
run: uv run generate-openapi
|
|
|
|
- name: Run generate-bank-template-schema
|
|
run: ./scripts/generate-bank-template-schema.sh
|
|
|
|
- name: Run generate-clients
|
|
run: ./scripts/generate-clients.sh
|
|
|
|
- name: Run generate-docs-skill
|
|
run: ./scripts/generate-docs-skill.sh
|
|
|
|
- name: Run lint
|
|
run: ./scripts/hooks/lint.sh
|
|
|
|
- name: Verify no uncommitted changes
|
|
run: |
|
|
if [ -n "$(git status --porcelain)" ]; then
|
|
echo "❌ Error: Generated files are out of sync with committed files."
|
|
echo ""
|
|
echo "The following files have changed after running generation scripts:"
|
|
git status --porcelain
|
|
echo ""
|
|
echo "Please run the following commands locally and commit the changes:"
|
|
echo " ./scripts/generate-openapi.sh"
|
|
echo " ./scripts/generate-bank-template-schema.sh"
|
|
echo " ./scripts/generate-clients.sh"
|
|
echo " ./scripts/generate-docs-skill.sh"
|
|
echo " ./scripts/hooks/lint.sh"
|
|
echo ""
|
|
git diff --stat
|
|
exit 1
|
|
fi
|
|
echo "✓ All generated files are up to date"
|
|
|
|
check-openapi-compatibility:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
fetch-depth: 0 # Fetch full git history to access base branch
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install hindsight-dev dependencies
|
|
run: |
|
|
cd hindsight-dev && uv sync --frozen --index-strategy unsafe-best-match
|
|
|
|
- name: Check OpenAPI compatibility with base branch
|
|
run: |
|
|
# Get the base branch (usually main)
|
|
BASE_BRANCH="${{ github.base_ref }}"
|
|
|
|
if [ -z "$BASE_BRANCH" ]; then
|
|
echo "⚠️ Warning: No base branch found (not a PR?). Skipping compatibility check."
|
|
exit 0
|
|
fi
|
|
|
|
# Compare against the point this branch was cut from, NOT the live tip of
|
|
# the base branch. Using the tip reports every endpoint main has gained
|
|
# since the branch was cut as "removed by this PR" — a false positive that
|
|
# fails PRs which touch no spec at all, and whose only cure is an unrelated
|
|
# rebase. The merge-base answers the question the check actually asks:
|
|
# did *this branch* remove something?
|
|
MERGE_BASE="$(git merge-base "origin/$BASE_BRANCH" HEAD)"
|
|
|
|
if [ -z "$MERGE_BASE" ]; then
|
|
echo "⚠️ Warning: Could not determine merge-base with $BASE_BRANCH. Skipping compatibility check."
|
|
exit 0
|
|
fi
|
|
|
|
echo "Checking OpenAPI compatibility against $BASE_BRANCH merge-base: $MERGE_BASE"
|
|
|
|
# Extract the old OpenAPI spec from the merge-base
|
|
git show "$MERGE_BASE:hindsight-docs/static/openapi.json" > /tmp/old-openapi.json
|
|
|
|
if [ ! -s /tmp/old-openapi.json ]; then
|
|
echo "⚠️ Warning: Could not find OpenAPI spec at the merge-base. Skipping compatibility check."
|
|
exit 0
|
|
fi
|
|
|
|
# Check compatibility using our tool
|
|
cd hindsight-dev
|
|
uv run check-openapi-compatibility /tmp/old-openapi.json ../hindsight-docs/static/openapi.json
|
|
|
|
check-cli-coverage:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.cli == 'true' ||
|
|
needs.detect-changes.outputs.dev == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install hindsight-dev dependencies
|
|
run: |
|
|
cd hindsight-dev && uv sync --frozen --index-strategy unsafe-best-match
|
|
|
|
- name: Check CLI covers every OpenAPI operation
|
|
run: |
|
|
cd hindsight-dev
|
|
uv run cli-coverage-check
|
|
|
|
# The wrapper clients (hindsight_client.py / src/index.ts) are hand-written
|
|
# convenience layers over the generated SDKs, so nothing regenerates them when
|
|
# the API gains a field. client-coverage-check has existed for that, but had no
|
|
# job: it sat red on main with reflect.apply_all_directives missing from both
|
|
# wrappers, and the bank-config surface had drifted to 22 settings reachable
|
|
# from neither. See #4029.
|
|
check-client-coverage:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.clients-ts == 'true' ||
|
|
needs.detect-changes.outputs.clients-python == 'true' ||
|
|
needs.detect-changes.outputs.dev == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install hindsight-dev dependencies
|
|
run: |
|
|
cd hindsight-dev && uv sync --frozen --index-strategy unsafe-best-match
|
|
|
|
- name: Check client wrappers cover every request param and bank config field
|
|
run: |
|
|
cd hindsight-dev
|
|
uv run client-coverage-check
|
|
|
|
# hindsight-dev/tests had no job of its own, so nothing ran it: the benchmark
|
|
# harness was only exercised by the nightly Performance Tests workflow, where a
|
|
# plain construction bug in the answer/judge LLM config surfaced as a red
|
|
# benchmark hours later instead of on the PR that introduced it.
|
|
test-dev:
|
|
needs: [detect-changes]
|
|
if: >-
|
|
(github.event_name == 'workflow_dispatch' ||
|
|
needs.detect-changes.outputs.core == 'true' ||
|
|
needs.detect-changes.outputs.dev == 'true' ||
|
|
needs.detect-changes.outputs.ci == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
ref: ${{ github.event.pull_request.head.sha || '' }}
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
with:
|
|
enable-cache: true
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version-file: ".python-version"
|
|
|
|
- name: Install hindsight-dev dependencies
|
|
run: |
|
|
cd hindsight-dev && uv sync --frozen --extra test --index-strategy unsafe-best-match
|
|
|
|
- name: Run hindsight-dev tests
|
|
working-directory: hindsight-dev
|
|
run: uv run pytest tests/ -v
|
|
|
|
# Report CI status back to the PR for pull_request_review events.
|
|
# GitHub does not automatically link pull_request_review check runs to the PR,
|
|
# so we create a commit status on the PR head SHA and post a comment.
|
|
report-pr-status:
|
|
if: github.event_name == 'pull_request_review' && github.event.review.state == 'approved' && always()
|
|
needs:
|
|
- detect-changes
|
|
- check-integration-lockfiles
|
|
- build-api-python-versions
|
|
- build-typescript-client
|
|
- build-openclaw-integration
|
|
- smoke-openclaw-install
|
|
- test-claude-code-integration
|
|
- test-cursor-integration
|
|
- test-cline-integration
|
|
- test-github-copilot-integration
|
|
- test-codex-integration
|
|
- test-cursor-cli-integration
|
|
- test-zcode-integration
|
|
- test-agent-plugin-integration
|
|
- build-ai-sdk-integration
|
|
- test-ai-sdk-integration-deno
|
|
- test-opencode-integration
|
|
- test-eve-integration
|
|
- test-omo-integration
|
|
- test-cloudflare-oauth-proxy-integration
|
|
- build-chat-integration
|
|
- test-paperclip-integration
|
|
- test-pipecat-integration
|
|
- test-gemini-spark-integration
|
|
- test-vapi-integration
|
|
- test-google-adk-integration
|
|
- test-roo-code-integration
|
|
- build-control-plane
|
|
- build-docs
|
|
- test-rust-cli
|
|
- lint-helm-chart
|
|
- test-standalone-start-script
|
|
- build-docker-images
|
|
- test-api
|
|
- test-api-oracle
|
|
- test-python-client
|
|
- test-python-client-oracle
|
|
- test-typescript-client
|
|
- test-typescript-client-oracle
|
|
- test-typescript-client-deno
|
|
- build-rust-cli-arm64
|
|
- test-rust-client
|
|
- test-go-client
|
|
- test-openclaw-integration
|
|
- test-integration
|
|
- test-ag2-integration
|
|
- test-aider-integration
|
|
- test-autogen-integration
|
|
- test-continue-integration
|
|
- test-smolagents-integration
|
|
- test-dify-integration
|
|
- test-flowise-integration
|
|
- test-obsidian-integration
|
|
- test-agent-framework-integration
|
|
- test-extension-supabase-tenant
|
|
- test-crewai-integration
|
|
- test-langgraph-integration
|
|
- test-superagent-integration
|
|
- test-litellm-integration
|
|
- test-pydantic-ai-integration
|
|
- test-llamaindex-integration
|
|
- test-openai-agents-integration
|
|
- test-openhands-integration
|
|
- test-devin-desktop-integration
|
|
- test-agentcore-integration
|
|
- test-haystack-integration
|
|
- test-pip-slim
|
|
- test-embed
|
|
- test-embed-windows
|
|
- verify-embed-control-center-bundle
|
|
- test-hermes-compat
|
|
- test-hindsight-all
|
|
- test-hindsight-agent-sdk
|
|
- test-claude-agent-sdk-integration
|
|
- test-doc-examples
|
|
- test-upgrade
|
|
- verify-generated-files
|
|
- check-openapi-compatibility
|
|
- check-cli-coverage
|
|
- check-client-coverage
|
|
- test-system
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
permissions:
|
|
statuses: write
|
|
pull-requests: write
|
|
steps:
|
|
- name: Determine overall result
|
|
id: result
|
|
uses: actions/github-script@v9
|
|
with:
|
|
script: |
|
|
const needs = ${{ toJSON(needs) }};
|
|
const entries = Object.entries(needs);
|
|
const failed = entries.filter(([, v]) => v.result === 'failure');
|
|
const skipped = entries.filter(([, v]) => v.result === 'skipped');
|
|
const succeeded = entries.filter(([, v]) => v.result === 'success');
|
|
const overall = failed.length > 0 ? 'failure' : 'success';
|
|
const icon = overall === 'success' ? '✅' : '❌';
|
|
const runUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
|
|
|
|
let body = `## ${icon} CI results\n\n`;
|
|
body += `| Status | Count |\n|--------|-------|\n`;
|
|
body += `| ✅ Passed | ${succeeded.length} |\n`;
|
|
if (failed.length > 0) body += `| ❌ Failed | ${failed.length} |\n`;
|
|
if (skipped.length > 0) body += `| ⏭️ Skipped | ${skipped.length} |\n`;
|
|
|
|
if (failed.length > 0) {
|
|
body += `\n### Failed jobs\n`;
|
|
for (const [name] of failed) {
|
|
body += `- \`${name}\`\n`;
|
|
}
|
|
}
|
|
|
|
body += `\n[View full run](${runUrl})`;
|
|
|
|
core.setOutput('state', overall);
|
|
core.setOutput('description', failed.length > 0 ? 'Some CI checks failed' : 'All CI checks passed');
|
|
core.setOutput('body', body);
|
|
core.setOutput('run_url', runUrl);
|
|
|
|
- name: Report status to PR
|
|
uses: actions/github-script@v9
|
|
with:
|
|
script: |
|
|
await github.rest.repos.createCommitStatus({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
sha: context.payload.pull_request.head.sha,
|
|
state: '${{ steps.result.outputs.state }}',
|
|
context: 'CI / full-tests',
|
|
description: '${{ steps.result.outputs.description }}',
|
|
target_url: '${{ steps.result.outputs.run_url }}'
|
|
});
|
|
|
|
- name: Comment on PR
|
|
uses: actions/github-script@v9
|
|
with:
|
|
script: |
|
|
const prNumber = context.payload.pull_request.number;
|
|
const marker = '<!-- ci-full-test-report -->';
|
|
const body = `${marker}\n${{ steps.result.outputs.body }}`;
|
|
|
|
// Update existing comment if present, otherwise create new one
|
|
const { data: comments } = await github.rest.issues.listComments({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
issue_number: prNumber,
|
|
});
|
|
const existing = comments.find(c => c.body.includes(marker));
|
|
|
|
if (existing) {
|
|
await github.rest.issues.updateComment({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
comment_id: existing.id,
|
|
body,
|
|
});
|
|
} else {
|
|
await github.rest.issues.createComment({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
issue_number: prNumber,
|
|
body,
|
|
});
|
|
}
|