Issue #270 reported `ccc search --path` crashing with `TypeError:
unsupported operand type(s) for *: 'NoneType' and 'NoneType'`. The crash
could not be reproduced, and the reported root cause does not hold:
`vec_distance_L2` never returns NULL, it raises (verified against
sqlite-vec 0.1.6-0.1.9, SQLite 3.46/3.53, multi-chunk tables and
re-index churn). What the report did expose is that the failure was
undiagnosable.
Two gaps, both fixed here:
- The daemon's search handler discarded the traceback
(`ErrorResponse(message=str(e))`), so the reporter saw only the
client's re-raise frames. It now sends `traceback.format_exc()` and
logs the exception; `_dispatch` does the same. On the client, the
`raise RuntimeError(f"Daemon error: ...")` pattern was duplicated at
five sites and only `doctor()` appended `resp.traceback` -- the search
path, which the reporter came through, dropped it. Consolidated into
one `_daemon_error()` helper used by all five.
- `_knn_query` now rejects rows with a NULL distance, raising a specific
error naming the query shape and offending file instead of dying in
`_l2_to_score`. `distance` is a hidden vec0 column that sqlite-vec
populates only under the KNN query plan and returns NULL for on a full
scan, so a NULL means the plan we asked for is not the plan we got.
The guard lives in `_knn_query` rather than the caller because the
multi-language merge path sorts on `r[5]` in `heapq.nsmallest`, where
a NULL would fail on `None < float` before any caller-side check ran.
`_full_scan_query` needs no guard: it computes `vec_distance_L2(...)`
itself, which works under any plan and raises rather than returning
NULL on bad input.
The new tests build a real in-memory vec0 table with the indexer's exact
DDL (no embedding model, ~0.2s) and cover both that `--path` filtering
yields usable distances and that the bare-`distance`-under-full-scan
shape is the one that does not.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The socket address is <runtime dir>/daemon.sock, but sun_path caps a
unix socket at 104 bytes on macOS (108 on Linux) where ordinary files
get PATH_MAX. A deep $HOME -- a sandbox, a container, a CI runner --
pushes past it, and bind() fails with "AF_UNIX path too long" surfacing
as "Daemon process exited before it became ready", which reads like a
broken install rather than a path-length problem.
Fall back to a short temp-dir address keyed by a hash of the runtime
dir when the natural path is too long, so distinct runtime dirs keep
distinct sockets; the uid is in the name because /tmp is shared on
Linux. Client and daemon both resolve through daemon_socket_path(), so
they agree either way.
Found by an eval agent working in a sandbox with a long $HOME; it lost
several turns diagnosing the daemon before deducing the
COCOINDEX_CODE_RUNTIME_DIR workaround. Note pytest's own tmp_path is
~118 bytes on macOS, so the overflow is not an exotic case.
Claude-Session: https://claude.ai/code/session_01LksW8LnigLAiFnrgLauW8M
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
`MCPServer`'s `version` parameter defaults to `""`, so the daemon-backed
server advertised an empty `serverInfo.version`. Under the v1 SDK the
field was auto-filled with the SDK's own version, which was already the
wrong number to report — clients saw the MCP SDK release, not ours.
Pass `_version.__version__`, the same value `ccc version`, the daemon
handshake, and the version-mismatch check already use.
* feat(settings): add max_file_size to filter oversized files
settings.yml could only filter by glob, so keeping bundled or generated
files out of the index meant enumerating them in exclude_patterns, which
does not scale for a project carrying several plugin trees of minified
JavaScript.
Add an optional max_file_size to project settings. It accepts a plain
byte count or a binary-unit suffix (500KB, 1.5MB), and is applied by a
matcher wrapper in file_walk, so every consumer of the project's file
matching honors it from one place rather than each walker growing its
own size check. Omitting the key keeps today's behavior of indexing
files of any size.
Files that cannot be stat'd are left to the underlying matcher instead
of being dropped, since a size limit should not be what decides an
unreadable file's fate.
Closes#179
* test: type the walked() helper instead of ignoring the error
CI mypy runs over tests/ too and flagged the type: ignore as unused;
importing FilePathMatcher gives the helper a real annotation.
Until now the only way to learn the version was `ccc daemon status` or
`ccc doctor`, both of which require reaching the daemon — so a user
whose daemon won't start had no way to report what client they're on.
`ccc version` prints the client version offline: no project discovery,
no daemon, no settings. It reuses the same `_version.__version__` the
daemon handshake compares against, so the CLI and the protocol check
can't disagree.
Claude-Session: https://claude.ai/code/session_016YGCBE6RbF1uvvPYjfCUoW
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fixes#237. Upgrading 0.2.37 → 0.2.38 bricked every ccc command with
"ValidationError: Object missing required field `pid`": 0.2.38 added a
required `pid` field to HandshakeResponse, so the reply of a still-running
pre-upgrade daemon no longer decoded — before the client could see the
`ok=False` version mismatch and restart it. The same uncaught error also
broke `ccc daemon stop`, the recovery path.
Three layers:
- `HandshakeResponse.pid` gets a default (None), with a comment stating
the wire-compat rule: handshake fields added after a release must have
defaults, since the handshake is the one message exchanged between
mismatched versions.
- An undecodable handshake reply now raises `DaemonProtocolError` instead
of escaping as a raw decode error; `_connect_and_handshake` treats it
like a version mismatch (restart on first contact, fail fast once a
matching daemon was ensured). This protects against any future wire
drift, not just this field.
- `stop_daemon` tolerates the decode failure and falls through to its
SIGTERM/SIGKILL escalation, so `ccc daemon stop` always works.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(daemon): idle timeout — exit after inactivity, configurable via settings
The daemon now tracks client activity (every accepted connection, plus each
handler task's completion so long streaming index runs count to their end)
and exits after daemon.idle_timeout_minutes (default 180, 0 = never) without
it, instead of holding the embedding model in RAM forever. The idle exit
goes through the existing graceful shutdown path, so it leaves a last_exit
marker (reason "idle_timeout") and clients transparently and silently
restart the daemon on the next request.
- settings: new DaemonSettings dataclass with idle_timeout_minutes, parsed
from an optional daemon: section in global_settings.yml (absent section
and missing file both fall back to the default)
- protocol: HeartbeatRequest/HeartbeatResponse (daemon-side dispatch only;
the MCP client loop comes separately) and idle observability fields on
DaemonStatusResponse (idle_seconds, idle_timeout_minutes,
last_heartbeat_seconds)
- daemon: IdleReaper with a pure should_exit predicate — exit requires a
positive timeout, no external supervisor (COCOINDEX_CODE_DAEMON_SUPERVISED),
no live handler task, no project indexing (new ProjectRegistry.any_indexing),
and idle time past the timeout. A periodic asyncio task (60 s interval,
injectable for tests along with a seconds-scale timeout on run_daemon)
triggers the shutdown.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JaFKwEvJXo7WQhZvrW5Qoe
* feat(mcp): heartbeat loop keeps the daemon warm under a live MCP session
Both MCP entry points (ccc mcp and the legacy cocoindex-code serve) now run
a background heartbeat loop alongside the initial background index. Each
heartbeat is one short-lived daemon connection that counts as activity for
the idle reaper, so the daemon never idle-exits while an MCP session is
live; when the MCP process dies (even SIGKILL) the heartbeats stop and the
daemon exits on the normal timeout.
- client.send_heartbeat(): uses the raw non-auto-starting connect path and
returns False when no compatible daemon answers — a heartbeat must never
start or restart a daemon, or `ccc daemon stop` mid-session would fight
the loop
- server.run_heartbeat_loop(): interval = timeout/3 clamped to [30 s, 300 s],
reading the timeout from global_settings.yml (default when missing); a
0 timeout disables the loop entirely
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JaFKwEvJXo7WQhZvrW5Qoe
* feat(cli): show idle timeout and heartbeat age in daemon status; document idle-exit
- ccc daemon status now prints idle duration with the configured timeout
(or "disabled" when 0) and the age of the last MCP heartbeat (or
"never")
- README: document daemon.idle_timeout_minutes in the global settings
reference
- CLAUDE.md: note the idle-exit + heartbeat + crash-marker behavior in the
process model section
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JaFKwEvJXo7WQhZvrW5Qoe
* fix(daemon): release Windows named pipe on shutdown by unblocking the accept thread
PipeListener.close() only closes the queued next-instance handle; it cannot
cancel an in-flight ConnectNamedPipe wait, so a blocked accept() kept a pipe
instance open after the daemon exited. The named pipe therefore still existed
when the in-process idle tests asserted cleanup, and the test's own
os.path.exists() probe completed the pending connection, making the accept
thread call call_soon_threadsafe on a closed loop (the
PytestUnhandledThreadExceptionWarning noise in CI).
Shutdown now sets a shutting_down flag, wakes a blocked accept() on Windows
with a dummy client connection, and joins the accept thread before returning,
so the last pipe-instance handle is closed before run_daemon exits. POSIX
behavior is unchanged (listener.close() already unblocks accept() there).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(daemon): type the idle timeout as timedelta
Review feedback on #215: a bare float-with-suffix-naming carries the
unit only by convention. IdleReaper and run_daemon now take timedelta,
converting to seconds only at the comparison against monotonic
timestamps and on the wire.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQpbhCAsYRJuwL7X83fzDu
* refactor(daemon): drop last_heartbeat tracking — heartbeats are just activity
Review feedback on #215: last_heartbeat is MCP-specific state that the
daemon has no use for — it played no part in idle reaping and existed
only so 'ccc daemon status' could echo it back. A heartbeat's real
effect is the connection itself, which the accept path already records
as activity. Remove the field from IdleReaper, the wire protocol, and
the status output.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQpbhCAsYRJuwL7X83fzDu
* refactor(daemon): send the shutdown wake-up connection unconditionally
POSIX doesn't need it (listener.close() makes accept() raise), but it's
harmless there and keeps the path exercised on every platform instead of
only on Windows CI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vgj6XtKW6dHST75F5VtavC
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): make 'ccc index' auto-initialize project settings as README claims
The README promised auto-init since 9c8a143 but 'ccc index' always hard-errored
without prior 'ccc init'. Auto-create the project half (anchored at the nearest
parent git root); global settings still require interactive 'ccc init'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M4wRn5LoKRrF2mxGqdU7Fk
* fix(cli): run interactive model setup from 'ccc index' when global settings are missing on a TTY
Previously auto-init always hard-errored without global settings. Now, on
an interactive terminal, it runs the same model setup as 'ccc init';
non-interactive runs (scripts, hooks, agents) still exit with an error
rather than silently committing to a default embedding model.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(client): crash-aware restart of a daemon that vanished mid-session
A ConnectionRefusedError after the daemon was already ensured used to be
surfaced as an error, even though a vanished daemon is usually restartable
(manual ccc daemon stop, and soon idle-exit). But blindly restarting would
silently mask daemon crashes and could relaunch a crash-looping daemon
forever. Distinguish the two:
- The daemon's graceful shutdown path (StopRequest, SIGTERM/SIGINT) now
writes a last_exit marker file {pid, reason, timestamp} in the runtime
dir, removed again at the next daemon startup. A crashed daemon (SIGKILL,
OOM, segfault) never writes it.
- HandshakeResponse carries the daemon's pid, so the client can match the
marker against the exact process it was talking to — race-free, no
wall-clock comparisons.
- On a vanished ensured daemon: marker with matching pid → graceful exit →
silent transparent restart. No marker / pid mismatch → crash → restart
once but warn on stderr pointing at the daemon log; after 3 consecutive
crashes raise instead of relaunching. The streak resets whenever the
daemon answers without needing a restart.
The fail-fast for a genuine version mismatch (binary swapped under us
mid-session) is kept unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JaFKwEvJXo7WQhZvrW5Qoe
* fix(tests): avoid signal.SIGKILL on Windows to satisfy mypy
signal.SIGKILL does not exist on Windows, failing the pre-commit mypy
check there. Guard on sys.platform and fall back to SIGTERM, which on
Windows means TerminateProcess — equally abrupt, so the crash-marker
assertion still holds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SeERy25Ct8dMFMVxYSk51c
* refactor(client): use _HandshakeResult field accessors at call sites
Per review feedback on #214: carry the handshake result as one NamedTuple
and access .conn / .resp by name instead of tuple-unpacking, so field
reordering can't silently misalign call sites.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Emit the SearchResponse as JSON on stdout for machine consumers
(e.g. editor plugins shelling out to ccc). On failure the JSON is
still emitted (machine-readable error) and the command exits 1.
Spinner and --refresh progress already render to stderr, so
--json --refresh composes with a clean stdout.
Claude-Session: https://claude.ai/code/session_01JaFKwEvJXo7WQhZvrW5Qoe
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Dart files were excluded from DEFAULT_INCLUDED_PATTERNS, so .dart sources were
not indexed. Add the **/*.dart glob and the README supported-languages row,
mirroring how Svelte/Vue were added in #158.
Closes#183
* feat: add `ccc grep` structural code search
Add a `ccc grep "PATTERN" [DIR/FILE]` subcommand backed by cocoindex's
structural `code_match` API. It compiles the pattern once per language,
walks the project (honoring configured include/exclude globs and nested
.gitignore, or the enclosing git repo when run outside a cocoindex
project), and matches files in parallel on a thread pool, streaming each
file's results as soon as it completes. Supports `--lang` and `--path`
filters like `ccc search`, and renders matches with colorized line
numbers and paths under a TTY.
Extract the shared source-file walking logic (the include/exclude +
nested-gitignore matcher and the os.walk-based file iteration) into a new
`file_walk` module, now the single source of truth used by the indexer,
the daemon's doctor file-walk, and grep.
Bump cocoindex to >=1.0.13 for the locked symmetric pattern syntax used
by `code_match`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(grep): normalize display paths to posix and strip CRLF in output
On Windows the directory walk rendered paths with backslashes (e.g.
`sub\b.py`) and CRLF files left a trailing `\r` on every rendered code
line, which broke two tests. Normalize all display paths via
`Path.as_posix()` — matching the indexer and `ccc search` — and strip the
trailing `\r` when splitting source into lines. Add a CRLF rendering
regression test that runs on all platforms.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(grep): assert posix display path in path_glob test
Completes the Windows path fix: test_grep_path_glob compared fm.path
(now normalized to posix) against a str()-built path, which renders with
backslashes on Windows. Use as_posix() so the assertion is platform
independent, matching the other display-path assertions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix: remove logging.basicConfig from __init__.py
Calling logging.basicConfig() at import time configures the root logger
for the entire Python process, which interferes with logging setups in
applications that use cocoindex-code as a dependency.
Libraries should not configure the root logger — that is the
application's responsibility.
Closes#124
* Update __init__.py
---------
Co-authored-by: Jiangzhou <jiangzhou@cocoindex.io>
Addresses the `ccc init` issues reported in #181:
- Relabel embedding providers so local Ollama is clearly under `litellm`
("litellm (100+ providers — cloud APIs & local Ollama)") and
sentence-transformers is marked as built-in HuggingFace models.
- Reject `ollama/` models inline at the sentence-transformers prompt, before
anything is written or tested, instead of crashing later.
- On a failed init model check, loop with an interactive "try a different
model / keep & finish" choice, pre-filling the previous provider and model
on retry, and print a prominent "Next steps" recovery block.
- Add `ccc doctor -v` to show full tracebacks; by default show the one-line
error plus a hint to rerun with `-v`.
- Fix the retry crash where a rewritten global_settings.yml made the already
-ensured daemon report a bogus "version mismatch": restart the daemon on a
stale-settings handshake even after it was ensured, while still failing fast
on a genuine mid-session version mismatch. Make DaemonVersionError's message
reflect the actual cause.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
When a daemon-side `ccc doctor` check failed (e.g. the model check), only a
one-line, 500-char-truncated summary reached the CLI; the full traceback was
logged to daemon.log but never surfaced, making failures hard to debug.
Carry the full traceback across both daemon→CLI error paths:
- Per-check failures: add `traceback` to `EmbeddingCheckResult` and
`DoctorCheckResult`; `check_embedding` captures `format_exc()` and
`_check_model` propagates it. The CLI prints it dimmed under the error.
- Streaming exceptions: add `traceback` to `ErrorResponse`, populated by the
daemon's streaming handler and appended to the client-raised RuntimeError.
Both new fields default to None, keeping the msgpack wire format
backward-compatible.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump cocoindex (and the sentence-transformers extra) lower bound to 1.0.6,
the first release including cocoindex-io/cocoindex#1992 which adds the
optional `COCOINDEX_APPLICATION_FOR_TRACKING` env var.
Set `COCOINDEX_APPLICATION_FOR_TRACKING=cocoindex-code` at the top of
`cocoindex_code/__init__.py` (before any submodule imports `cocoindex`)
so aggregate telemetry can identify this application. Uses `setdefault`
to leave any explicit user value untouched.
Document the telemetry behavior and `COCOINDEX_DISABLE_USAGE_TRACKING`
opt-out in the README.
Cuts `ccc status` cold-cache startup from ~0.5-0.9s to ~0.15s.
- `cocoindex_code/__init__.py`: replace eager `from .server import main`
with a PEP 562 `__getattr__` so importing the package no longer pulls
`mcp.server.fastmcp` (~300ms). The `cocoindex-code = "cocoindex_code:main"`
console script still resolves.
- `settings.py`: defer `from pathspec import GitIgnoreSpec` into
`load_gitignore_spec()` (only called by indexer/daemon).
- `cli.py`: move protocol type-only imports under `TYPE_CHECKING`
(safe with `from __future__ import annotations`); lazy-import
`DaemonStartError` inside the wrapper and `DoctorCheckResult` next to
the existing lazy `client` import.
Also drops a stale paragraph from CLAUDE.md.
Wire up the new Svelte and Vue tree-sitter languages from cocoindex 1.0.3
(cocoindex-io/cocoindex#1937) so `.svelte` and `.vue` files are picked up
by the default include patterns and chunked syntax-aware.
- Add `**/*.svelte` and `**/*.vue` to DEFAULT_INCLUDED_PATTERNS
- Document svelte/vue in the Supported Languages table
- Bump cocoindex dev-group floor to >=1.0.3 (the version that adds the
tree-sitter parsers); lockfile follows
Voyage models (e.g. voyage/voyage-code-3) reject encoding_format="float"
and require base64. Only inject encoding_format/drop_params for providers
that accept the float hint, leaving voyage/ and bedrock/ to use their
native defaults.
Fixes#148
- Remove `dimensions` from the litellm whitelist in `_ACCEPTED_KWARGS`.
Output dimension must be identical for indexing and query for vectors to
be comparable, so it's a model-wide setting, not a per-side knob —
exposing it under `indexing_params` / `query_params` invited
misconfiguration. Updated comment template, README, design doc, and
testing plan accordingly.
- Plumb `indexing_params` into `create_embedder` and pass them as
constructor kwargs to `PacedLiteLLMEmbedder`. The values land in
`self._kwargs` and become defaults forwarded into every
`litellm.aembedding` call — including paths that don't go through the
`INDEXING_EMBED_PARAMS` context var (e.g. the dim probe in `_get_dim`).
Per-call overrides (`query_params` spread at query time) still win
because `_embed` overlays kwargs on top of `self._kwargs`. Sentence-
transformers ignores `indexing_params` (its constructor doesn't accept
arbitrary kwargs; `prompt_name` is per-call only).
Users can now set `indexing_params` and `query_params` under `embedding:` in
`global_settings.yml` to pass extra kwargs to the embedder separately for
indexing vs. query — supporting asymmetric retrieval models (Cohere v3,
Voyage, Nvidia NIM, Gemini, nomic-ai code/text models, Snowflake arctic,
etc.).
- `ccc init` auto-populates these from a curated table of known models and
prints the applied defaults; unknown models get a commented-out template
for the accepted keys (`prompt_name` for sentence-transformers;
`input_type`, `dimensions` for litellm).
- Daemon validates the effective params at startup; invalid keys fail fast
with a clear error.
- Backward compat: configs for `nomic-ai/CodeRankEmbed` /
`nomic-ai/nomic-embed-code` that predate this feature keep the previous
hardcoded `prompt_name=query` behavior, and a one-time handshake warning
asks users to make the setting explicit. The warning is suppressible by
any non-None `query_params` (including `{}`).
- `ccc doctor` now tests indexing and query separately so asymmetric
misconfigurations surface independently.
Drops the legacy `shared.query_prompt_name` module variable and
`_QUERY_PROMPT_MODELS` set; the new resolution path is centralized in
`embedder_params.resolve_embedder_params` and the curated defaults live in
`embedder_defaults._DEFAULT_PARAMS`.
Also enables `litellm.drop_params = True` so provider-specific kwargs that
a particular model doesn't accept are silently dropped instead of failing.
The module-level `shared.embedder` global was written by `create_embedder()`
but never read from production code — the embedder flows through the
`EMBEDDER` ContextKey via `context.provide` / `use_context`.
Drop the global along with the two test readers that kept it alive:
- `tests/test_daemon.py` — the `daemon_sock` fixture pre-loaded an embedder
and monkeypatched `dm.create_embedder` to reuse it. That was only useful
as a cross-module cache via the now-dead global; with a session-scoped
fixture, `run_daemon()` loads the embedder once from the saved settings
regardless.
- `tests/test_chunker_registry.py` — removed a stale
`monkeypatch.setattr(_shared, "embedder", stub)` whose comment claimed
CodeChunk.embedding read the global at schema resolution time. It reads
the EMBEDDER ContextKey instead; the stub is already wired through
`Project.create(..., stub, ...)`.
LiteLLM documents encoding_format as defaulting to float, but in
practice some providers (e.g. nvidia_nim) error out when it's not
provided. Pass it explicitly as a workaround.
- Bump cocoindex from 1.0.0a43 to >=1.0.0,<1.1.0
- Drop `[tool.uv] prerelease = "explicit"` now that the dep is stable
- Adapt ContextKey usage to 1.0 API: opt-in `detect_change=True`
replaces opt-out `tracked=False`
* perf(docker): split install into stable deps + per-release layers; add GHA cache
Dockerfile previously installed cocoindex-code, cocoindex, torch,
sentence-transformers, and all transitive deps in one RUN. Any change to
the source tree (via COPY . /ccc-src) invalidated that single layer,
forcing a full re-install — ~1 GB of wheels for torch + friends — on
every release. Under QEMU for the arm64 cross-build this was slow
enough to be painful.
Split into two stages:
- `deps`: install cocoindex + cocoindex-code[default] from PyPI. Cache
key is just the RUN command string, so this layer is reused across
releases until we bump the pins.
- `builder`: overlay the release version via
`CCC_INSTALL_SPEC=/ccc-src[default]` with `--no-deps
--force-reinstall` — only the cocoindex-code package is touched; the
heavy deps layer stays untouched.
Also add BuildKit layer cache (`type=gha`) to the publish-docker job so
the deps layer persists across workflow runs, not just within a single
build.
* feat(docker,packaging): slim/full image variants; rename [default]→[full] extra
Build two Docker image variants per release:
- slim (:latest, default) — ~450 MB. LiteLLM-only. cocoindex + cocoindex-code
without sentence-transformers. Targets cloud-backed embeddings.
- full (:full) — ~5 GB. Bundles sentence-transformers + torch +
a pre-baked default model. Targets offline-ready local embeddings.
Dockerfile gains a CCC_VARIANT build arg that gates stage 1's
sentence-transformers install and stage 3's model bake. Release workflow
matrices on {slim, full}; each variant has its own GHA cache scope so
layer reuse works across releases without the variants evicting each
other.
Also rename the PyPI `[default]` umbrella extra to `[full]` so pip and
Docker names match. `[embeddings-local]` remains the canonical primary
extra (the one that specifically pulls in sentence-transformers); `[full]`
is its umbrella alias that may bundle additional optional niceties later.
CLI hints that point at missing sentence-transformers continue to name
`[embeddings-local]` directly — the most specific pointer for that case.
README documents both image variants with a comparison table and narrows
the Mac-on-Docker MPS note to only :full users (slim + LiteLLM is
unaffected).
* feat: unified Docker workspace mount with supervised daemon
Reshape the Docker experience around a single bind mount and a single
named volume. Global settings live on the host under
$HOME/.cocoindex_code/ (visible and editable); index data and the model
cache persist in one cocoindex-data volume; daemon runtime state stays
on the container's native filesystem.
CLI and MCP output now show host-side paths via a bidirectional
COCOINDEX_CODE_HOST_PATH_MAPPING translator. A shell wrapper that
forwards $PWD (COCOINDEX_CODE_HOST_CWD) lets ccc work from any project
subdirectory on the host.
The daemon tolerates a missing global_settings.yml (starts in
no-settings mode) so ccc init's interactive picker works in Docker on
first run. A supervisor restart loop in the entrypoint, driven by a new
COCOINDEX_CODE_DAEMON_SUPERVISED contract, makes settings-change
auto-restart safe — editing global_settings.yml triggers an in-place
daemon respawn without taking the container down.
Linux ownership alignment via PUID/PGID, gosu privilege drop, and a
coco user baked into the image. Release workflow now publishes to both
Docker Hub (cocoindex/cocoindex-code) and GHCR
(ghcr.io/cocoindex-io/cocoindex-code).
Also:
- Merge cocoindex-db and cocoindex-model-cache into a single volume
- find_parent_with_marker requires .cocoindex_code/settings.yml, so a
workspace-root global-only dir doesn't trigger nested-init warnings
- New pytest marker `docker_e2e` gates the Docker-backed E2E suite
(excluded from default pytest runs)
* fix: mypy on Windows for POSIX-only os.getuid/getgid calls
- Move `sentence-transformers` behind `[embeddings-local]` and `[default]`
extras (via `cocoindex[sentence-transformers]`), so `pip install
cocoindex-code` is LiteLLM-only. Closes#117.
- `ccc init` is now interactive when global settings don't exist: pick
provider (sentence-transformers / litellm) and model via a
questionary TUI. New `--litellm-model MODEL` flag skips prompts and
is the non-TTY escape hatch for LiteLLM. Closes#70.
- Change the default sentence-transformers model from
`all-MiniLM-L6-v2` to `Snowflake/snowflake-arctic-embed-xs`
(lighter, better quality for code).
- Generated `global_settings.yml` now includes a `ccc doctor` reminder
and commented-out env-var examples (OPENAI_API_KEY, GEMINI_API_KEY,
ANTHROPIC_API_KEY, VOYAGE_API_KEY).
- Model test during init runs in the daemon via the existing
`DoctorRequest` path; the daemon loads the model once and stays
running, so the user's next `ccc index` starts warm.
- Docker image now installs `cocoindex-code[default]` and pre-caches
the new default model. The `COCOINDEX_CODE_EMBEDDING_MODEL` env var
is no longer documented for Docker; users mount a
`global_settings.yml` or pass `--litellm-model`.
- Extract `check_embedding` + `EmbeddingCheckResult` into `shared.py`;
refactor daemon `_check_model` to delegate. Error messages in doctor
output now include the exception type name (strictly more
informative).
- Tests switch to a lighter `paraphrase-MiniLM-L3-v2` model via a new
`make_test_user_settings()` helper in `conftest.py`, leaving CI
cache costs unchanged.
Update mount_each call to match the new API where component
subpath is passed as a positional argument instead of using a
context manager.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: extract daemon path helpers to avoid CLI importing cocoindex
Move daemon_dir, daemon_socket_path, daemon_pid_path, daemon_log_path,
and connection_family from daemon.py into a new lightweight _daemon_paths.py
module. This prevents the CLI client from transitively importing cocoindex
and its heavy dependencies (numpy, torch, etc.) when it only needs path
utilities.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ci: mark free-threaded Python CI jobs as continue-on-error
tokenizers lacks pre-built wheels for cp314t (free-threaded ABI),
so uv falls back to compiling from Rust source. This source build
is fragile across macOS runner image updates and broke after the
20260406 image bump. Mark 3.14t jobs as continue-on-error since
free-threaded wheel coverage across the ecosystem is still limited.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 19 file extensions from the supported languages table in README.md
that were missing from the default include patterns: Ruby, Swift, Kotlin,
Scala, R, HTML, CSS/SCSS, JSON, XML, YAML, TOML, Solidity, Pascal/Delphi,
DTD, and Fortran.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: enable mypy explicit_package_bases, remove stale type: ignore comments
explicit_package_bases = true resolves module paths from the repo root,
preventing double-discovery of files in tests/ under different module names.
Required for tests/example_toml_chunker.py to be importable as
example_toml_chunker rather than an ambiguous bare module.
Also sets asyncio_mode = auto so pytest-asyncio behaviour is explicit
(default in 1.3.0 is strict).
With explicit_package_bases active, mypy can fully resolve the Response
union in test_daemon.py — the 16 type: ignore[union-attr] and
type: ignore[attr-defined] comments that were suppressing false positives
under the old resolution are now unused and removed.
# Conflicts:
# tests/test_daemon.py
* feat: pluggable chunker registry with settings integration
Improves retrieval precision by letting users split specific file types at
semantic boundaries (e.g. TOML sections, SQL statements) instead of the
default line-window splitter.
## What
- cocoindex_code/chunking.py: public API module exporting ChunkerFn
(Callable alias), CHUNKER_REGISTRY context key, and re-exports of
Chunk/TextPosition from upstream. Single import path for chunker authors.
- tests/example_toml_chunker.py: demo chunker splitting at [section]
headers; excludes [[array_of_tables]] via negative lookahead. Lives in
tests/ to signal it belongs to a separate package, not the core library.
- ProjectSettings.chunkers: new list[ChunkerMapping] field, serialised as
YAML. Each entry maps a file extension to a 'module.path:callable' string.
Users activate chunkers by editing .cocoindex_code/settings.yml — no code
changes required.
- daemon.py: _resolve_chunker_registry resolves ChunkerMapping entries via
importlib at project load time and passes the result to Project.create().
callable() guard gives a clear error at startup rather than a TypeError
per file.
- Project.create(chunker_registry=...): new optional parameter. Injected as
a cocoindex context key (tracked=False) rather than exposed via env internals.
Empty registry by default — zero behavioural delta for existing users.
- indexer.py: process_file checks the registry per file suffix; falls through
to RecursiveSplitter unchanged when no chunker is registered.
## Design decisions
- ChunkerFn returns (language_override, chunks): language_override=None keeps
detect_code_language() result; non-None lets the chunker correct it (e.g.
.sls files starting with #!py).
- tracked=False is consistent with SQLITE_DB, CODEBASE_DIR, and other
non-serialisable context keys. Changing a chunker requires a daemon restart,
which triggers a full re-index anyway.
- _resolve_chunker_registry lives in daemon.py, its only call site, keeping
settings.py as pure schema/IO and chunking.py as pure type definitions.
# Conflicts:
# src/cocoindex_code/daemon.py
# src/cocoindex_code/indexer.py
# src/cocoindex_code/project.py
# tests/test_settings.py
Add target_sqlite_db_path() and cocoindex_db_path() to settings.py as
the single source of truth for database paths, replacing scattered
hardcoded "target_sqlite.db" and "cocoindex.db" strings across cli.py,
daemon.py, project.py, and config.py. Also use daemon_log_path() for
daemon log references and project_settings_path() for settings file
references.
Remove config.py (unused legacy module) and its tests — no production
code imported it.
Add settings/index-db location display to `ccc status`.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add COCOINDEX_CODE_DB_PATH_MAPPING for custom database locations
Allow remapping database file locations via environment variable, enabling
Docker deployments where databases live on the container's native filesystem
while source code is mounted from the host.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use = delimiter for DB path mapping (avoid Windows colon conflict)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: use platform-agnostic tmp_path in DB mapping tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: revisit and simplify CLI-daemon connection model
- CLI-daemon connections are stateless now.
- Also stop caching project settings in memory. Much easier to pick up
new version.
* fix: type annotation
Background indexing during MCP startup shared the same DaemonClient
(multiprocessing connection) as foreground search requests. Concurrent
sends/recvs on the single pipe corrupted data, causing:
Query failed: Input data was truncated
_bg_index() now acquires a fresh DaemonClient via ensure_daemon(),
uses it for indexing, and always closes it in finally. Both MCP
startup paths are fixed (cli.py `ccc mcp` and server.py `cocoindex-code`).
Use CREATE_NO_WINDOW instead of DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP
when starting the daemon on Windows. DETACHED_PROCESS detaches from the
parent console but still creates a new visible console window.
CREATE_NO_WINDOW suppresses the window entirely.
Fixes#100
When the SQLite index table hasn't been created yet, ccc status
raised 'no such table: code_chunks_vec'. Now catches the
OperationalError and reports 'Index not created yet' instead.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dave Clausen