Add a Troubleshooting entry for `MDB_MAP_FULL: Environment mapsize limit
reached`, explaining the 4 GiB LMDB default and how to raise it via
`COCOINDEX_LMDB_MAP_SIZE` (both global_settings.yml `envs:` and shell
forms). Note the value is read at daemon startup and that the workaround
is temporary pending cocoindex#2108. Cross-link from the Large codebase
section.
Rewrite the Contributing section to match the actual dev setup. The
old instructions used pip + upstream pre-commit, but the repo uses
prek (a dev dependency) and CI runs `uv run prek run --all-files`.
New steps: `uv sync` to install dev deps, `uv run prek run
--all-files` to reproduce the full CI gate before opening a PR, and
an optional `uv run prek install` for the per-commit git hook.
* feat: ship as a Claude Code plugin marketplace
Adds `.claude-plugin/marketplace.json` and `.claude-plugin/plugin.json`
so this repo can be installed directly from inside Claude Code:
/plugin marketplace add cocoindex-io/cocoindex-code
/plugin install cocoindex-code@cocoindex-code
The existing `skills/ccc/SKILL.md` is auto-discovered as the plugin's
skill — no file moves, the `npx skills add` workflow keeps working
unchanged. README gets a short subsection under "Skill (Recommended)"
linking the new install path.
Refs: https://code.claude.com/docs/en/plugin-marketplaces
* chore: point plugin manifests to Roxabi fork
Update owner, homepage, repository, and README install command
from cocoindex-io to Roxabi/cocoindex-code.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Bump cocoindex (and the sentence-transformers extra) lower bound to 1.0.6,
the first release including cocoindex-io/cocoindex#1992 which adds the
optional `COCOINDEX_APPLICATION_FOR_TRACKING` env var.
Set `COCOINDEX_APPLICATION_FOR_TRACKING=cocoindex-code` at the top of
`cocoindex_code/__init__.py` (before any submodule imports `cocoindex`)
so aggregate telemetry can identify this application. Uses `setdefault`
to leave any explicit user value untouched.
Document the telemetry behavior and `COCOINDEX_DISABLE_USAGE_TRACKING`
opt-out in the README.
Add an "OpenAI-compatible (custom endpoint)" section under Embedding
Models showing the `openai/` prefix plus `OPENAI_BASE_URL` /
`OPENAI_API_KEY` env vars, and link the LiteLLM "Setting API Keys" doc
from the general LiteLLM note so users of any provider can find the
canonical env-var list.
Wire up the new Svelte and Vue tree-sitter languages from cocoindex 1.0.3
(cocoindex-io/cocoindex#1937) so `.svelte` and `.vue` files are picked up
by the default include patterns and chunked syntax-aware.
- Add `**/*.svelte` and `**/*.vue` to DEFAULT_INCLUDED_PATTERNS
- Document svelte/vue in the Supported Languages table
- Bump cocoindex dev-group floor to >=1.0.3 (the version that adds the
tree-sitter parsers); lockfile follows
- Remove `dimensions` from the litellm whitelist in `_ACCEPTED_KWARGS`.
Output dimension must be identical for indexing and query for vectors to
be comparable, so it's a model-wide setting, not a per-side knob —
exposing it under `indexing_params` / `query_params` invited
misconfiguration. Updated comment template, README, design doc, and
testing plan accordingly.
- Plumb `indexing_params` into `create_embedder` and pass them as
constructor kwargs to `PacedLiteLLMEmbedder`. The values land in
`self._kwargs` and become defaults forwarded into every
`litellm.aembedding` call — including paths that don't go through the
`INDEXING_EMBED_PARAMS` context var (e.g. the dim probe in `_get_dim`).
Per-call overrides (`query_params` spread at query time) still win
because `_embed` overlays kwargs on top of `self._kwargs`. Sentence-
transformers ignores `indexing_params` (its constructor doesn't accept
arbitrary kwargs; `prompt_name` is per-call only).
Users can now set `indexing_params` and `query_params` under `embedding:` in
`global_settings.yml` to pass extra kwargs to the embedder separately for
indexing vs. query — supporting asymmetric retrieval models (Cohere v3,
Voyage, Nvidia NIM, Gemini, nomic-ai code/text models, Snowflake arctic,
etc.).
- `ccc init` auto-populates these from a curated table of known models and
prints the applied defaults; unknown models get a commented-out template
for the accepted keys (`prompt_name` for sentence-transformers;
`input_type`, `dimensions` for litellm).
- Daemon validates the effective params at startup; invalid keys fail fast
with a clear error.
- Backward compat: configs for `nomic-ai/CodeRankEmbed` /
`nomic-ai/nomic-embed-code` that predate this feature keep the previous
hardcoded `prompt_name=query` behavior, and a one-time handshake warning
asks users to make the setting explicit. The warning is suppressible by
any non-None `query_params` (including `{}`).
- `ccc doctor` now tests indexing and query separately so asymmetric
misconfigurations surface independently.
Drops the legacy `shared.query_prompt_name` module variable and
`_QUERY_PROMPT_MODELS` set; the new resolution path is centralized in
`embedder_params.resolve_embedder_params` and the curated defaults live in
`embedder_defaults._DEFAULT_PARAMS`.
Also enables `litellm.drop_params = True` so provider-specific kwargs that
a particular model doesn't accept are silently dropped instead of failing.
Now that cocoindex 1.0.0 is stable, we no longer need `--prerelease`
or explicit prerelease version pins in install instructions.
- README.md: simplify `uv tool install` commands
- docker/Dockerfile: drop `--prerelease=allow` and the redundant
`cocoindex>=1.0.0a33` pin (pyproject.toml already constrains
`cocoindex[litellm]>=1.0.0,<1.1.0`)
- README: bash/zsh one-liner using `docker compose -f <(curl ...)`,
plus keep the clone-the-file path for non-POSIX shells.
- docker-compose.yml: `image:` now reads `${COCOINDEX_CODE_IMAGE:-...}`
so users can switch to `:full` or GHCR without editing the file.
- pyproject: scope mypy to `src/` and collapse single-entry extras arrays.
* perf(docker): split install into stable deps + per-release layers; add GHA cache
Dockerfile previously installed cocoindex-code, cocoindex, torch,
sentence-transformers, and all transitive deps in one RUN. Any change to
the source tree (via COPY . /ccc-src) invalidated that single layer,
forcing a full re-install — ~1 GB of wheels for torch + friends — on
every release. Under QEMU for the arm64 cross-build this was slow
enough to be painful.
Split into two stages:
- `deps`: install cocoindex + cocoindex-code[default] from PyPI. Cache
key is just the RUN command string, so this layer is reused across
releases until we bump the pins.
- `builder`: overlay the release version via
`CCC_INSTALL_SPEC=/ccc-src[default]` with `--no-deps
--force-reinstall` — only the cocoindex-code package is touched; the
heavy deps layer stays untouched.
Also add BuildKit layer cache (`type=gha`) to the publish-docker job so
the deps layer persists across workflow runs, not just within a single
build.
* feat(docker,packaging): slim/full image variants; rename [default]→[full] extra
Build two Docker image variants per release:
- slim (:latest, default) — ~450 MB. LiteLLM-only. cocoindex + cocoindex-code
without sentence-transformers. Targets cloud-backed embeddings.
- full (:full) — ~5 GB. Bundles sentence-transformers + torch +
a pre-baked default model. Targets offline-ready local embeddings.
Dockerfile gains a CCC_VARIANT build arg that gates stage 1's
sentence-transformers install and stage 3's model bake. Release workflow
matrices on {slim, full}; each variant has its own GHA cache scope so
layer reuse works across releases without the variants evicting each
other.
Also rename the PyPI `[default]` umbrella extra to `[full]` so pip and
Docker names match. `[embeddings-local]` remains the canonical primary
extra (the one that specifically pulls in sentence-transformers); `[full]`
is its umbrella alias that may bundle additional optional niceties later.
CLI hints that point at missing sentence-transformers continue to name
`[embeddings-local]` directly — the most specific pointer for that case.
README documents both image variants with a comparison table and narrows
the Mac-on-Docker MPS note to only :full users (slim + LiteLLM is
unaffected).
* feat: unified Docker workspace mount with supervised daemon
Reshape the Docker experience around a single bind mount and a single
named volume. Global settings live on the host under
$HOME/.cocoindex_code/ (visible and editable); index data and the model
cache persist in one cocoindex-data volume; daemon runtime state stays
on the container's native filesystem.
CLI and MCP output now show host-side paths via a bidirectional
COCOINDEX_CODE_HOST_PATH_MAPPING translator. A shell wrapper that
forwards $PWD (COCOINDEX_CODE_HOST_CWD) lets ccc work from any project
subdirectory on the host.
The daemon tolerates a missing global_settings.yml (starts in
no-settings mode) so ccc init's interactive picker works in Docker on
first run. A supervisor restart loop in the entrypoint, driven by a new
COCOINDEX_CODE_DAEMON_SUPERVISED contract, makes settings-change
auto-restart safe — editing global_settings.yml triggers an in-place
daemon respawn without taking the container down.
Linux ownership alignment via PUID/PGID, gosu privilege drop, and a
coco user baked into the image. Release workflow now publishes to both
Docker Hub (cocoindex/cocoindex-code) and GHCR
(ghcr.io/cocoindex-io/cocoindex-code).
Also:
- Merge cocoindex-db and cocoindex-model-cache into a single volume
- find_parent_with_marker requires .cocoindex_code/settings.yml, so a
workspace-root global-only dir doesn't trigger nested-init warnings
- New pytest marker `docker_e2e` gates the Docker-backed E2E suite
(excluded from default pytest runs)
* fix: mypy on Windows for POSIX-only os.getuid/getgid calls
- Move `sentence-transformers` behind `[embeddings-local]` and `[default]`
extras (via `cocoindex[sentence-transformers]`), so `pip install
cocoindex-code` is LiteLLM-only. Closes#117.
- `ccc init` is now interactive when global settings don't exist: pick
provider (sentence-transformers / litellm) and model via a
questionary TUI. New `--litellm-model MODEL` flag skips prompts and
is the non-TTY escape hatch for LiteLLM. Closes#70.
- Change the default sentence-transformers model from
`all-MiniLM-L6-v2` to `Snowflake/snowflake-arctic-embed-xs`
(lighter, better quality for code).
- Generated `global_settings.yml` now includes a `ccc doctor` reminder
and commented-out env-var examples (OPENAI_API_KEY, GEMINI_API_KEY,
ANTHROPIC_API_KEY, VOYAGE_API_KEY).
- Model test during init runs in the daemon via the existing
`DoctorRequest` path; the daemon loads the model once and stays
running, so the user's next `ccc index` starts warm.
- Docker image now installs `cocoindex-code[default]` and pre-caches
the new default model. The `COCOINDEX_CODE_EMBEDDING_MODEL` env var
is no longer documented for Docker; users mount a
`global_settings.yml` or pass `--litellm-model`.
- Extract `check_embedding` + `EmbeddingCheckResult` into `shared.py`;
refactor daemon `_check_model` to delegate. Error messages in doctor
output now include the exception type name (strictly more
informative).
- Tests switch to a lighter `paraphrase-MiniLM-L3-v2` model via a new
`make_test_user_settings()` helper in `conftest.py`, leaving CI
cache costs unchanged.
- docker/Dockerfile — multi-stage build (builder → model_cache →
runtime);
copies both `cocoindex-code` and `ccc` binaries; sets
COCOINDEX_CODE_DB_PATH_MAPPING=/workspace=/db so index databases live
in the container's native filesystem (avoids slow cross-OS volume I/O)
- docker/entrypoint.sh — creates user settings on first start via
`ccc init`, then runs `ccc run-daemon` in the foreground to keep the
container alive as a persistent daemon
- README — adds Docker section: persistent container as primary pattern
(`docker run -d` + `docker exec`), named volumes for DB and model cache,
Claude Code / Codex MCP configuration via `docker exec -i ... ccc mcp`
* chore: enable mypy explicit_package_bases, remove stale type: ignore comments
explicit_package_bases = true resolves module paths from the repo root,
preventing double-discovery of files in tests/ under different module names.
Required for tests/example_toml_chunker.py to be importable as
example_toml_chunker rather than an ambiguous bare module.
Also sets asyncio_mode = auto so pytest-asyncio behaviour is explicit
(default in 1.3.0 is strict).
With explicit_package_bases active, mypy can fully resolve the Response
union in test_daemon.py — the 16 type: ignore[union-attr] and
type: ignore[attr-defined] comments that were suppressing false positives
under the old resolution are now unused and removed.
# Conflicts:
# tests/test_daemon.py
* feat: pluggable chunker registry with settings integration
Improves retrieval precision by letting users split specific file types at
semantic boundaries (e.g. TOML sections, SQL statements) instead of the
default line-window splitter.
## What
- cocoindex_code/chunking.py: public API module exporting ChunkerFn
(Callable alias), CHUNKER_REGISTRY context key, and re-exports of
Chunk/TextPosition from upstream. Single import path for chunker authors.
- tests/example_toml_chunker.py: demo chunker splitting at [section]
headers; excludes [[array_of_tables]] via negative lookahead. Lives in
tests/ to signal it belongs to a separate package, not the core library.
- ProjectSettings.chunkers: new list[ChunkerMapping] field, serialised as
YAML. Each entry maps a file extension to a 'module.path:callable' string.
Users activate chunkers by editing .cocoindex_code/settings.yml — no code
changes required.
- daemon.py: _resolve_chunker_registry resolves ChunkerMapping entries via
importlib at project load time and passes the result to Project.create().
callable() guard gives a clear error at startup rather than a TypeError
per file.
- Project.create(chunker_registry=...): new optional parameter. Injected as
a cocoindex context key (tracked=False) rather than exposed via env internals.
Empty registry by default — zero behavioural delta for existing users.
- indexer.py: process_file checks the registry per file suffix; falls through
to RecursiveSplitter unchanged when no chunker is registered.
## Design decisions
- ChunkerFn returns (language_override, chunks): language_override=None keeps
detect_code_language() result; non-None lets the chunker correct it (e.g.
.sls files starting with #!py).
- tracked=False is consistent with SQLITE_DB, CODEBASE_DIR, and other
non-serialisable context keys. Changing a chunker requires a daemon restart,
which triggers a full re-index anyway.
- _resolve_chunker_registry lives in daemon.py, its only call site, keeping
settings.py as pure schema/IO and chunking.py as pure type definitions.
# Conflicts:
# src/cocoindex_code/daemon.py
# src/cocoindex_code/indexer.py
# src/cocoindex_code/project.py
# tests/test_settings.py
* feat: add COCOINDEX_CODE_DB_PATH_MAPPING for custom database locations
Allow remapping database file locations via environment variable, enabling
Docker deployments where databases live on the container's native filesystem
while source code is mounted from the host.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use = delimiter for DB path mapping (avoid Windows colon conflict)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: use platform-agnostic tmp_path in DB mapping tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add .lua files to DEFAULT_INCLUDED_PATTERNS to enable automatic
indexing of Lua code. The underlying cocoindex library already
supports Lua language detection, this change makes it available
by default without requiring COCOINDEX_CODE_EXTRA_EXTENSIONS.
Changes:
- Add **/*.lua to DEFAULT_INCLUDED_PATTERNS in indexer.py
- Add Lua to supported languages table in README.md
* feat: allow adding file extensions via COCOINDEX_CODE_EXTRA_EXTENSIONS env var
Users can now specify additional file extensions to index without editing
source code, e.g. COCOINDEX_CODE_EXTRA_EXTENSIONS="inc,tpl"
* feat: support language mapping in COCOINDEX_CODE_EXTRA_EXTENSIONS
Allow `ext:lang` format (e.g. "inc:php,yaml,tpl:html") to override
language detection for extra extensions, enabling proper AST-based
chunking for unrecognized extensions.
Jiangzhou