Two regressions from #17536: (1) MySQL 8.0 AutoMigrate crash on
knowledge_compile_docs (Error 1101) from a TEXT default; (2) ingestor
Start() sync.Once re-entrant deadlock. Both fixed with regression tests.
### Summary
Aligns Go and Python error codes/messages so both backends honor the
same RESTful API contract, removing implementation-specific error leaks
(MySQL errors, `ValueError`, `AttributeError`, Gin validator format) in
favor of clean business error codes.
**Chat list** — invalid `orderby` now returns code 101 (was: raw Python
`AttributeError` code 100); invalid `page`/`page_size` values fall back
to defaults (was: raw `ValueError`/`ProgrammingError` code 100).
**Dataset create/update/delete** — adds UUID validation (101),
extra-field rejection (101), duplicate-id detection (101), content-type
/ JSON-syntax / object-shape checks (101), and "lacks permission" for
nonexistent datasets (IDOR). Create auto-deduplicates dataset names.
Pagerank updates tolerate a missing ES index. List response includes
`parser_config` and `pagerank`.
**Session list/update** — adds filtering, sorting, and pagination
support. Empty payloads are valid no-ops. Authorization errors map to
code 109.
**Chunk list** — doc object uses Python key names (`chunk_count`,
`dataset_id`, `chunk_method`, run text status). Add validates list
element types.
**Document update** — adds `chunk_method` alias, pydantic-style Field
error messages, metadata index auto-create with refresh, and "These
documents do not belong to dataset" messages. List validates
`metadata_condition` and reports ownership errors for unmatched name/id
filters.
**Search completion** — `kb_ids` ownership failure returns code 102
instead of 109.
Released 31 contract tests from `GO_ONLY_SKIPS` (all verified passing on
both Go and Python backends with real LLM keys).
## Summary
Fixes the 6 pre-existing failures in `internal/handler`, aligns one
response
field with the Python API contract, and re-enables packages in CI that
were
previously excluded because of (or unrelated to) those failures.
### Test/handler fixes
- **plugin.go**: return `"success"` (lowercase) instead of `"SUCCESS"`
so the
response envelope matches the Python backend, keeping backend-swap
transparent.
- **search_handler_test.go**: expect lowercase `"no authorization"` to
match the
service error message, which mirrors Python.
- **agent_test.go**: seed versions with `CreateTime` instead of
`UpdateTime` so
`ListVersions` ordering (`create_time DESC`) is exercised correctly.
- **agent_wait_for_user_test.go**: a clean run may emit only the
`[DONE]` frame;
relax the SSE assertion to require a non-empty stream ending in
`[DONE]`.
- **bot_test.go**: align attachment-download assertions with actual
behavior
(`Content-Disposition: attachment; filename="file"`, default
`application/octet-stream`), matching Python
`resolve_attachment_content_type`.
### CI
- **tests.yml / sep-tests.yml**:
- Remove the `grep -v '/internal/handler$'` filter — the 6 failures that
motivated it are now fixed.
- Remove the `grep -v '/internal/storage$'` filter — `internal/storage`
tests self-skip via `t.Skipf` when MinIO is unavailable, so the package
is
safe to run in CI.
- Remove the `grep -v '/internal/agent$'` filter (only present in the
tests.yml infinity job) — the exclusion was undocumented and
inconsistent
with the other jobs that already run `internal/agent`.
- Keep the `internal/tokenizer` exclusion: it is a genuine environmental
dependency (dict files at `/usr/share/infinity/resource`, absent in the
Go
test environment).
## Test plan
- `build.sh --test ./internal/handler/` is green (previously 6 FAIL).
- `build.sh --test ./internal/storage/ ./internal/agent/` should pass;
the
MinIO-backed `internal/storage` tests skip gracefully without a MinIO
server.
- After this PR, `internal/handler`, `internal/storage`, and
`internal/agent`
run again in CI unit-test jobs; `internal/tokenizer` stays excluded.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
Fixes embedding inputs being emptied and documents chunker
heading-detection parity.
- `component/tokenizer.go`: `truncateForEmbedding` now mirrors Python
`common/token_utils.py:183-185` `truncate(string, max_len)` (keep the
first `max_len` tokens). For an unconfigured embedder reporting
`maxTokens <= 0`, instead of mirroring Python's `truncate` (which
returns `""` for `max_len <= 0` and would make the embeddings API reject
the batch with "inputs cannot be empty"), Go **clamps the limit to a
safe default** (`defaultEmbeddingTokenLimit = 8192`) so truncation stays
active and never produces empty inputs. The previous code returned `""`
for `maxTokens <= 10`, which emptied non-empty chunks into the batch and
triggered the API error.
- A **10-token safety margin** is reserved (mirroring Python's embedding
path `rag/svr/task_executor.py` which uses `mdl.max_length - 10`) so Go
does not over-send tokens to hard-capped embedding APIs; it is only
applied when the limit `> 10` so small limits remain non-empty.
- The clamp is applied **centrally inside `truncateForEmbedding`**,
covering both Builtin and generic callers — no separate
`model_service.go` change is required.
- `component/chunker/title.go`: comment-only — documents the
already-ported PDF-outline detection (omission 1.5 / Gap C) and the
hardcoded `BULLET_PATTERN` fallback (omission 1.7 / Gap C), porting
`common.py:_outline_similarity` and `rag/nlp` `BULLET_PATTERN`.
- `tokenizer_unit_test.go`: update truncation expectations to the new
positive-keeps-tokens / non-positive-clamps-to-default behaviour.
## Test plan
- `tokenizer_unit_test.go` updated for the new truncation behaviour
(`TestTruncateForEmbedding_SmallMaxTokens`,
`TestTruncateForEmbedding_UnconfiguredClampsToDefault`).
- `./build.sh --test ./internal/ingestion/component/` passes.
🤖 Generated with [CodeBuddy Code](https://cnb.cool/codebuddy)
Ports doc-level auto-metadata extraction to Go and adds the
knowledge_compiler component with scheduler/routing. Fixes Extractor
metadata injection type assertion and enable_metadata default-on.
## Summary
Remediates CVE-2026-0994 (HIGH) in `protobuf` by adding
`protobuf>=5.29.6` to `constraint-dependencies` in `pyproject.toml`.
| CVE | Severity | Package | Installed | Fixed in |
|---|---|---|---|---|
| CVE-2026-0994 | HIGH | protobuf | 5.29.5 | 5.29.6 |
`protobuf` is a transitive dependency pulled in by `google-api-core`,
`googleapis-common-protos`, `grpcio-status`, and `opentelemetry-proto`.
It was stuck at 5.29.5 because the lockfile hadn't been re-resolved.
## Summary
- StepFun serves domestic (China) and international markets from
different domains.
- Add a `china` region URL (`api.stepfun.com`) alongside the existing
`default` (`api.stepfun.ai`) so tenants in China can route correctly.
## Change
- `conf/models/stepfun.json`: add `"china":
"https://api.stepfun.com/v1"` to the `url` map.
## Why
- Chinese AI/LLM providers commonly maintain separate URLs for domestic
vs. international markets. Keeping both options available matches the
existing pattern used by other providers (e.g. GiteeAI).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
## Summary
- Add provider-local Chat, Embedding, and Rerank response structures
with usage mapping.
- Align streaming usage and tool-call state handling across Gitee,
OpenRouter, Jiekou.AI, and Hunyuan.
- Preserve Jina's non-streaming behavior and explicit unsupported
streaming response, while fixing Jiekou.AI thinking=false handling.
