## What this PR does
Removes the self-concatenation of the vision model response in the video
parsing path, so each generated video description is tokenized and
indexed exactly once.
A focused regression test exercises the public `picture.chunk` video
path with a mocked vision model and asserts that the returned
description is passed to `tokenize` once without duplication.
## Root cause
The original video parsing implementation used:
```python
ans += "\n" + ans
tokenize(doc, ans, ...)
```
This duplicates the same model response. The adjacent image path
combines two distinct values (`OCR text + vision description`); the
video path has only the model response, so concatenating it with itself
is an unintended copy/paste error from that image logic.
## Impact
Before this fix, every successfully parsed video stored repeated text,
increasing token and embedding input and potentially distorting indexed
chunk content and retrieval scoring.
## Compatibility
The change affects only the video branch in `rag/app/picture.py`. Image
parsing, model invocation, prompts, callbacks, and error handling remain
unchanged.
## Validation
- `pytest --confcutdir=test/unit_test/rag/app
test/unit_test/rag/app/test_picture_video.py -q`: 1 passed
- Ruff check: passed
- Ruff format check for the new test: passed
- `git diff --check`: passed
Closes#16846.
---------
Co-authored-by: openhands <openhands@all-hands.dev>
## What this PR does
Adds support for Alibaba Cloud's hosted Fun-ASR-Flash snapshots to the
existing Tongyi-Qianwen speech-to-text provider.
- registers `fun-asr-flash-2026-06-15` as a speech-to-text model;
- routes only `fun-asr-flash*` models to the documented workspace-native
multimodal-generation endpoint;
- supports local audio through size-checked data URIs as well as
URL/data-URI inputs;
- uses the documented SSE response mode for incremental streaming
transcription;
- closes the streamed HTTP response on completion, failure, or early
consumer cancellation;
- preserves the existing `dashscope.MultiModalConversation` path for all
other Qwen audio models;
- keeps RAGFlow's existing synchronous and streaming adapter interfaces.
## Why
Fun-ASR-Flash does not use the legacy Qwen audio request shape currently
used by `QWenSeq2txt`. Its synchronous API expects `input_audio` at:
`/api/v1/services/aigc/multimodal-generation/generation`
Without a narrowly scoped adapter path, the hosted model cannot be
selected successfully through RAGFlow's Tongyi-Qianwen speech-to-text
provider.
Closes#16843.
## Compatibility
The new behavior is gated by the `fun-asr-flash` model-name prefix.
Existing Qwen audio models continue through the original code path
unchanged.
## Validation
- `pytest test/unit_test/rag/llm/test_sequence2txt_model.py`: 10 passed
- Ruff check: passed
- Ruff format check: passed
- `llm_factories.json` validation: passed
- Real hosted-API validation with WAV audio
- Real RAGFlow upload/indexing validation with MP3 audio
The unit tests cover the native Fun-ASR-Flash request, regression
behavior for the legacy Qwen path, SSE streaming, and early response
cleanup.
## Documentation
- https://help.aliyun.com/document_detail/2979031.html
- https://help.aliyun.com/document_detail/2869541.html
### Why a dedicated adapter path is necessary (official evidence)
Alibaba Cloud's [Fun-ASR RESTful API
reference](https://help.aliyun.com/en/model-studio/fun-asr-recorded-speech-recognition-http-api)
makes the incompatibilities with RAGFlow's existing Qwen audio path
explicit:
| Adapter change | Official API requirement | Why the existing path is
insufficient |
| --- | --- | --- |
| Call the workspace-native HTTP endpoint | The Fun-ASR-Flash
synchronous section states that SDK calls are not supported and
specifies `POST /api/v1/services/aigc/multimodal-generation/generation`.
| The existing adapter calls `dashscope.MultiModalConversation`, so a
direct HTTP path is required. |
| Use the `input_audio` message shape | `input.messages`, `content`,
`type: input_audio`, `input_audio`, and `input_audio.data` are
documented as required for an audio request. | The existing Qwen path
sends the legacy `audio` content shape, which does not match this API
contract. |
| Send `parameters.format` | The request schema marks `parameters` and
`format` as **Required**, and says the value must match the actual audio
format. | The legacy request has no Fun-ASR-Flash `parameters.format`
field, so the adapter must derive and send it. |
| Encode local files as Data URIs | `input_audio.data` accepts either a
public URL or a Base64 Data URI; the reference gives the exact
`data:{MIME_TYPE};base64,...` form. | RAGFlow supplies local file paths,
which the remote API cannot read directly. |
| Parse `output.text` | The documented non-streaming response returns
the accumulated transcription in `output.text`. | The legacy Qwen
response parser reads `output.choices[].message.content`, so a separate
response parser is required. |
| Enforce the Base64 input limit | The reference requires the
Base64-encoded audio to remain within the 10 MB input limit. | The
adapter checks encoded size before reading/sending local audio and
directs oversized inputs to the existing public-URL path. |
| Use SSE for streaming | The reference specifies `X-DashScope-SSE:
enable` and documents intermediate and final SSE events. | The adapter
parses those events instead of wrapping one blocking response as a
synthetic stream. |
| Release streamed responses | Streaming responses must be closed when
iteration completes or stops early. | A `finally` cleanup releases the
HTTP response on completion, errors, and consumer cancellation. |
`sample_rate` is documented as **Optional**. The implementation omits it
instead of declaring a fixed value that may not match remote or
compressed audio.
The [official speech-to-text model
list](https://help.aliyun.com/en/model-studio/asr-model/) separately
confirms that `fun-asr-flash-2026-06-15` is an offline HTTP model with a
five-minute audio limit.
---------
Signed-off-by: LauraGPT <LauraGPT@users.noreply.github.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
### Summary
We have updated our model driver to work with go.
It is based on OpenAI-API-Compatible model provider.
Draft
#15519
Our old model provider
#13425
### Summary
Adds FunASR as a self-hosted speech-to-text provider through its
OpenAI-compatible `/v1/audio/transcriptions` endpoint.
This is a focused replacement for #15526 by @Rene0422 and relates to
#15448. The unrelated Markdown parser changes from the previous branch
are intentionally removed so this PR contains only the FunASR provider
integration.
- register FunASR as a `SPEECH2TEXT` factory;
- add `FunASRSeq2txt` with `sensevoice` and `http://localhost:8000/v1`
defaults, an optional API key, URL normalization, and inherited
transcription handling;
- wire FunASR into the current local-provider schema with a prefilled
local URL and official documentation link;
- discover the server's `/v1/models` dynamically and expose every
returned model as speech-to-text in the model picker;
- use RAGFlow's existing default provider icon fallback instead of
referencing a missing `funasr` asset;
- list FunASR in the supported-provider documentation;
- add focused backend and frontend regression tests.
### Validation
- focused backend pytest suite -> `7 passed`
- real CPU `funasr-server` + RAGFlow provider smoke test -> discovered
`fun-asr-nano`, `sensevoice`, and `paraformer`; transcribed a real WAV
as `我现在在录一段测试音频` (`10` tokens, `0.504s`)
- `ruff check` and `ruff format --check` on the changed Python files
- `python3 -m py_compile` on the provider and its test
- JSON parse and a semantic assertion for exactly one enabled FunASR
`SPEECH2TEXT` factory
- focused frontend Jest test -> `2 passed`
- ESLint and Prettier on all changed TypeScript files
- `npm run build` -> production build succeeded (`14,181` modules
transformed)
- `git diff --check`
### Deployment
Run FunASR separately and point the RAGFlow provider at it:
```bash
pip install funasr
funasr-server --device cuda --model sensevoice
```
The API key remains optional because the stock local server does not
require authentication. A key can still be supplied when the endpoint is
protected by a gateway.
---------
Signed-off-by: LauraGPT <LauraGPT@users.noreply.github.com>
Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
### What problem does this PR solve?
Improve concurrency in the knowledge compilation pipeline:
- Run Compile LLM requests concurrently while preserving ordered
commits.
- Run merge flush tasks concurrently while keeping ES writes ordered.
- Improve concurrency for local deduplication, chain validation, and ES
deduplication.
- Remove temporary debugging instrumentation and unused timing
variables.
### Type of change
- [x] Refactor (no functional change)
Fixes#16812
### Problem
In the `rag/flow` ingestion pipeline, when `TitleChunker` feeds
`TokenChunker`, the chapter-aware chunks are silently discarded and the
parser's raw flat json is re-chunked instead.
`TitleChunker` emits `output_format="chunks"` and writes its
chapter-aware output to the `chunks` field
(`rag/flow/chunker/title_chunker/common.py`,
`set_output("output_format", "chunks")`). But `TokenChunker._invoke`
only handles `output_format` in `["markdown", "text", "html"]`, then
falls through to the `# json` path which reads
`from_upstream.json_result`. There is no branch for `"chunks"`, so
`from_upstream.chunks` is never read.
Downstream effects reported in #16812: PageIndex/TOC extraction receives
flat line-level text instead of structured chapter blocks
(incorrect/duplicate/missing chapters), and retrieval quality degrades
because chunks are no longer aligned to document structure.
### Fix
Select the source list based on `output_format`, mirroring the exact
pattern already used in `title_chunker/common.py`:
```python
json_result = (from_upstream.chunks if from_upstream.output_format == "chunks" else from_upstream.json_result) or []
```
`chunks` items share the same dict shape as `json_result` items (both
consumed via `.get("text")`, `.get("doc_type_kwd")`, etc.), so they flow
through the existing token-sizing path unchanged. One-line change, no
behavior change for the `json`/`markdown`/`text`/`html` paths.
### Test
Adds `rag/flow/tests/test_token_chunker.py`, an isolated unit test that
runs the real `TokenChunker._invoke` (heavy deps stubbed; real pydantic
schema used when available) and asserts that with
`output_format="chunks"` the upstream `chunks` are consumed rather than
the raw parser `json`.
Verified RED -> GREEN: the test fails against the current code (reads
the raw json) and passes with the fix.
Signed-off-by: Yash Raj Pandey <yashpn62@gmail.com>
### What problem does this PR solve?
- Clear stale pipeline IDs and generated data when updating documents
without `pipeline_id`.
- Support tree compilation results in pipeline workflows.
- Update compilation templates in place while preserving existing
template IDs.
- Improve duplicate-template validation messages.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
## Summary
- Add language-aware Snowball stemmer to `RagTokenizer` supporting 16
languages (Dutch, German, French, Spanish, etc.)
- Thread the KB `language` parameter through the full tokenization
pipeline (14 parser modules + task executor)
- Add Dutch to the frontend language lists and cross-language form
## Problem
RAGFlow uses the English Porter stemmer + WordNet lemmatizer for **all**
BM25 tokenization, regardless of the knowledge base language setting.
This produces incorrect stems for non-English text. For example:
| Dutch word | Dutch stemmer | English Porter |
|---|---|---|
| documenten | document | documenten (unchanged!) |
| gebruikers | gebruiker | gebruik (over-stemmed) |
| instellingen | instell | instellingen (unchanged!) |
This degrades BM25 recall for any non-English knowledge base.
## Solution
NLTK already ships Snowball stemmers for 16 languages. This PR:
1. **`rag/nlp/rag_tokenizer.py`**: Overrides `tokenize()` with
`set_language()` and `_normalize_token()` that selects the correct NLTK
Snowball stemmer. Falls back to Porter for unmapped languages (Chinese,
Japanese, Korean, etc. — these use character-based tokenization anyway).
2. **`rag/nlp/__init__.py`** + **14 `rag/app/*.py` parsers** +
**`rag/svr/task_executor.py`**: Threads the `language` parameter through
`tokenize()`, `tokenize_chunks()`, `tokenize_table()`, and all callers.
3. **Frontend**: Adds Dutch (`Nederlands`) to `LanguageList`,
`LanguageMap`, `LanguageAbbreviationMap`, `LanguageTranslationMap`,
cross-language form field, and `en.ts` locale.
## Backward Compatibility
- Default language is `"English"`, preserving existing behavior for all
current users
- Languages without a Snowball stemmer mapping fall back to Porter (no
change)
- No new dependencies — NLTK Snowball is already bundled
## Problem
`raptor.py` computes `n_neighbors = int((len(embeddings) - 1) ** 0.8)`
and
passes it to `umap.UMAP(...)`. In a dataset-scope RAPTOR build the first
layer's `embeddings` is the entire KB's chunk set, so this is
effectively
unbounded: ~93k chunks → n_neighbors ≈ 9,446.
UMAP's k-NN graph is `N × n_neighbors`; at these values the raw neighbor
arrays alone are ~14 GB (93k × 9446 × 16 B), and the symmetrized fuzzy
simplicial set + spectral init push peak well past 30 GB. The task
executor is OOM-killed inside `fit_transform` before any clustering runs
—
the log shows "Task has been received" with no "Cluster one layer" line
—
after which the unacked task re-queues and OOMs again in a loop.
The line above already flags this: `# Degrade too much ??`.
## Fix
Cap `n_neighbors` at 100. UMAP's neighborhood size has strongly
diminishing returns well below this (default 15; a few dozen already
captures global structure), so the ceiling preserves — likely improves —
cluster quality while bounding memory to O(N). Mirrors the existing
`n_components=min(12, len(embeddings) - 2)` clamp two lines down.
```diff
- n_neighbors = int((len(embeddings) - 1) ** 0.8)
+ n_neighbors = min(int((len(embeddings) - 1) ** 0.8), 100)
```
## Repro
Dataset-scope RAPTOR over a KB with ~90k+ chunks on a box with <~64 GB
available: executor OOM-killed in the first-layer UMAP `fit_transform`.
With the cap, first-layer UMAP peaks in low single-digit GB and the
build
proceeds to completion.
## Scope
Only affects large dataset-scope builds; file-scope RAPTOR already had
n_neighbors well under 100. No behavior change beyond the ceiling.
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe):
## Summary
Agent (Canvas) runs previously did not surface token usage in the SSE
stream, and RAGFlow's own Langfuse generations for agent runs were
missing the prompt/completion split and the session/user correlation.
This made it impossible for an external caller (or Langfuse) to
reconcile an agent turn's cost with the upstream provider (e.g.
OpenRouter), because a single turn can issue several distinct LLM calls
(query rewriting / cross-language translation, multi-round tool
reasoning, nested sub-agents, and the final answer).
This PR introduces a per-run token usage sink so that **every** LLM call
in a run is aggregated and reported once, and enriches Langfuse
generations with the prompt/completion split plus session/user
attributes.
## What changes
### 1. Per-run token usage sink (`common/token_utils.py`)
- Adds two `contextvars`: `token_usage_sink` (a mutable per-run
accumulator) and `langfuse_run_attrs` (session_id/user_id for the run).
- Adds `record_run_token_usage(...)` (thread-safe via a lock, because
`thread_pool_exec` copies the context into worker threads that share the
sink dict) and `usage_from_response(...)` which extracts a
`{prompt_tokens, completion_tokens, total_tokens}` split from
OpenAI/OpenRouter-style responses.
### 2. Provider layer captures the prompt/completion split
(`rag/llm/chat_model.py`)
- `LiteLLMBase` and `Base` now store `self.last_usage`
(prompt/completion/total) for the most recent chat call, in both the
plain and tool-calling paths.
- Streaming requests set `stream_options.include_usage = True` (LiteLLM
path) so the authoritative usage arrives on the final chunk; this is
read even on the usage-only chunk that carries no `choices`.
- Fixes a multi-round accounting bug in `*_with_tools`: token totals
were **overwritten** by each round (`total_tokens = tol`) instead of
accumulated, undercounting multi-round tool conversations. Each round is
now committed to a running aggregate.
### 3. LLMBundle reports usage once, per call
(`api/db/services/llm_service.py`)
- New `_report_usage(total_tokens)` records the call's usage into the
active run sink and returns the prompt/completion/total split for
Langfuse. The split is only used when it is consistent with the
authoritative total; otherwise only the total is reported.
- All three chat entry points (`async_chat`, `async_chat_streamly`,
`async_chat_streamly_delta`) now emit `usage_details` with
`input`/`output`/`total` instead of total-only.
- `_start_langfuse_observation` now applies `session_id`/`user_id` from
the per-run context (`langfuse_run_attrs`) so agent-run generations are
correctly grouped, even though agent LLMBundles are constructed without
those attributes.
### 4. Canvas installs the sink and emits the aggregate
(`agent/canvas.py`)
- `Canvas.run()` installs a fresh `token_usage_sink` and
`langfuse_run_attrs` (from `user_id`/`session_id`) at the start of every
turn.
- `message_end` now includes an aggregated `usage` object:
`{prompt_tokens, completion_tokens, total_tokens, calls}` covering all
LLM calls in the run.
### 5. Pass session id into the run
(`api/db/services/canvas_service.py`)
- `completion()` forwards `session_id` to `Canvas.run()` for Langfuse
session correlation.
## Why a context variable
LLM calls in an agent run originate from many places that each build
their own `LLMBundle` (e.g. `cross_languages`/`keyword_extraction`
helpers, the Agent component, and nested sub-agents invoked as tools). A
run-scoped context variable is the only non-invasive chokepoint that
captures all of them exactly once, including nested agents (which run in
the same async context) and thread-pool tools (the executor copies the
context).
## Behavior / compatibility
- No public API or wire-format removal: `message_end` gains an
additional optional `usage` field; existing consumers are unaffected.
- When a provider does not return authoritative usage, behavior falls
back to the previous token estimate (total only, no split).
- Non-agent flows (Dataflow `Pipeline`, sync `Graph.run`) are untouched.
## Testing
- [x] Simple agent answer: `message_end.usage.total_tokens` matches
provider usage.
- [x] Agent with cross-language retrieval: aggregate equals the sum of
both provider calls.
- [x] Tool-calling agent (multi-round): total accumulates across rounds.
- [x] Nested agent (agent-as-tool): sub-agent tokens included in the
parent run total.
- [x] Langfuse: agent generations show input/output split and are
grouped by session/user.
---------
Co-authored-by: yzc <yuzhichang@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
## Problem
When building or updating a knowledge graph with a large number of
entities and edges, `set_graph()` in `rag/graphrag/utils.py` creates one
`asyncio` task per entity and one per edge, each calling
`embd_mdl.encode([single_name])` — a single-item HTTP request to the
embedding server.
For a graph with 17,000+ nodes and edges (real case reported in #16205),
this generates **34,000+ individual embedding API round-trips** instead
of ~266 batched calls at the default `_INSERT_BULK_SIZE=128`. The
asyncio gather over thousands of tasks makes the embedding server the
bottleneck; under load, a single slow/failed call aborts all remaining
tasks, causing the pipeline to stall and never complete.
Closes#16205. Related: #15921.
## Root Cause
```python
# Before (in set_graph, node loop):
tasks = [asyncio.create_task(graph_node_to_chunk(n, ...)) for n in nodes]
# Each task calls embd_mdl.encode([single_name]) — 1 HTTP call per node
```
`graph_node_to_chunk` checks the embed cache first, but the cache is
cold on first build, so every task makes a live API call.
## Fix
Pre-warm the embedding cache with batched calls before spawning tasks.
Each batch pre-warm calls `embd_mdl.encode(batch_of_128)` once,
populating the cache. Then every individual task hits the cache and
makes zero embedding API calls.
- Only encodes names not already in cache (no-op on warm cache / small
incremental updates)
- Uses existing project idioms: `thread_pool_exec`, `chat_limiter`,
`_INSERT_BULK_SIZE`, `get_embed_cache`, `set_embed_cache`
- Mirrors the `ENABLE_TIMEOUT_ASSERTION` timeout pattern from
`graph_node_to_chunk`
- Zero behavior change: per-task encode logic remains as a correct
fallback
## Result
| Graph size | Before | After |
|---|---|---|
| 17,576 edges | ~17,576 embedding calls → stall | ~138 batched calls |
| 17,509 nodes | ~17,509 embedding calls → stall | ~137 batched calls |
| **Total** | **~35,000 calls** | **~275 calls** |
---------
Co-authored-by: Oti_B <oti@mac.speedport.ip>
## Summary
- Treat `max_tokens=0` as unset (`or 8192`) when building model context
budgets, fixing agents that silently zeroed prompts when a vLLM model
had `max_tokens: 0` in tenant config
- Replace trailing same-role canvas history in `LLM._sys_prompt_and_msg`
instead of skipping the current user prompt
- Add `LLM.fit_messages()` validation after `message_fit_in` on agent
paths so empty user content fails fast with a clear error instead of
reaching vLLM
Fixes#16411
## Root cause
Agent canvas workflow called `message_fit_in` with `int(max_length *
0.97)`. When `max_length` was `0`, both system and user content were
trimmed to empty strings. The `[HISTORY STREAMLY]` log showing only
`{"role":"user","content":""}` matches this. A secondary bug skipped
appending the formatted user prompt when history ended with a `user`
role message.
## Test plan
- [x] Added `test/unit_test/agent/component/test_llm_prompt.py` for
role-replace, validation, and zero-budget fitting
- [x] Added
`test_message_fit_in_zero_budget_preserves_non_empty_messages` in
`test_generator_message_fit_in.py`
- [ ] CI unit tests
- [ ] Manual: agent canvas `begin → Retrieval → Agent → Message` with
vLLM Qwen3; confirm user message reaches LLM
Made with [Cursor](https://cursor.com)
---------
Co-authored-by: Taranum Wasu <taranumwasu@Taranums-MacBook-Pro.local>
Co-authored-by: Cursor <cursoragent@cursor.com>
### What problem does this PR solve?
This PR adds Google BigQuery as a first-class data source connector in
RAGFlow.
It enables users to ingest and sync BigQuery data using the same
row-to-document model used by relational database connectors: selected
content columns become document text, metadata columns become document
metadata, an optional ID column provides stable document IDs, and an
optional timestamp column enables cursor-based incremental sync.
The connector supports service-account JSON credentials, table mode,
custom query mode, GoogleSQL queries, cursor-based incremental sync,
deleted-row pruning support, configurable query limits such as
`maximum_bytes_billed`, dry-run validation, batch loading, stable
document IDs, and BigQuery-aware value serialization.
### What problem does this PR solve?
This PR adds an Agent LLM setting to control thinking mode for official
providers that expose a thinking switch.
Related to #12842.
Closes#15445.
Some providers expose thinking controls through provider-specific
request fields, but Agent LLM settings did not have a unified option for
users to enable or disable thinking mode.
This PR adds a `Thinking` selector with:
- System default
- Enabled
- Disabled
<img width="452" height="278" alt="8566b0b4-0546-4c8a-913d-f9bbd38319f6"
src="https://github.com/user-attachments/assets/25b497f7-1ba0-4bfe-940d-6fe79287d6ab"
/>
<img width="471" height="971" alt="8a0a6bee-f45f-48d5-bd83-17af260de3db"
src="https://github.com/user-attachments/assets/41ad43c1-5087-48f1-bf37-f2ca14c2be2f"
/>
Initial support is limited to the verified official providers:
- Qwen / DashScope: `enable_thinking`
- Kimi / Moonshot: `thinking.type`
- GLM / ZHIPU-AI: `thinking.type`
For LiteLLM-based providers, provider-specific fields are forwarded
through `extra_body` before `drop_params` filtering so the request
parameters are preserved.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: jiashi <jiashi19@outlook.com>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
## Summary
After #16407 merged, 44 of the original 93 CodeQL alerts were still open
on the default branch. This PR closes the remaining ones by:
1. **Moving 32 existing `// codeql[...]` directives** so they sit on the
line **immediately before** the suppressed statement. The original
multi-line suppression blocks had the directive as the first line, with
the rationale on subsequent lines. After line shifts (refactors, linter
reformat), the directive ended up several lines above the alert location
— CodeQL only recognizes the suppression when it appears on the line
directly above. (32 alerts across 27 files.)
2. **Adding 9 new `// codeql[...]` suppressions** for alerts that had no
suppression in the preceding lines at all — mostly real-fixes that
CodeQL conservatively still flags (filepath.Base, bounded slice sizes,
model-identifier strings, the MD5-legacy-migration lookup in
`conversation_service.py`).
## Files changed
- `api/db/services/conversation_service.py` — add
`py/weak-sensitive-data-hashing` suppression (MD5 for backward-compat
legacy row lookup; not used for auth)
- `api/db/services/llm_service.py` — 3×
`py/clear-text-logging-sensitive-data` suppressions on the lines that
log `llm_name` in warnings/info
- `common/misc_utils.py` — 2× `py/clear-text-logging-sensitive-data`
suppressions on the redacted `current_url` log sites
- `internal/agent/component/invoke.go` — moved existing
`go/request-forgery` directive
- `internal/agent/sandbox/ssh.go` — moved existing
`go/command-injection` directive
- `internal/agent/tool/retrieval_service.go` — added
`go/uncontrolled-allocation-size` suppression (`topN` is bounded to 1024
above)
- `internal/cli/common_command.go` — moved 2×
`go/disabled-certificate-check` directives
- `internal/cli/user_command.go` — added `go/clear-text-logging`
suppression (filepath.Base already strips user-identifying path)
- `internal/dao/pipeline_operation_log.go` — moved 2× `go/sql-injection`
directives
- `internal/dao/user_canvas.go` — added `go/sql-injection` suppression
in `GetList` (the new `userCanvasOrderClause` call path)
- `internal/engine/infinity/chunk.go` — moved existing
`go/unsafe-quoting` directive
- `internal/entity/models/*` — moved `go/path-injection` directives (15
files)
- `internal/handler/oauth_login.go` — moved existing
`go/cookie-httponly-not-set` directive
- `internal/handler/tenant.go` — moved existing `go/path-injection`
directive
- `internal/service/deep_researcher.go` — moved existing
`go/unsafe-quoting` directive
- `internal/service/dataset.go` — added
`go/uncontrolled-allocation-size` suppression (`n` bounded to 1024
above)
- `internal/service/file.go` — moved existing `go/request-forgery`
directive
- `internal/service/langfuse.go` — moved 2× `go/request-forgery`
directives
- `internal/utility/mcp_client.go` — moved 3× `go/request-forgery`
directives
- `internal/utility/smtp.go` — moved existing `go/email-injection`
directive
- `rag/prompts/generator.py` — added
`py/clear-text-logging-sensitive-data` suppression
- `web/.../use-provider-fields.tsx` — added
`js/prototype-pollution-utility` suppression (FORBIDDEN_KEYS guard is on
the line above)
## Why the previous PR left alerts open
`// codeql[query-id] explanation` must be on the line **immediately
before** the suppressed statement per the [GitHub CodeQL suppression
spec](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/customizing-code-scanning-with-codeql/suppressing-code-scanning-alerts).
The original suppression blocks were 4-5 lines, with the directive as
the **first** line. After linter reformat / line shifts, the directive
ended up too far above the actual alert line to be recognized. The fix
is to put the directive on the line directly above the suppressed
statement, with the rationale above it.
## Test plan
- All 9 modified Python files `ast.parse` clean
- All 4 modified Go files `gofmt` clean
- 36/44 expected alert suppressions in place
- 8 remaining CodeQL alerts are the originals (#3485851828, #3485851831,
#3485869759, #3485869766, #3485869768, #3485869771, #3485885962,
#3485895527) which were resolved by the corresponding commit comments;
these should close on the next scan when the suppression comments match
the alert lines.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
Add support for **"New API"** as a model provider, enabling connection
to [New API](https://github.com/QuantumNous/new-api) /
[one-api](https://github.com/songquanpeng/one-api) compatible gateways
that aggregate multiple LLM backends behind a unified OpenAI-compatible
`/v1` endpoint.
### Features
- **All model types**: Chat, Embedding, Rerank, Image2Text, TTS,
Speech2Text
- **List Models discovery**: `NewAPI(OpenAIAPICompatible)` class in
`model_meta.py` queries the gateway's `/v1/models` to auto-discover
available models via the native `GET /api/v1/providers/<name>/models`
endpoint
- **Model parameter editing**: Pencil icon on each discovered model row
to edit `model_type`, `max_tokens`, and `features` (e.g. tool call
support) before submitting
- **Custom model addition**: "Add Custom Model" button at the bottom of
the List Models dropdown for models not returned by the API
- **Gear icon settings**: Enabled the Settings gear button on provider
instances to manage models on existing instances (viewMode)
- **viewMode credential passthrough**: Fixed List Models in viewMode —
merges `initialValues` credentials when `api_key`/`base_url` fields are
hidden by `hideWhenInstanceExists`
### Changes
**Backend** (8 files):
- `rag/llm/chat_model.py` — `NewAPIChat(Base)` class
- `rag/llm/embedding_model.py` — `NewAPIEmbed(OpenAIEmbed)` class (no
auto `/v1` append)
- `rag/llm/rerank_model.py` — `NewAPIRerank(Base)` class (uses `/rerank`
endpoint)
- `rag/llm/cv_model.py` — `NewAPICv(GptV4)` class
- `rag/llm/tts_model.py` — `NewAPITTS(OpenAITTS)` class
- `rag/llm/sequence2txt_model.py` — `NewAPISeq2txt(GPTSeq2txt)` class
- `rag/llm/model_meta.py` — `NewAPI(OpenAIAPICompatible)` class for List
Models discovery
- `conf/llm_factories.json` — New API factory entry with all model type
tags
**Frontend** (8 files + 1 new SVG):
- `web/src/assets/svg/llm/new-api.svg` — New API logo icon
- `web/src/constants/llm.ts` — `LLMFactory.NewAPI` enum + `IconMap`
entry
- `web/src/components/svg-icon.tsx` — `NewAPI` added to `svgIcons`
-
`web/src/pages/user-setting/setting-model/modal/provider-modal/field-config/local-llm-configs.ts`
— New API `buildLocalConfig`
-
`web/src/pages/user-setting/setting-model/modal/provider-modal/constants.ts`
— `LIST_MODEL_PROVIDERS` includes NewAPI
- `web/src/pages/user-setting/setting-model/components/used-model.tsx` —
Enable Settings gear button
-
`web/src/pages/user-setting/setting-model/modal/provider-modal/hooks/use-list-models-picker.ts`
— viewMode credential merge + model editing state/handlers
-
`web/src/pages/user-setting/setting-model/modal/provider-modal/hooks/use-list-models-options.tsx`
— Pencil edit icon per model row
-
`web/src/pages/user-setting/setting-model/modal/provider-modal/index.tsx`
— `AddCustomModelDialog` import + edit dialog rendering
**Note on Go implementation**: A Go model driver (`NewAPIModel`
delegating to `OpenAIModel`) has been prepared but is deferred until the
Go runtime is enabled in a future release (current v0.26.0 images use
`API_PROXY_SCHEME=python` and do not compile Go binaries). Will submit
as a follow-up PR.
## Related
- Depends on: #15996 (provider instance API improvements — server-side
credential lookup, idempotent `add_model`, security fixes — required for
viewMode gear icon and batch model submission)
## Test plan
- [ ] Add New API provider with api_key and base_url pointing to an
OpenAI-compatible gateway
- [ ] Click "List Models" — should discover and display available models
from `/v1/models`
- [ ] Click pencil icon on a model — should open edit dialog to change
model_type, max_tokens, features
- [ ] Select multiple models and click OK — should add all selected
models
- [ ] Click gear icon on the added instance — should open viewMode with
List Models working
- [ ] In viewMode, select new models including pre-existing ones, click
OK — should succeed (requires #15996)
- [ ] Verify all model types work: create a Chat assistant, Embedding
KB, Rerank setting
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Tim Wang <wanghualoong@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
