Commit Graph

204 Commits

Author SHA1 Message Date
Zane
eeb59ec4f2 feat(stt): support Fun-ASR-Flash in Tongyi-Qianwen provider (#16844)
## What this PR does

Adds support for Alibaba Cloud's hosted Fun-ASR-Flash snapshots to the
existing Tongyi-Qianwen speech-to-text provider.

- registers `fun-asr-flash-2026-06-15` as a speech-to-text model;
- routes only `fun-asr-flash*` models to the documented workspace-native
multimodal-generation endpoint;
- supports local audio through size-checked data URIs as well as
URL/data-URI inputs;
- uses the documented SSE response mode for incremental streaming
transcription;
- closes the streamed HTTP response on completion, failure, or early
consumer cancellation;
- preserves the existing `dashscope.MultiModalConversation` path for all
other Qwen audio models;
- keeps RAGFlow's existing synchronous and streaming adapter interfaces.

## Why

Fun-ASR-Flash does not use the legacy Qwen audio request shape currently
used by `QWenSeq2txt`. Its synchronous API expects `input_audio` at:

`/api/v1/services/aigc/multimodal-generation/generation`

Without a narrowly scoped adapter path, the hosted model cannot be
selected successfully through RAGFlow's Tongyi-Qianwen speech-to-text
provider.

Closes #16843.

## Compatibility

The new behavior is gated by the `fun-asr-flash` model-name prefix.
Existing Qwen audio models continue through the original code path
unchanged.

## Validation

- `pytest test/unit_test/rag/llm/test_sequence2txt_model.py`: 10 passed
- Ruff check: passed
- Ruff format check: passed
- `llm_factories.json` validation: passed
- Real hosted-API validation with WAV audio
- Real RAGFlow upload/indexing validation with MP3 audio

The unit tests cover the native Fun-ASR-Flash request, regression
behavior for the legacy Qwen path, SSE streaming, and early response
cleanup.

## Documentation

- https://help.aliyun.com/document_detail/2979031.html
- https://help.aliyun.com/document_detail/2869541.html
### Why a dedicated adapter path is necessary (official evidence)

Alibaba Cloud's [Fun-ASR RESTful API
reference](https://help.aliyun.com/en/model-studio/fun-asr-recorded-speech-recognition-http-api)
makes the incompatibilities with RAGFlow's existing Qwen audio path
explicit:

| Adapter change | Official API requirement | Why the existing path is
insufficient |
| --- | --- | --- |
| Call the workspace-native HTTP endpoint | The Fun-ASR-Flash
synchronous section states that SDK calls are not supported and
specifies `POST /api/v1/services/aigc/multimodal-generation/generation`.
| The existing adapter calls `dashscope.MultiModalConversation`, so a
direct HTTP path is required. |
| Use the `input_audio` message shape | `input.messages`, `content`,
`type: input_audio`, `input_audio`, and `input_audio.data` are
documented as required for an audio request. | The existing Qwen path
sends the legacy `audio` content shape, which does not match this API
contract. |
| Send `parameters.format` | The request schema marks `parameters` and
`format` as **Required**, and says the value must match the actual audio
format. | The legacy request has no Fun-ASR-Flash `parameters.format`
field, so the adapter must derive and send it. |
| Encode local files as Data URIs | `input_audio.data` accepts either a
public URL or a Base64 Data URI; the reference gives the exact
`data:{MIME_TYPE};base64,...` form. | RAGFlow supplies local file paths,
which the remote API cannot read directly. |
| Parse `output.text` | The documented non-streaming response returns
the accumulated transcription in `output.text`. | The legacy Qwen
response parser reads `output.choices[].message.content`, so a separate
response parser is required. |
| Enforce the Base64 input limit | The reference requires the
Base64-encoded audio to remain within the 10 MB input limit. | The
adapter checks encoded size before reading/sending local audio and
directs oversized inputs to the existing public-URL path. |
| Use SSE for streaming | The reference specifies `X-DashScope-SSE:
enable` and documents intermediate and final SSE events. | The adapter
parses those events instead of wrapping one blocking response as a
synthetic stream. |
| Release streamed responses | Streaming responses must be closed when
iteration completes or stops early. | A `finally` cleanup releases the
HTTP response on completion, errors, and consumer cancellation. |

`sample_rate` is documented as **Optional**. The implementation omits it
instead of declaring a fixed value that may not match remote or
compressed audio.

The [official speech-to-text model
list](https://help.aliyun.com/en/model-studio/asr-model/) separately
confirms that `fun-asr-flash-2026-06-15` is an offline HTTP model with a
five-minute audio limit.

---------

Signed-off-by: LauraGPT <LauraGPT@users.noreply.github.com>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
2026-07-16 09:37:37 +08:00
SYED ALI ABBAS RAHIL
bda703b588 test: add regression coverage for metadata filter pagination beyond push-down cap (#16932)
### Summary

#16524 reports that a manual metadata filter matching more documents
than the ES push-down cap (`filter_doc_ids_by_meta_pushdown`'s default
`limit=10000`) drops documents once the request falls back to the
in-memory path — e.g. a `canon Not in ["0"]` filter over a
39,573-document KB where ~38,500 matching documents never come back.

I traced through the current code path for this exact scenario:
- `_filter_doc_ids_by_metadata_es` correctly detects when the match
total exceeds the push-down cap and bails to the in-memory fallback
instead of returning a truncated slice.
- `get_flatted_meta_by_kbs` (fixed by #16095) now fully paginates
through every document in the KB rather than stopping after the first
page.
- `es_conn.py`'s `search()` already switches to `search_after`-based
pagination once `offset + limit` would exceed ES's `max_result_window`
(10,000), so the outer pagination loop doesn't get cut off by that
ceiling either.
- `meta_filter()` then aggregates over the complete flattened metadata
with no additional cap.

I couldn't reproduce the drop against current `main` following that
path. This PR adds a test that simulates the exact reported scenario
(12,000 synthetic documents, `canon Not in ["0"]` matching all but 30 of
them) against a fake, paginated `docStoreConn` standing in for
Elasticsearch — both assertions pass on current `main`.

To make sure this is a meaningful regression test and not a false
positive, I temporarily reverted `get_flatted_meta_by_kbs` to stop after
the first page (the pre-#16095 behavior) and confirmed the test
correctly fails (970 of the expected 11,970 documents), then restored
the original code before committing.

Given all of that, it looks like #16524 may already be fixed by the
combination of #16095 and the existing `search_after` handling in
`es_conn.py`, but I could be missing something about the reporter's
specific deployment or a scenario I haven't considered (e.g. a
downstream cap once matched doc_ids feed into the content-chunk
retrieval query). I've left a comment on the issue with this same
analysis so a maintainer familiar with the history here can confirm or
point me at what I'm missing. Either way, this test is a useful
regression guard for the pagination behavior going forward.
2026-07-16 09:33:48 +08:00
Harsh Kashyap
5c96fa51f0 fix(docling): detect chunked response by shape, not request payload (#16921)
Fixes #16917.

## Problem

`deepdoc/parser/docling_parser.py::_parse_pdf_remote` decides whether
the
response is chunked based on which payload was sent, not on what came
back.
Docling Serve silently drops unknown fields such as `do_chunking`
(Pydantic
`extra="ignore"`) and returns a standard `{"document": ..., "status":
...}`
conversion response. The code then:

1. sets `is_chunked_response = True` from the request shape,
2. logs `Successfully used native chunking on: <endpoint>`,
3. extracts 0 chunks from `response_json.get("results", [])`,
4. logs `Native chunks received: 0`,
5. falls through to the existing `md_content` fallback.

The `md_content` fallback path is fine. The misleading log lines are the
problem: operators see "Successfully used native chunking" immediately
followed by "Native chunks received: 0" and "No chunk built", which
looks
like an internal regression rather than a server contract gap.

## Fix

Decide chunked-vs-standard from the **response shape**, not the request:

```python
response_is_chunk = self._looks_like_chunk_response(response_json)
is_chunked_response = chunk_flag and response_is_chunk
```

`_looks_like_chunk_response` returns True iff the response is a
non-empty
list or a dict with a non-empty `results` or `chunks` list. A standard
conversion response (`{"document": ..., "status": ...}`) does not match,
so
a server that ignored the chunking flag is correctly classified as
standard
even when the request payload asked for chunking.

When chunking was requested but the server returned a standard response,
log a single WARNING ("Server ignored chunking request on <endpoint>;
treating response as standard conversion.") instead of the INFO success
line. The misleading "Prioritizes native chunking endpoints" docstring
is
replaced with what the code actually does.

## Tests

`test/unit_test/deepdoc/parser/test_docling_parser_remote.py` (6 tests,
all passing):

- `test_remote_chunked_200_standard_payload_falls_back` (existing —
still
  passes; the `md_content` path is unchanged)
- `test_chunk_shape_helper_recognises_chunk_payloads`
- `test_chunk_shape_helper_rejects_standard_payloads`
- `test_remote_chunked_request_with_results_list_is_treated_as_chunked`
- `test_remote_top_level_list_response_is_treated_as_chunked`
- `test_remote_chunked_request_with_ignored_flag_does_not_log_success`

```
$ uv run pytest test/unit_test/deepdoc/parser/test_docling_parser_remote.py -v
============================== 6 passed in 0.26s ==============================
```

## Files changed

- `deepdoc/parser/docling_parser.py` (+35 / -5)
- `test/unit_test/deepdoc/parser/test_docling_parser_remote.py` (+89 /
-4)

## Backward compatibility

- All four payload/endpoint combinations continue to be tried in the
same order.
- The bundled-docling happy path (`parse_pdf`, not `_parse_pdf_remote`)
is
  untouched.
- A server that returns a real chunked response to a chunked request
still
goes down the chunked branch. A server that returns a standard response
  to a chunked request now goes down the standard branch with
  `is_chunked_response=False` instead of misleadingly logging success.

## Follow-up (out of scope)

Calling the real Docling-Serve native chunk endpoints
(`/v1/chunk/hybrid/source`, `/v1/chunk/hierarchical/source`) with
`HybridChunkerOptions` is a larger feature change and warrants its own
PR after this lands.

Co-authored-by: Harsh23Kashyap <harsh@example.com>
2026-07-16 09:29:09 +08:00
zhifu gao
06e36d24f4 feat(stt): add FunASR / SenseVoice provider (#16473)
### Summary

Adds FunASR as a self-hosted speech-to-text provider through its
OpenAI-compatible `/v1/audio/transcriptions` endpoint.

This is a focused replacement for #15526 by @Rene0422 and relates to
#15448. The unrelated Markdown parser changes from the previous branch
are intentionally removed so this PR contains only the FunASR provider
integration.

- register FunASR as a `SPEECH2TEXT` factory;
- add `FunASRSeq2txt` with `sensevoice` and `http://localhost:8000/v1`
defaults, an optional API key, URL normalization, and inherited
transcription handling;
- wire FunASR into the current local-provider schema with a prefilled
local URL and official documentation link;
- discover the server's `/v1/models` dynamically and expose every
returned model as speech-to-text in the model picker;
- use RAGFlow's existing default provider icon fallback instead of
referencing a missing `funasr` asset;
- list FunASR in the supported-provider documentation;
- add focused backend and frontend regression tests.

### Validation

- focused backend pytest suite -> `7 passed`
- real CPU `funasr-server` + RAGFlow provider smoke test -> discovered
`fun-asr-nano`, `sensevoice`, and `paraformer`; transcribed a real WAV
as `我现在在录一段测试音频` (`10` tokens, `0.504s`)
- `ruff check` and `ruff format --check` on the changed Python files
- `python3 -m py_compile` on the provider and its test
- JSON parse and a semantic assertion for exactly one enabled FunASR
`SPEECH2TEXT` factory
- focused frontend Jest test -> `2 passed`
- ESLint and Prettier on all changed TypeScript files
- `npm run build` -> production build succeeded (`14,181` modules
transformed)
- `git diff --check`

### Deployment

Run FunASR separately and point the RAGFlow provider at it:

```bash
pip install funasr
funasr-server --device cuda --model sensevoice
```

The API key remains optional because the stock local server does not
require authentication. A key can still be supplied when the endpoint is
protected by a gateway.

---------

Signed-off-by: LauraGPT <LauraGPT@users.noreply.github.com>
Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
2026-07-15 19:02:05 +08:00
Lynn
bd7a3bb878 Fix: align go provider apis with python apis (#16867)
### Summary

As title.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-07-14 21:06:11 +08:00
Wang Qi
3087eb854c Fix Agent bot 2 permission bugs (#818) (#16909) 2026-07-14 20:11:16 +08:00
hyotaek kim
760203be42 fix(agent): preserve zero temperature setting (#16897)
### Summary

- Preserve an explicitly enabled `temperature=0` in the Agent LLM
configuration.
  - Continue excluding invalid negative temperature values.
  - Add a focused regression test to the existing Agent LLM test file.

  Fixes #16683
2026-07-14 20:10:51 +08:00
euvre
3bfad1f00e fix: correct model type mappings and improve system setting persistence (#16501) 2026-07-13 16:42:55 +08:00
Sbaaoui Idriss
d35e957252 fix: test drift on go specific proxy scheme (#16796)
### Summary

certain tests fail because of test drift and were fixed, other because
of go issues

---------

Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-07-13 16:41:38 +08:00
boskodev790
80a7a87427 fix(agent): port QWeather to ToolBase so it works as an Agent tool (#16692)
### Summary

Port the **QWeather** agent tool to the modern `ToolBase` / `_invoke`
interface. It was still written against the removed legacy
`ComponentBase` / `_run` / `be_output` API, so it was non-functional as
an Agent tool — adding it to an Agent raised `AttributeError` because it
had no `get_meta()`. This is the same defect that was fixed for the
AkShare tool in #16417.

**Changes**
- `QWeatherParam` now extends `ToolParamBase` with a `meta` exposing a
`query` (location) parameter, and adds `get_input_form()`. Existing
config (`web_apikey`, `lang`, `type`, `user_type`, `time_period`) is
preserved.
- `QWeather` now extends `ToolBase` and implements `_invoke(**kwargs)`
with the standard retry loop, cancellation checks,
`set_output("formalized_content", ...)`, and `thoughts()`. The weather /
indices / air-quality branches and the API error-code messages are kept.
- Added `test/unit_test/agent/component/test_qweather.py` covering the
restored `meta`, param validation, the weather-now and multi-day and
indices branches, the empty-query short-circuit, and the location-lookup
error message.

**Testing**
- `ruff check agent/tools/qweather.py
test/unit_test/agent/component/test_qweather.py` — clean
- `ruff format --check` — clean
- `pytest test/unit_test/agent/component/test_qweather.py`
2026-07-13 16:31:19 +08:00
Harsh Kashyap
8a3699fa87 fix(agent): clear component inputs on canvas re-run (#16790)
### What problem does this PR solve?

Issue [#16758](https://github.com/infiniflow/ragflow/issues/16758) —
clicking a chunk whose data references a single-line variable from an
Await-Response (UserFillUp) component, the Agent's `user_prompt` is
being resolved against the **previous** canvas run's captured value
instead of the current run's value. The system-prompt path works only
because the system prompt is computed upstream and re-reads the value on
the new run.

### Root cause

`Canvas._run_impl` reset every path component with `only_output=True`,
so `_param.inputs` was never cleared between runs.
`ComponentBase.get_input()` calls `set_input_value(var, resolved)` at
line 482, which writes the resolved variable into
`self._param.inputs[var]["value"]`. On the next canvas run, that input
was never cleared, so the previous run's resolved value stuck around.
The Agent's `kwargs.get("user_prompt")` then read the stale string and
forwarded it to the LLM, which produced the "Understood. Please provide
the text..." fallback because the prompt looked empty.

### What changed?

- `agent/canvas.py` — differentiate `begin` (still `only_output=True`,
since it has no inputs and the webhook payload branch below populates
`request` explicitly) from non-begin path components (reset with
`only_output=False`, which clears both `inputs` and `outputs`).
- `test/unit_test/agent/test_canvas_input_reset.py` — new pytest module.
Pinned the contract: non-begin path components receive
`only_output=False`. The fix is small enough to verify with a stub
canvas rather than a full canvas-runtime test (the existing agent
conftest hits an unrelated `scholarly` import on Python 3.13, so a real
canvas import would require fixing that first).

### Backward compatibility

- `Begin` behaviour unchanged.
- All non-begin path components: previously persisted inputs across runs
(the bug); now reset between runs. Components that were relying on stale
inputs (none found in the existing test suite) would lose that as a side
effect, but that is the entire point of the fix.
- No API surface change. No backend change.

### Testing

```
$ uv run pytest test/unit_test/agent/test_canvas_input_reset.py -v
collected 4 items
test/unit_test/agent/test_canvas_input_reset.py::test_begin_is_reset_with_only_output_true PASSED
test/unit_test/agent/test_canvas_input_reset.py::test_non_begin_path_components_are_reset_with_only_output_false PASSED
test/unit_test/agent/test_canvas_input_reset.py::test_only_path_components_are_reset PASSED
test/unit_test/agent/test_canvas_input_reset.py::test_inputs_reset_flag_is_passed_to_non_begin_components PASSED
4 passed in 0.14s
```

`python3 -m py_compile agent/canvas.py` clean. Existing agent test files
(`test_switch.py`, `test_llm_prompt.py`) hit a pre-existing `scholarly`
import error on Python 3.13 (unrelated to this PR), so I couldn't run
the full agent suite. Recommend fixing the `scholarly` import
separately.

### Files changed

- `agent/canvas.py` (+9 / −1)
- `test/unit_test/agent/test_canvas_input_reset.py` (new, +104)

Fixes #16758

---------

Co-authored-by: Harsh Kashyap <harshkashyap@Harshs-MacBook-Pro.local>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-07-11 16:33:37 +08:00
Taranum Wasu
0ee02fb6d8 [Fix] Rename StandardizeImag -> StandardizeImage to fix deepdoc OCR preprocessing (#7316) (#16785)
Fixes #7316.

## Problem

`deepdoc/vision/operators.py` defines the image-standardize
preprocessing op as `class StandardizeImag` (missing the final `e`), but
every caller — including
`deepdoc/vision/recognizer.py::Recognizer.preprocess` — looks the class
up by the canonical string `"StandardizeImage"` via:

```python
op_type = new_op_info.pop("type")  # "StandardizeImage"
preprocess_ops.append(getattr(operators, op_type)(**new_op_info))
```

So `getattr(operators, "StandardizeImage")` raised `AttributeError`, and
the "StandardizeImage" preprocessing step silently never ran for any
image pipeline that used the dynamic dispatch (LayoutLMv3 and friends).
The user-visible symptom is that the standardize step is missing
entirely from the preprocessing chain, so the model gets un-normalized
images.

## Production fix

```diff
-class StandardizeImag:
+class StandardizeImage:
     """normalize image
     Args:
         mean (list): im - mean
         std (list): im / std
         is_scale (bool): whether need im / 255
         norm_type (str): type in ['mean_std', 'none']
     """
```

That's the entire production change — a one-character class rename. The
misnamed `StandardizeImag` had no other references in the codebase
(verified via `git grep`), so removing it is safe; every caller uses the
canonical `"StandardizeImage"` string and will now resolve correctly.

## Tests

New `test/unit_test/deepdoc/vision/test_operators_standardize_image.py`
with six regression tests, all green locally:

```
test_standardize_image_class_resolves_by_canonical_name            PASSED
test_standardize_image_callable_matches_legacy_alias_name          PASSED
test_standardize_image_normalizes_input_with_mean_std_and_is_scale PASSED
test_standardize_image_skips_scaling_when_is_scale_false           PASSED
test_standardize_image_norm_type_none_passes_image_through         PASSED
test_standardize_image_via_module_getattr_dispatch_path            PASSED
6 passed in 0.18s
```

The tests:
1. **Pin the dispatch contract** (`hasattr(operators,
"StandardizeImage")`) — this is the exact check the recognizer's
`getattr` would do, so any future regression fails the same way the
runtime would.
2. **Pin that the misspelled name is gone** — if a downstream caller
ever relied on it, this fails loudly.
3–5. **Behavioural coverage** of the three documented code paths:
`is_scale=True, norm_type="mean_std"`, `is_scale=False,
norm_type="mean_std"`, and `norm_type="none"`.
6. **End-to-end via the same `getattr(operators, "StandardizeImage")`
call** the recognizer uses, with a real numpy image, so any rename or
removal surfaces as `AttributeError` instead of silently skipping the
step.

Verified both ways:
- Without the fix → **all 6 tests fail** (Python even suggests
`'StandardizeImag' → 'StandardizeImage'`)
- With the fix → all 6 pass in 0.15s

The test file follows the project's existing pattern
(`test/unit_test/deepdoc/parser/test_html_parser.py`): load the target
module via `importlib.util.spec_from_file_location`, stub the only
project-internal import (`rag.utils.lazy_image`), and assert against the
loaded module — no full RAGFlow runtime required.

## Risk

Very low. The class is renamed; no public Python API was using the
misnamed class. The only reference path is the `"StandardizeImage"`
string in `recognizer.py:270`, which now resolves correctly.

## Out of scope

- No other ops in `operators.py` are affected; checked all the others
(DecodeImage, NormalizeImage, Permute, etc.) and they all use correct
names.
- The dynamic-dispatch lookups in `recognizer.py` for `LinearResize`,
`StandardizeImage`, `Permute`, `PadStride` all use the same dispatch
path; only the `StandardizeImage` key was broken. No other keys need
fixing.

Made with [Cursor](https://cursor.com)

---------

Co-authored-by: Taranum01 <Taranum01@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-07-11 16:32:03 +08:00
hyotaek kim
9b60870fd6 feat: make blob storage size threshold configurable (#16806)
### Summary

- Make `BLOB_STORAGE_SIZE_THRESHOLD` configurable through an environment
variable.
  - Preserve the existing 20 MiB default.
  - Add tests for the default and configured values.

  ### Why

Blob storage, Seafile, and WebDAV connectors currently use a hardcoded
20 MiB limit. Self-hosted users
cannot raise this limit without modifying the source code inside the
container.

  ### Testing

  - `test/unit_test/data_source/test_config.py`: 2 passed
- `ruff check common/data_source/config.py
test/unit_test/data_source/test_config.py`

  Fixes #16634
2026-07-10 21:36:12 +08:00
Harsh Kashyap
289ca28ce2 Fix OpenAI agent stream chunk shape (#16402)
### What problem does this PR solve?

Closes #8175.

The Agent OpenAI-compatible streaming path uses `get_data_openai(...,
stream=True)`, but that helper currently returns a minimal chunk shape.
The main OpenAI-compatible chat endpoint already includes chunk metadata
such as `created`, `system_fingerprint`, `usage`, `logprobs`, and
assistant role/tool placeholders.

This PR aligns the Agent stream helper with that existing
OpenAI-compatible chunk shape while keeping the current `delta.content`
behavior and existing reference injection path intact.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Verification

- `./.venv/bin/python -m pytest
test/unit_test/api/utils/test_api_utils.py -q`
- `python3 -m py_compile api/utils/api_utils.py
test/unit_test/api/utils/test_api_utils.py`
- `uvx ruff check api/utils/api_utils.py
test/unit_test/api/utils/test_api_utils.py`

---------

Co-authored-by: Harsh Kashyap <harshkashyap@Harshs-MacBook-Pro.local>
2026-07-10 10:59:32 +08:00
buua436
74bbbba3e0 fix: align model default handling (#16782) 2026-07-10 10:34:19 +08:00
buua436
6a77523bf0 refa: resolve tenant model refs consistently (#16744) 2026-07-09 14:02:08 +08:00
qinling0210
ae96e636e9 Handle searching dataset without embedding model (#16742)
### Summary

Handle searching dataset without embedding model

In this PR, Searching datasets with different embedding models or
searching dataset with/without embedding models are not allowed. We will
improve the behavior later.
2026-07-09 11:38:55 +08:00
Lynn
1430d0e431 Fix: provider name (#16733) 2026-07-09 10:19:10 +08:00
Lynn
0ae5961e1c Feat: v0.27.0 model provider (#16604) 2026-07-08 09:47:29 +08:00
Öndery
28a41ed070 fix(task_executor): fix Langfuse flush/shutdown deadlock that freezes document parsing (#16502) 2026-07-07 19:06:30 +08:00
S
f477d3329d Fix: ValueError: too many values to unpack in list_tenant_added_models for model IDs containing '@' (#16467) (#16468) 2026-07-07 09:40:27 +08:00
Mattie Schraeder
8a19c6aa5a Make RAPTOR GMM robust on small reduced clusters (#16632) 2026-07-06 21:09:35 +08:00
Harsh Kashyap
98189cd20a Fix OpenAI response created timestamp (#16401)
## What this fixes

Closes #16400.

`get_data_openai()` currently returns `created: null` when callers do
not pass a timestamp, and it replaces explicit timestamp values with the
current time. This makes non-streaming OpenAI-compatible responses
inconsistent with the expected integer `created` timestamp field.

## Change

- Preserve explicit `created` values when provided.
- Default non-streaming responses to `int(time.time())` when `created`
is not provided.
- Add focused unit coverage for default timestamps, explicit timestamps,
and unchanged streaming chunk shape.

## Verification

- `./.venv/bin/python -m pytest
test/unit_test/api/utils/test_api_utils.py -q`
- `python3 -m py_compile api/utils/api_utils.py
test/unit_test/api/utils/test_api_utils.py`
- `uvx ruff check api/utils/api_utils.py
test/unit_test/api/utils/test_api_utils.py`

---------

Co-authored-by: Harsh Kashyap <harshkashyap@Harshs-MacBook-Pro.local>
2026-07-06 14:16:16 +08:00
Zhichang Yu
014c3f634f Align Go ingestion boundaries with Python (#16647)
Moves doc_id blob resolution into Parser, tightens chunker/tokenizer to
Python output_format semantics, updates extractor list handling, and
fixes real-template integration tests.
2026-07-05 20:43:52 +08:00
Wang Qi
a0e65637eb Delete canvas_app.py and evaluation_service.py (#16614)
Follow on PR #13295
2026-07-03 21:03:54 +08:00
monsterDavid
7da4f200e5 fix(agent): enable MCP file preview via doc_id (#15399)
## Summary
MCP-wrapped agents could only force-download files looked up by
`doc_id`. This adds an explicit preview path and inline response headers
for previewable file types.

- **New** `GET /api/v1/agents/attachments/{attachment_id}/preview` —
inline preview for PDFs, images, and other safe types (pass `ext` and/or
`mime_type`)
- **Improved** `GET /api/v1/documents/{doc_id}/preview` — sets inline
disposition using the document filename
- **Improved** attachment download routing — resolves `mime_type` /
`ext` query params (no default `markdown`), supports
`disposition=inline`
- **DocGenerator output** — includes URL-encoded `preview_url` for MCP
clients
- **Legacy `/document/download/...` aliases** — still use download
semantics; MCP clients should call `/preview` explicitly

Fixes #15398

## Test plan
- [x] `pytest test/unit_test/api/utils/test_file_response_headers.py`
(6/6)

---------

Co-authored-by: MkDev11 <mkdev11@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Ling Qin <qinling0210@163.com>
2026-07-03 19:56:01 +08:00
S
1861087787 fix(agent): defend against @ in var names at all template-split sites (#16469)
## Summary

While fixing #16467 (IterationItem crash on `@` in user-defined output
keys), an audit of `agent/**/*.py` revealed **three additional sites**
with the same vulnerability. This PR hardens all of them with
`maxsplit=1` and adds regression tests.

This is **defense-in-depth hardening**, not a behavior change. The
current `variable_ref_patt` regex constrains `var_nm` to
`[A-Za-z0-9_.-]+`, so single-`@` templates resolve exactly as before.
The `maxsplit=1` only kicks in if the trailing side itself contains `@`
— currently unreachable from the public DSL surface, but trivially
exploitable the moment a user-defined output key happens to contain `@`
(e.g. `user@email`) or the regex is ever relaxed.

> **Note on issue scope**: The primary fix for #16467 (the
`list_tenant_added_models` `ValueError` crash on `@` in model names) is
in PR #16468. This PR is a **follow-up hardening sweep** of the same
vulnerability class found in `agent/` during that audit; it does not
duplicate or replace #16468.

## Sites hardened

| File | Line | Method |
|------|------|--------|
| `agent/canvas.py` | 206 | `Graph.get_variable_value` |
| `agent/canvas.py` | 256 | `Graph.set_variable_value` |
| `agent/component/base.py` | 533 |
`ComponentBase.get_input_elements_from_text` |
| `agent/component/iterationitem.py` | 88 |
`IterationItem.output_collation` |

All now use `split("@", 1)` with an inline comment explaining the
rationale. The trailing side keeps any embedded `@`.

## Sites already safe (audited but left alone)

| File | Reason safe |
|------|------------|
| `agent/canvas.py:708` (`is_reff`) | Pre-checks `len(arr) != 2` |
| `agent/component/categorize.py` | Uses `rsplit` |
| `agent/component/iteration.py` | Pre-validates via regex |
| Other call sites | `rsplit` or regex pre-validation |

## Regression tests

9 new tests across 2 files, all `pytest.mark.p2`:

| File | Tests |
|------|-------|
| `test/unit_test/agent/test_canvas_at_split.py` | 6 —
`get_variable_value`, `set_variable_value`, round-trip, single-`@`,
missing-component |
| `test/unit_test/agent/component/test_iterationitem_at_split.py` | 3 —
`output_collation` with `@` in var, single-`@`, non-matching cid |

Each test was **verified to fail with `ValueError: too many values to
unpack (expected 2)`** when the corresponding fix is temporarily
reverted, confirming the tests actually catch the bug rather than just
exercising the happy path.

## Test results

```
9 passed in 0.04s
```

Full agent unit suite also clean (38 passed, 3 skipped; 6 unrelated
pre-existing collection errors from missing `peewee`/`requests` in local
venv — not caused by this PR).

## Related

- Issue: #16467
- Primary fix PR: #16468 (closes the issue)
- This PR: defense-in-depth follow-up, intentionally non-blocking on
#16467

---------

Co-authored-by: skbs-eng <skbs-eng@users.noreply.github.com>
2026-07-03 19:26:27 +08:00
Muhammad Furqan
3cba34d67f fix(agent/tools): port Crawler to ToolBase so it can load and run (#16415)
### What problem does this PR solve?

Closes #16414.

The **Crawler** agent tool (`agent/tools/crawler.py`) was never ported
to the modern `ToolBase`/`_invoke` interface during the agent module
redesign, so it was broken in three independent ways:

1. **Crashed on construction.** `CrawlerParam` extends `ToolParamBase`,
whose `__init__` reads `self.meta["parameters"]`, but `CrawlerParam`
defined no `meta`. Constructing it raised `AttributeError:
'CrawlerParam' object has no attribute 'meta'`. Because
`agent/canvas.py` instantiates `component_class(component_name +
"Param")()` while loading a canvas, **any agent containing a Crawler
node failed to load.**
2. **`_invoke` missing.** It extends `ToolBase` (whose `invoke()`
dispatches to `self._invoke`) but only implemented the legacy `_run`, so
`_invoke` resolved to `ComponentBase._invoke` → `NotImplementedError`.
3. **`be_output` removed.** `_run` called `Crawler.be_output(...)`,
which no longer exists on the base classes.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes

- Add a `ToolMeta` to `CrawlerParam` (defined before
`super().__init__()`, matching every other ported tool such as
`ArXivParam`/`TavilyExtractParam`) advertising a required `query`
parameter — the URL to crawl, default `{sys.query}`, consistent with the
`{sys.query}` convention shared by the other tools.
- Replace the legacy `_run`/`be_output` with `_invoke`/`set_output`,
writing the extracted page content to `formalized_content` (errors
surfaced via `_ERROR`), consistent with the other tools.
- Preserve the existing SSRF guard (`assert_url_is_safe` +
`pin_dns_global`).
- Add regression tests
(`test/unit_test/agent/component/test_crawler.py`) covering param
construction, validation, and the tool descriptor.

Same class of defect as #16329 (DeepL). Backend-only; no frontend
changes.

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-07-03 17:15:48 +08:00
Wang Qi
6a4b9be426 Refactor: reformat all code for lefthook using ruff and gofmt (#16585) 2026-07-03 12:53:39 +08:00
Muhammad Furqan
83540185e1 fix(agent/tools): port AkShare to ToolBase so it works as an Agent tool (#16417)
### What problem does this PR solve?

Closes #16416.

The **AkShare** agent tool (`agent/tools/akshare.py`) was never ported
to the modern `ToolBase`/`_invoke` interface during the agent module
redesign and was still written against the removed legacy
`_run`/`be_output` API, so it was non-functional:

1. **Adding it to an Agent raised `AttributeError`.** `AkShare` extended
`ComponentBase` (not `ToolBase`) and `AkShareParam` defined no `meta`,
so it had no `get_meta()`. `agent/component/agent_with_tools.py` builds
each tool's function descriptor via `cpn.get_meta()`, so constructing an
Agent that includes the AkShare tool raised `AttributeError: 'AkShare'
object has no attribute 'get_meta'`.
2. **It could never run.** `invoke()` dispatches to `self._invoke`, but
`AkShare` only implemented the legacy `_run`, so `_invoke` fell through
to `ComponentBase._invoke` → `NotImplementedError`. `_run` also called
`be_output(...)`, which no longer exists on the base classes.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes

- Port `AkShareParam` to `ToolParamBase` with a `ToolMeta` (defined
before `super().__init__()`, matching `ArXivParam`/`TavilyExtractParam`)
exposing a required `query` parameter — the stock symbol to look up,
default `{sys.query}`. `query` matches the `{sys.query}` convention
shared by the other tools.
- Rewrite the component with `_invoke`/`set_output("formalized_content",
...)` (errors surfaced via `_ERROR`), keeping `top_n` and importing
`akshare` lazily.
- Add regression tests
(`test/unit_test/agent/component/test_akshare.py`) covering param
construction, validation, and the tool descriptor.

Same class of defect as #16329 (DeepL) and #16414 (Crawler).
Backend-only; no frontend changes.

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-07-03 11:39:26 +08:00
Kevin Hu
62f94cd59b Feat: Add knowledge compilation workflows (#16515)
## Summary
- Add knowledge compilation template APIs, services, and builtin
template seed data
- Add advanced knowledge compile structure/artifact/RAPTOR workflow
support
- Update parsing, dataset/document APIs, and supporting services for
compilation workflows
2026-07-02 23:22:07 +08:00
Harsh Kashyap
d770217b25 fix(api): fall back to factory max_tokens for tenant models (#16364) 2026-07-01 16:00:13 +08:00
Harsh Kashyap
b8e960e6c8 fix(qa): preserve final CSV pair row number (#16433) 2026-07-01 14:52:08 +08:00
Harsh Kashyap
b42414b64a fix(deepdoc): parse bodyless HTML fragments (#16423) 2026-07-01 14:45:22 +08:00
Harsh Kashyap
508f6226f8 fix(agent): filter TuShare news with upstream keyword input (#16361)
## Summary

TuShare required non-empty upstream input but filtered fetched news with
the static `keyword` param (default empty string), so agent-provided
keywords were ignored.

Use `self._param.keyword or ans` when filtering, matching how AkShare
uses upstream input for its query.

Fixes #16360

## Test plan

- [x] `test_tushare_filters_with_upstream_keyword_when_param_empty`
mocks the API and asserts only rows matching the upstream keyword are
returned

---------

Co-authored-by: yzc <yuzhichang@gmail.com>
Co-authored-by: Harsh Kashyap <harshkashyap@Harshs-MacBook-Pro.local>
2026-07-01 13:51:39 +08:00
Lynn
400476f0b3 Feat: SoMark (#16482)
Follow #15486
Co-authored-by: limuting <limuting233@gmail.com>
Co-authored-by: lutianyi <lutianyi233@163.com>
Co-authored-by: justinychuang <huangyicheng@soulcode.cn>
Co-authored-by: maybehokori <138367708+maybehokori@users.noreply.github.com>
2026-07-01 13:29:28 +08:00
Muhammad Furqan
828c5789f6 fix(agent/tools): GoogleScholar empty json output and ignored top_n (#16419)
### What problem does this PR solve?

Closes #16418.

`scholarly.search_pubs(...)` returns a **lazy generator**, but
`agent/tools/googlescholar.py` treated it as a re-iterable, bounded
list:

```python
scholar_client = scholarly.search_pubs(kwargs["query"], ...)   # lazy generator
self._retrieve_chunks(scholar_client, ...)                     # (1) iterates -> exhausts it
self.set_output("json", list(scholar_client))                  # (2) already empty -> []
```

1. **`json` output was always empty.** `_retrieve_chunks` iterates
`scholar_client`, exhausting the generator; `list(scholar_client)` then
returns `[]`.
2. **`top_n` was never applied.** Unlike `ArXiv`
(`max_results=self._param.top_n`), the unbounded generator was passed
straight to `_retrieve_chunks`, which has no internal limit — so the
tool kept paginating well past Top N (until an error, rate-limit/block,
or `COMPONENT_EXEC_TIMEOUT`).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes

- Materialize at most `top_n` results once with `itertools.islice`, and
reuse that list for both `_retrieve_chunks` and the `json` output.
- Add regression tests
(`test/unit_test/agent/component/test_googlescholar.py`, stubbing
`scholarly.search_pubs`) covering the `top_n` bound, the non-empty
`json` output, and the empty-query short-circuit.

Verified: against `main` the new tests fail with `assert 30 == 5` (top_n
ignored) and `assert 0 == 5` (empty json); with this fix all pass.
Backend-only.

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-07-01 10:47:39 +08:00
Taranum Wasu
e23f63bd93 fix(agent): prevent empty LLM user message after prompt fitting (#16413)
## Summary
- Treat `max_tokens=0` as unset (`or 8192`) when building model context
budgets, fixing agents that silently zeroed prompts when a vLLM model
had `max_tokens: 0` in tenant config
- Replace trailing same-role canvas history in `LLM._sys_prompt_and_msg`
instead of skipping the current user prompt
- Add `LLM.fit_messages()` validation after `message_fit_in` on agent
paths so empty user content fails fast with a clear error instead of
reaching vLLM

Fixes #16411

## Root cause
Agent canvas workflow called `message_fit_in` with `int(max_length *
0.97)`. When `max_length` was `0`, both system and user content were
trimmed to empty strings. The `[HISTORY STREAMLY]` log showing only
`{"role":"user","content":""}` matches this. A secondary bug skipped
appending the formatted user prompt when history ended with a `user`
role message.

## Test plan
- [x] Added `test/unit_test/agent/component/test_llm_prompt.py` for
role-replace, validation, and zero-budget fitting
- [x] Added
`test_message_fit_in_zero_budget_preserves_non_empty_messages` in
`test_generator_message_fit_in.py`
- [ ] CI unit tests
- [ ] Manual: agent canvas `begin → Retrieval → Agent → Message` with
vLLM Qwen3; confirm user message reaches LLM

Made with [Cursor](https://cursor.com)

---------

Co-authored-by: Taranum Wasu <taranumwasu@Taranums-MacBook-Pro.local>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-01 09:30:54 +08:00
Harsh Kashyap
45fc7feab4 fix(common/time_utils): correct None/empty timestamp fallback and ISO 8601 parsing (#16483)
Recovery PR for #16173 after the fork branch was accidentally reset
during rewrite-cleanup.

Cherry-picked onto current `main`:
- fix(common/time_utils): correct fallback timestamp and ISO-8601
normalization
- fix(common/time_utils): preserve zero timestamps and mark regression
tests
- test(common/time_utils): make fallback assertions deterministic

Supersedes closed #16173 — same branch
`Harsh23Kashyap/fix/time-utils-edgecases`, rebuilt per @yuzhichang
recovery steps in
https://github.com/infiniflow/ragflow/pull/16173#issuecomment-4829663835

---------

Co-authored-by: Harsh Kashyap <harshkashyap@Harshs-MacBook-Pro.local>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-30 22:30:44 +08:00
Rene Arredondo
dc8b6d767c fix(agent): inject uploaded attachments into LLM context (#15215) (#15220)
## Summary

Fixes #15215 — attachments uploaded to an agent were not reaching the
LLM.

When a user uploads a file in an agent chat, `canvas.run` parses it into
the `sys.files` global (text content for documents, `data:image/...`
URIs
for images — see `agent/canvas.py:752-768`). But the LLM/Agent
component's
`_prepare_prompt_variables` only substitutes variables the user's prompt
template explicitly references via `{var}` placeholders. The default
prompt is `[{"role": "user", "content": "{sys.query}"}]` with no
`{sys.files}`, so the parsed attachment content never reaches the model.

In the reporter's logs, this is why the agent saw only the bare query
`附件 摘要 attachment summary` and went searching the dataset instead of
reading the uploaded PDF.

## Fix

`agent/component/llm.py` — added `_collect_sys_files()` and an
auto-injection step in `_prepare_prompt_variables`:

- If `sys.files` is non-empty **and** neither `sys_prompt` nor any entry
  in `prompts` already contains `{sys.files}` (no double-injection),
  split the entries into text vs. `data:image/...` URIs.
- Image URIs are merged into `self.imgs`, which the existing logic uses
  to switch the chat model to `IMAGE2TEXT` and pass `images=...` to
  `async_chat`.
- Text content is appended to the last `user` role message in `msg`,
  mirroring how `dialog_service.async_chat_solo` handles attachments for
  the non-agent chat path (`api/db/services/dialog_service.py:318-321`).

Both `LLM._invoke_async` and `Agent._invoke_async` (tool-using) go
through `_prepare_prompt_variables`, so plain LLM nodes and Agent nodes
are fixed in both streaming and non-streaming paths.

## Test plan

- [ ] Upload a PDF attachment to an agent with the default `{sys.query}`
prompt and ask "summarize the attachment" — the model should answer
      from the file content rather than searching the knowledge base.
- [ ] Upload an image attachment to an agent and ask about its contents
—
      the model should switch to the vision-capable LLM and answer from
      the image.
- [ ] Verify that an agent whose prompt **does** include `{sys.files}`
      still works and does **not** include the file content twice.
- [ ] Verify that an agent run with no attachments behaves unchanged.
- [ ] Run `uv run pytest` to make sure no existing tests regress.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

---------

Co-authored-by: yzc <yuzhichang@gmail.com>
2026-06-30 15:48:59 +08:00
Attili-sys
5fc254eb2e Feature big query connector (#15871)
### What problem does this PR solve?

This PR adds Google BigQuery as a first-class data source connector in
RAGFlow.

It enables users to ingest and sync BigQuery data using the same
row-to-document model used by relational database connectors: selected
content columns become document text, metadata columns become document
metadata, an optional ID column provides stable document IDs, and an
optional timestamp column enables cursor-based incremental sync.

The connector supports service-account JSON credentials, table mode,
custom query mode, GoogleSQL queries, cursor-based incremental sync,
deleted-row pruning support, configurable query limits such as
`maximum_bytes_billed`, dry-run validation, batch loading, stable
document IDs, and BigQuery-aware value serialization.
2026-06-29 22:08:40 +08:00
Harsh Kashyap
ebd4f4e633 fix(rag/nlp): handle non-numbered DOCX heading styles (#16219)
## What problem does this PR solve?

DOCX parsing could crash when a paragraph used a `Heading`-prefixed
style without a trailing numeric level, such as `Heading`, `Heading1`,
or `Heading Title`.

`docx_question_level()` assumed every heading style looked like `Heading
N` and called `int(p.style.name.split(" ")[-1])`. For non-numbered
heading styles, that raises `ValueError` and breaks Manual, Q&A, and
Laws chunking.

This PR parses heading levels safely and falls back to level 1 for
Heading-prefixed styles without an explicit numeric suffix.

Closes #16163.

## Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Test Case (non-breaking change which adds test coverage)
2026-06-29 15:21:17 +08:00
jony376
8fb692f10a fix(agent): enforce document access on POST /api/v1/agents/rerun (#15145)
## Related issues

Closes #15144

### What problem does this PR solve?

`POST /api/v1/agents/rerun` loaded a pipeline operation log by UUID via
`PipelineOperationLogService.get_documents_info` with no authorization,
then wiped chunks, reset document counters, deleted tasks, and re-queued
dataflow for the victim document.

Any authenticated user who knew a victim's pipeline log id could disrupt
parsing on documents they did not own.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Changes

| File | Change |
|------|--------|
| `api/apps/restful_apis/agent_api.py` | Call
`DocumentService.accessible(doc["id"], tenant_id)` before destructive
rerun operations; deny with generic `"Document not found."` |
|
`test/unit_test/api/apps/restful_apis/test_rerun_agent_authorization.py`
| Unit tests: cross-tenant log rejected, missing/unauthorized same
message, authorized rerun proceeds |

### Security notes

- **CWE-639:** Closes cross-tenant pipeline rerun / chunk wipe via
leaked log UUID.
- `tenant_id` from `@add_tenant_id_to_kwargs` is `current_user.id`;
`DocumentService.accessible` covers team-shared KBs.

### Test plan

- [ ] `pytest
test/unit_test/api/apps/restful_apis/test_rerun_agent_authorization.py`
- [ ] Manual: attacker cannot rerun victim pipeline log id

```bash
cd ragflow
uv run pytest test/unit_test/api/apps/restful_apis/test_rerun_agent_authorization.py -q
```

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-06-29 09:45:17 +08:00
Tim Wang
f0f10b6092 Fix: UserFillUp interactive forms not working in agent explore mode (#14589)
## Summary

- **Backend**: `_iter_session_completion_events` in `agent_api.py` was
filtering out `user_inputs` and `workflow_finished` SSE events, causing
agents with UserFillUp components to silently fail in explore mode — the
interactive form never appeared, while the same agent worked correctly
in run (editor) mode.
- **Frontend**: `SessionChat` component in explore mode was missing
`DebugContent` children rendering inside `MessageItem`, so even if the
backend forwarded the events, the form UI would not render. Added
`DebugContent`, `MarkdownContent`, `useAwaitCompentData` hook, and
input-disabling logic to match the run mode's `chat/box.tsx` behavior.

## What was changed

### Backend (`api/apps/restful_apis/agent_api.py`)
- Line 266: Added `"user_inputs"` and `"workflow_finished"` to the
allowed event filter in `_iter_session_completion_events`

### Frontend (`web/src/pages/agent/explore/components/session-chat.tsx`)
- Added imports: `DebugContent`, `MarkdownContent`,
`useAwaitCompentData`, `useParams`
- Added `sendFormMessage` from `useSendSessionMessage()` hook
- Added `useAwaitCompentData` hook for form state management
- Added `DebugContent` as `MessageItem` children for the latest
assistant message (renders UserFillUp form)
- Added `MarkdownContent` + submitted values display for previous
assistant messages
- Updated `NextMessageInput` disabled states to respect `isWaitting`
(form submission in progress)

## Test plan

- [x] Agent with UserFillUp component (e.g., email draft with
send/edit/cancel options) shows interactive form in **explore mode**
- [x] Same agent continues to work correctly in **run (editor) mode**
- [x] Form submission sends data back to the agent and workflow
continues
- [x] Input field is disabled while waiting for form submission
- [ ] Agents without UserFillUp components are unaffected in explore
mode

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-06-29 09:45:17 +08:00
philluiz2323
43a9d53c72 fix(agent): enforce tenant ownership on agentbots completions/inputs (#15457)
### What problem does this PR solve?

Fixes #15456.

The SDK agent-bot routes `POST /api/v1/agentbots/<agent_id>/completions`
and `GET /api/v1/agentbots/<agent_id>/inputs`
(`api/apps/restful_apis/bot_api.py`) authenticate the caller with a beta
API token — which only yields the caller's `tenant_id` — but then load
and run the agent named in the URL **without verifying the agent belongs
to the caller's tenant**. `UserCanvasService.get_agent_dsl_with_release`
even accepts a `tenant_id` it never uses, and `begin_inputs` calls
`get_by_id` directly. Any holder of a single valid beta token could
therefore run another tenant's agent (leaking its DSL/prompts/tool
config) or read another tenant's agent metadata and begin input form,
just by substituting a victim `agent_id`.

This PR adds the project's existing ownership gate,
`UserCanvasService.accessible(agent_id, tenant_id)`, to both endpoints
right after token authentication — mirroring the checks already enforced
on the equivalent first-party routes in
`api/apps/restful_apis/agent_api.py` (lines 75/578/775) and on the
sibling `chatbot_completions` / `create_agent_session` /
`delete_agent_session` handlers in the same file. On failure it returns
the same `Can't find agent by ID: <id>` message already used by
`begin_inputs`, so it does not reveal whether an `agent_id` exists in
another tenant.

Added a regression test
(`test/unit_test/api/apps/restful_apis/test_agentbots_access_control.py`,
following the existing stubbed-loader pattern from
`test_get_agent_session.py`) asserting that an inaccessible `agent_id`
is rejected before the agent is loaded (`begin_inputs`) or executed
(`completions`), and that an accessible agent still proceeds.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-06-29 09:45:16 +08:00
jony376
7b81f63653 fix(agent): bind session_id to path agent_id on GET/DELETE agent sessions (#15374)
## Related issues

Closes #15128

### What problem does this PR solve?

`GET` and `DELETE` `/api/v1/agents/<agent_id>/sessions/<session_id>`
verified canvas access for `agent_id` in the URL but loaded/deleted
sessions only by `session_id`, without checking `conv.dialog_id ==
agent_id`.

Any user with access to **any** agent could read or delete another
agent's `API4Conversation` session (messages, references, DSL, etc.)
when they knew the session UUID.

Agent completions in the same file already enforce this binding; chat
sessions do too — these two routes were inconsistent.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Changes

| File | Change |
|------|--------|
| `api/apps/restful_apis/agent_api.py` | Require `conv.dialog_id ==
agent_id` in `get_agent_session` and `delete_agent_session_item`; return
generic `"Session not found!"` on mismatch |
| `test/unit_test/api/apps/restful_apis/test_get_agent_session.py` | Add
IDOR regression tests for GET/DELETE; fix success fixture to include
`dialog_id`; track `delete_by_id` calls |

### Test plan

- [x] Unit tests added for GET/DELETE IDOR and success paths
- [ ] `pytest
test/unit_test/api/apps/restful_apis/test_get_agent_session.py`

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-06-29 09:45:16 +08:00
seekmistar01
608fc5df4d fix(agent): Switch no longer matches an empty condition (all([]) is True) (#15644)
## Summary
Fixes the agent `Switch` component matching an **empty/all-skipped
condition** unconditionally because `all([]) is True`.

## Root cause
`res` only accumulates for items with a non-empty `cpn_id` (blank ones
`continue`). For a condition with empty `items` (or all-blank `cpn_id`),
`res == []`, and `if all(res):` is `True`, so the Switch routes to that
condition's `to` target before reaching the else/`end_cpn_ids` branch.

## Fix
```diff
-            if all(res):
+            if res and all(res):
```
An empty result set no longer counts as a match; genuinely-satisfied
"and" conditions still route (the real `all(res)` path is preserved).

## Files changed
- `agent/component/switch.py`
- `test/unit_test/agent/component/test_switch_empty_condition.py` (new)

## Verification
- `ruff check` / `ruff format --check` — clean
- Added unit tests (mirroring the existing `_FakeCanvas` component-test
pattern): an empty/all-skipped "and" condition now falls through to
`end_cpn_ids`; a genuinely-satisfied "and" condition still routes to its
target.
- Local full pytest not run (heavy RAG deps); CI validates.

## Note
Implemented with LLM assistance (model: claude-opus-4-8).

Closes #15643

---------

Co-authored-by: seekmistar01 <seekmistar01@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-06-29 09:45:16 +08:00
philluiz2323
e256d91ade fix: guard SSRF in ExeSQL agent tool DB host (#15609)
### What problem does this PR solve?

Closes #15608.

The ExeSQL agent tool (`agent/tools/exesql.py`) opens database
connections to a node-author-controlled host/port with no SSRF
validation. The sibling `test_db_connection` endpoint already validates
the host via `common.ssrf_guard.assert_host_is_safe` (added by PR
#14860), but the tool that actually performs the connection at agent run
time was left unguarded — so the guard is bypassed simply by running the
agent. An agent author can point the host at `127.0.0.1`,
`169.254.169.254` (cloud metadata), or any internal RFC1918 host/port,
turning ExeSQL into an internal port-scanner / metadata-fetch primitive.

### Fix

Mirror the accepted endpoint guard: validate (and resolve) the host
once, before the `db_type` dispatch, and connect to the validated public
IP so a later DNS change cannot rebind the host to an internal address.

- Add `from common.ssrf_guard import assert_host_is_safe`.
- `safe_host = assert_host_is_safe(self._param.host)` before the
dispatch (rejects loopback, link-local/metadata, RFC1918, and
unresolvable hosts).
- Substitute the validated IP into all 6 driver branches: mysql/mariadb,
oceanbase, postgres, mssql, trino, IBM DB2.

Adds `test/unit_test/agent/tools/test_exesql_ssrf.py` covering loopback,
link-local/metadata, RFC1918, and empty-host rejection (before any
connection), plus an allowed host dialing the validated IP.

### Validation

- `python3 -m py_compile agent/tools/exesql.py`
- `ruff check agent/tools/exesql.py
test/unit_test/agent/tools/test_exesql_ssrf.py`
- `pytest test/unit_test/agent/tools/test_exesql_ssrf.py` — 5 passed

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-06-29 09:45:16 +08:00
jiashi19
0d7ad0ed0c Feat/agent thinking switch (#15446)
### What problem does this PR solve?

This PR adds an Agent LLM setting to control thinking mode for official
providers that expose a thinking switch.

Related to #12842.  
Closes #15445.

Some providers expose thinking controls through provider-specific
request fields, but Agent LLM settings did not have a unified option for
users to enable or disable thinking mode.

This PR adds a `Thinking` selector with:

- System default
- Enabled
- Disabled
<img width="452" height="278" alt="8566b0b4-0546-4c8a-913d-f9bbd38319f6"
src="https://github.com/user-attachments/assets/25b497f7-1ba0-4bfe-940d-6fe79287d6ab"
/>
<img width="471" height="971" alt="8a0a6bee-f45f-48d5-bd83-17af260de3db"
src="https://github.com/user-attachments/assets/41ad43c1-5087-48f1-bf37-f2ca14c2be2f"
/>
Initial support is limited to the verified official providers:

- Qwen / DashScope: `enable_thinking`
- Kimi / Moonshot: `thinking.type`
- GLM / ZHIPU-AI: `thinking.type`

For LiteLLM-based providers, provider-specific fields are forwarded
through `extra_body` before `drop_params` filtering so the request
parameters are preserved.



### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: jiashi <jiashi19@outlook.com>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-06-29 09:45:16 +08:00
Harsh Kashyap
6a4de82a80 fix(agent): restore be_output and test DeepL error return (#16363)
## Summary

#16332 fixed the missing `return` in DeepL's except branch, but
`ComponentBase.be_output` was removed during the agent refactor (#9113)
while several components still call it. DeepL (and other tools) would
raise `AttributeError` before any error message could be returned.

- Restore `ComponentBase.be_output` as `pd.DataFrame([{"content": v}])`
(same as pre-refactor behavior)
- Add regression test that `_run` returns the `**Error**:` message when
translation fails

Related to #16329

## Test plan

- [x] `test_run_returns_error_on_translation_failure`
- [x] Existing `test_deepl.py` check() tests still pass

---------

Co-authored-by: Harsh Kashyap <harshkashyap@Harshs-MacBook-Pro.local>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-06-29 09:45:16 +08:00