An image whose OCR text is shorter than the CV LLM threshold produces zero chunks when the tenant has no image2text model configured. The extracted text is discarded.
## Summary
- Pass `vector_similarity_weight` from Python and Go retrieval requests
into Infinity's weighted fusion expression.
- Keep fusion weights ordered as text first and vector second, with the
existing default vector weight of `0.3`.
---------
Co-authored-by: chenglinpeng <1042527908@qq.com>
### Summary
Clear the paired tenant model ID when a model selection is explicitly
cleared in a whitelisted API request.
Replaces #18205.
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
### Summary
This PR adds [Serply](https://serply.io) as a third web search provider
for chat assistants, alongside the existing Tavily and Querit options.
## Summary
Closes#5703.
Users who delete the hardcoded `{knowledge}` placeholder from the system
prompt while datasets are selected can still retrieve the right chunks,
but the assistant answers as if nothing were found — because the
retrieved content has nowhere to be injected. Likewise, a non-empty
*empty response* with **no** dataset selected fires on every turn
(nothing can ever be retrieved). This PR adds a save-time self-check
that prompts the user about both contradictory configurations, as
requested in the issue.
Co-authored-by: peewee92 <20059253+peewee92@users.noreply.github.com>
### Summary
Closes#18414.
`rag/res/term.freq` is not shipped, and both term-weight implementations
therefore assigned the same `300` fallback frequency to every lowercase
Latin token. With no tokenizer frequency, NER, or POS signal, function
words and content words received identical lexical boosts.
This PR adds the same bounded out-of-vocabulary prior to Python and Go:
- Use it only when the explicit DF dictionary or tokenizer has no
frequency.
- Count Latin, Greek, and Cyrillic letters, including uppercase and
accented forms.
- Keep the existing frequency of `300` for words up to three letters,
halve it every two additional letters, and clamp it at `10`.
- Reject digits, underscores, and logographic terms so Chinese and other
existing fine-grained-tokenizer paths are unchanged.
- Treat an absent optional `term.freq` as the supported fallback path
without a startup warning, while still logging inaccessible or malformed
dictionaries.
A corpus-derived table was intentionally not added: that would require
provenance/licensing decisions, language detection, and handling
cross-language homographs. The bounded prior is deterministic,
dependency-free, and fixes the equal-weight degradation for
whitespace-delimited alphabetic languages without claiming
corpus-specific precision.
Python and Go consume one shared fixture covering ASCII, uppercase,
accented Latin, Greek, Cyrillic, separators, invalid mixed tokens, and a
CJK non-match. Both sides also verify the issue's ordering (`was <
largest < supplier < equipment`) and that an explicit dictionary entry
still takes precedence.
Co-authored-by: Loong <184861530+yzl0ng@users.noreply.github.com>
## Summary
- require a `valid_at` timestamp in the semantic-memory output schema
- tell the extraction model to use conversation time when a fact has no
date of its own
- add a regression test for the assembled semantic prompt
This addresses the deterministic prompt inconsistency reported in
#18415. The invalid-timestamp fallback is intentionally left unchanged
because selecting its replacement policy requires a separate design
decision.
## Summary
- allow ISO 8601 normalization callers to provide an explicit fallback
while preserving the existing default behavior
- use the extraction conversation time when `valid_at` is missing or
invalid
- clear an invalid optional `invalid_at` instead of writing an
unparseable value
- include the rejected timestamp value in the error log
This addresses the timestamp write-through portion of #18415. It is
intentionally separate from #18462, which fixes the semantic output
prompt.
Co-authored-by: Loong <184861530+yzl0ng@users.noreply.github.com>
### Summary
- Forward each memory's stored system_prompt and user_prompt to
extract_by_llm.
- Cover both immediate save and queued extraction paths with focused
regression tests.
- Preserve the existing default-prompt fallback when stored prompts are
empty.
Fixes#18413.
### Summary
RAGFlow's "Create empty document" flow accepts names without a file
extension. The `POST /datasets/<dataset_id>/documents?type=empty` route
calls `_upload_empty_document()`, where `Path(name).suffix.lstrip(".")`
returns `""`.
In GaussDB's A/ORA compatibility mode, that empty string is persisted as
SQL `NULL`. Because `document.suffix` was defined as `NOT NULL`, the
insert failed with a constraint violation.
### Summary
Providers whose static catalogue is empty discover their models by
calling the base URL the user typed. `verify_api_key` wrapped that call
in a bare `except Exception: pass` and then returned a flat `No models
found for provider 'X'`, so an unreachable host, a closed port, a wrong
scheme and a bad TLS setup all produced the same sentence, with the
actual error discarded and not even logged.
### What problem does this PR solve?
- Update version tags in README files (including translations) from
v0.26.4 to v0.27.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files
### Type of change
- [x] Documentation Update
## Summary
- treat empty Markdown binaries as in-memory content instead of local
file paths
- add a regression test ensuring empty content does not access the
filesystem
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
### Summary
`AsyncExecutor.Execute` acquires a worker-pool slot, while
`ExecuteWithRetry` duplicated the task lifecycle without acquiring one.
Pregel supplies a retry configuration for every node, so those
executions
bypassed `WithMaxConcurrency`.
This change:
- routes retry execution through the shared `Execute` path;
- waits on the task-owned context and rechecks cancellation after slot
acquisition;
- adds deterministic regression coverage for worker-pool occupancy and
queued-task cancellation.
## Summary
- Frontend `Images` already includes `bmp`
(`web/src/constants/common.ts`), and picture parsing already accepts
`.bmp` (`internal/parser/parser/picture_parser.go`, `rag/app/picture.py`
via Pillow).
- This PR adds `bmp` to both whitelist sites and a small Go unit test.
Co-authored-by: zhangjiangshan1 <zhangjiangshan1@kingsoft.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Refactor the Extractor component into a pure, unified **5-in-1 modular extraction engine** across both Dataset (`knowledgebase.parser_config`) and Pipeline (Canvas DSL).
## Summary
- Disable DeepSeek V4 thinking (chain-of-thought) by default.
- litellm 1.82.x drops `thinking: disabled`; carry the toggle through
`extra_body.thinking.type` and strip `reasoning_effort` to avoid the
400.
- Use local timezone for agent `sys.date` instead of UTC.
Reference: https://api-docs.deepseek.com/guides/thinking_mode
---------
Co-authored-by: Claude <claude@example.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude <noreply@anthropic.com>
This PR fixes two issues with MinerU PDF parser where table and image chunks were not properly classified or associated:
1. **Table chunks** were incorrectly classified as `text` instead of `table`
2. **Image chunks** were missing image resource association and classified as `text`
