zlei6/ragflow
1
0
Fork 0
mirror of https://github.com/infiniflow/ragflow.git synced 2026-08-15 21:16:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
8,331 Commits 5 Branches 109 Tags
fa5762336baa721dbbb06ec5f2b6449d3daca768
Commit Graph

1064 Commits

Author SHA1 Message Date
Jin Hai
fa5762336b Go: fix context, part2 (#18296) 
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2026-08-14 19:42:41 +08:00
euvre
01d351b54a Go: align cancel-parse error with Python for un-started documents (#18245) 2026-08-14 19:22:36 +08:00
Jack
2678a0c5f5 deepdoc: send lossless PNG instead of JPEG to the DeepDoc inference service (#18282) 2026-08-14 18:49:33 +08:00
euvre
3d50f8bcfc fix(go-agent): forward sys.files uploads to vision LLMs (#18246) 2026-08-14 18:26:09 +08:00
Jin Hai
2c775d0549 Go: fix context (#18291) 
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2026-08-14 17:50:13 +08:00
Haruko386
5bb5ba2216 fix: return total numbers of the search result (#18071) 
### Summary

As title
 2026-08-14 17:44:38 +08:00
Haruko386
d77b514d3e feat[syncer]: add checkpoint resume run for Github (#18260) 
### Summary

As title

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
 2026-08-14 17:44:25 +08:00
Haruko386
a2e6ddb56d feat[syncer]: add checkpoint resume run for RSS (#18257) 
### Summary

As title

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
 2026-08-14 17:44:10 +08:00
Haruko386
cdbd82df4f fix: return total number of docs when sync has no update (#18255) 
### Summary

As title

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
 2026-08-14 17:43:55 +08:00
euvre
fe963e69ba fix: route TokenChunker delimiter_mode "one" to OneChunker in Go ingestion (#18121) 2026-08-14 16:55:32 +08:00
Jack
ae256bcf59 feat(chunker): add ManualChunker for manual-layout PDFs (#18272) 
Add `ManualChunker`, the Go port of Python's `manual` doc-type chunk method (`rag/app/manual.py`). Like `GroupTitleChunker` it merges adjacent text records into heading-bounded groups, but it first re-sorts the records into physical reading order before grouping.
 2026-08-14 16:55:08 +08:00
euvre
e1330423ac fix(agent): keep think tags in persisted agent session messages (#18222) 2026-08-14 16:00:08 +08:00
Jack
c23d5fc819 fix(parser): support .msg parsing and make email attachments retrievable (#18198) 
- Add Go `EmailParser` support for Outlook `.msg` (OLE2/CFB) files via
the `gomsg` library, in addition to the existing `.eml` (RFC 5322)
support. The `.msg` hard-error is gone; emails with `.msg` attachments
are now ingested end-to-end.
- Re-chunk email attachments into retrievable text (user-oriented). Each
attachment is re-parsed by its file extension through the shared parser
registry and folded back into the same document, so attachment content
becomes searchable. This mirrors Python's legacy `rag/app/email.py`.
Binary attachments (images/audio/video/folders) are skipped by design.
- Restore a corrupted `sample.msg` test fixture and add guards so binary
fixtures are never mangled again (`.gitattributes` marks `*.msg` binary;
`check_files.py` skips NUL-byte files). Also made `check_files.py`
ruff-clean.
 2026-08-14 10:38:38 +08:00
Zhichang Yu
620f807cb5 Port agentic search high/ultra research loop to Go (#18242) 
Implement two-level research loop, sufficiency ladder, AutoRater, grounded review, pipeline, inspector tools, and graph exploration for high/ultra agentic search modes.
 2026-08-14 10:16:46 +08:00
Jack
423c8489b5 fix(chunker): carry overlap-head PDF positions into new chunk (#18148) (#18227) 
When `TokenChunker` starts a fresh chunk with an overlap prefix (Go `computeOverlapPrefix` / Python visible-text cut), the previous chunk's **tail PDF coordinates were dropped**. As a result, the overlap head of a PDF chunk is displayed but **not highlighted** — the highlight box is shifted/truncated relative to the displayed span (infiniflow/ragflow#18148).
 2026-08-13 22:18:33 +08:00
Jin Hai
811f9dd0df Go: refactor embed and rerank interface (#18240) 
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2026-08-13 22:14:23 +08:00
Jin Hai
e2acf3aebb Go: add error logs when fail to init infinity (#18235) 
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
 2026-08-13 20:17:53 +08:00
mkaaad
3ce1b9be3d Add MySQL and PostgreSQL connectors to Go syncer (#18162) 
As title

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
 2026-08-13 20:16:44 +08:00
mkaaad
0dbfaa8b67 feat(cli): add sync logs listing commands for the user CLI (#18195) 
### Summary
```
RAGFlow(api/default)> LIST SYNC LOGS WITH PAGE_SIZE 2 PAGE 2;
Total: 764
+----------------------------------+-------------------------+-------------+-----------+----------------------------------+----------------------------------+---------+------------------+------------+--------------+--------+-----------+---------------------+--------------------+---------------------+
| connector_id                     | docs_removed_from_index | error_count | error_msg | id                               | kb_id                            | kb_name | new_docs_indexed | prune_freq | refresh_freq | status | task_type | time_started        | total_docs_indexed | update_date         |
+----------------------------------+-------------------------+-------------+-----------+----------------------------------+----------------------------------+---------+------------------+------------+--------------+--------+-----------+---------------------+--------------------+---------------------+
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |           | fb55b5f1fb754475834cd9eac8b52054 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:45 | 16                 | 2026-08-13 09:49:46 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |           | 3cad779d7d2e4efb8d8d2dbf05d45653 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:42 | 16                 | 2026-08-13 09:49:43 |
+----------------------------------+-------------------------+-------------+-----------+----------------------------------+----------------------------------+---------+------------------+------------+--------------+--------+-----------+---------------------+--------------------+---------------------+

```
```
RAGFlow(api/default)> LIST SYNC LOGS;
Total: 764
+----------------------------------+-------------------------+-------------+--------------------------------------------------------------------------------------------+----------------------------------+----------------------------------+---------+------------------+------------+--------------+--------+-----------+---------------------+--------------------+---------------------+
| connector_id                     | docs_removed_from_index | error_count | error_msg                                                                                  | id                               | kb_id                            | kb_name | new_docs_indexed | prune_freq | refresh_freq | status | task_type | time_started        | total_docs_indexed | update_date         |
+----------------------------------+-------------------------+-------------+--------------------------------------------------------------------------------------------+----------------------------------+----------------------------------+---------+------------------+------------+--------------+--------+-----------+---------------------+--------------------+---------------------+
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 1           | sync task failed: failed to fetch URL: Get "https://xxxxxxxxxxxx/rss.xml": EOF             | 87592e5623fc4284bf69b9290f322718 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 4      | sync      | 2026-08-13 09:49:48 | 16                 | 2026-08-13 09:51:01 |
| 570138201cf44e1180711f1ebdf8492f | 0                       | 1           | sync task failed: failed to fetch URL: Get "https://www.ruanyifeng.com/blog/atom.xml": EOF | 456adbbf21a147af99adfce4405abd99 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 1            | 4      | sync      | 2026-08-13 09:50:00 | 3                  | 2026-08-13 09:50:18 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | fb55b5f1fb754475834cd9eac8b52054 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:45 | 16                 | 2026-08-13 09:49:46 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | 3cad779d7d2e4efb8d8d2dbf05d45653 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:42 | 16                 | 2026-08-13 09:49:43 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | dd09320aa5b34d6cbcfa7b03ede68c4a | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:36 | 16                 | 2026-08-13 09:49:38 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | afbcee73240f420ca1997e6506f387fb | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:33 | 16                 | 2026-08-13 09:49:34 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | 7d018146a0f14a25b7263d5980fdc30f | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:30 | 16                 | 2026-08-13 09:49:32 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | 520e0d6f421e40628ab541fc4fee1a69 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:27 | 16                 | 2026-08-13 09:49:28 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | 3d3ce49825de4ef99642e7994ae954e3 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:21 | 16                 | 2026-08-13 09:49:22 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | 1c511f9b4a7c425fb30a956589ca56fa | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:18 | 16                 | 2026-08-13 09:49:19 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | b9ab73c3f48c4f47a84fd55b542b32b4 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:15 | 16                 | 2026-08-13 09:49:16 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | db0cad90a7744d758b24d2e3d3828963 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:12 | 16                 | 2026-08-13 09:49:13 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | a501eadb2cce452187f3c86431352c70 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:08 | 16                 | 2026-08-13 09:49:10 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | b4265a3b7baf4466a525b91a9fe21b71 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:03 | 16                 | 2026-08-13 09:49:05 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | a6ad590931c046dda5fc3222b4010c06 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:00 | 16                 | 2026-08-13 09:49:02 |
......
+----------------------------------+-------------------------+-------------+--------------------------------------------------------------------------------------------+----------------------------------+----------------------------------+---------+------------------+------------+--------------+--------+-----------+---------------------+--------------------+---------------------+

```
list all logs
```
RAGFlow(api/default)> LIST DATASET 'test' SYNC LOGS;
Total: 764
+----------------------------------+-------------------------+-------------+--------------------------------------------------------------------------------------------+----------------------------------+----------------------------------+---------+------------------+------------+--------------+--------+-----------+---------------------+--------------------+---------------------+
| connector_id                     | docs_removed_from_index | error_count | error_msg                                                                                  | id                               | kb_id                            | kb_name | new_docs_indexed | prune_freq | refresh_freq | status | task_type | time_started        | total_docs_indexed | update_date         |
+----------------------------------+-------------------------+-------------+--------------------------------------------------------------------------------------------+----------------------------------+----------------------------------+---------+------------------+------------+--------------+--------+-----------+---------------------+--------------------+---------------------+
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 1           | sync task failed: failed to fetch URL: Get "https://xxxxxxxxxxxx/rss.xml": EOF             | 87592e5623fc4284bf69b9290f322718 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 4      | sync      | 2026-08-13 09:49:48 | 16                 | 2026-08-13 09:51:01 |
| 570138201cf44e1180711f1ebdf8492f | 0                       | 1           | sync task failed: failed to fetch URL: Get "https://www.ruanyifeng.com/blog/atom.xml": EOF | 456adbbf21a147af99adfce4405abd99 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 1            | 4      | sync      | 2026-08-13 09:50:00 | 3                  | 2026-08-13 09:50:18 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | fb55b5f1fb754475834cd9eac8b52054 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:45 | 16                 | 2026-08-13 09:49:46 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | 3cad779d7d2e4efb8d8d2dbf05d45653 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:42 | 16                 | 2026-08-13 09:49:43 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | dd09320aa5b34d6cbcfa7b03ede68c4a | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:36 | 16                 | 2026-08-13 09:49:38 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | afbcee73240f420ca1997e6506f387fb | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:33 | 16                 | 2026-08-13 09:49:34 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | 7d018146a0f14a25b7263d5980fdc30f | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:30 | 16                 | 2026-08-13 09:49:32 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | 520e0d6f421e40628ab541fc4fee1a69 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:27 | 16                 | 2026-08-13 09:49:28 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | 3d3ce49825de4ef99642e7994ae954e3 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:21 | 16                 | 2026-08-13 09:49:22 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | 1c511f9b4a7c425fb30a956589ca56fa | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:18 | 16                 | 2026-08-13 09:49:19 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | b9ab73c3f48c4f47a84fd55b542b32b4 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:15 | 16                 | 2026-08-13 09:49:16 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | db0cad90a7744d758b24d2e3d3828963 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:12 | 16                 | 2026-08-13 09:49:13 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | a501eadb2cce452187f3c86431352c70 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:08 | 16                 | 2026-08-13 09:49:10 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | b4265a3b7baf4466a525b91a9fe21b71 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:03 | 16                 | 2026-08-13 09:49:05 |
| 02b5b14183884445bd55ad6b9d322204 | 0                       | 0           |                                                                                            | a6ad590931c046dda5fc3222b4010c06 | 0e6e28e65b7044e58070e0e4252319a6 | test    | 0                | 5          | 0            | 3      | sync      | 2026-08-13 09:49:00 | 16                 | 2026-08-13 09:49:02 |
......
+----------------------------------+-------------------------+-------------+--------------------------------------------------------------------------------------------+----------------------------------+----------------------------------+---------+------------------+------------+--------------+--------+-----------+---------------------+--------------------+---------------------+
```
 2026-08-13 18:39:29 +08:00
Haruko386
96f7c39753 feat[Go]: resuming transmission from the point of interruption during data source synchronisation (#18176) 
### Summary

As title
 2026-08-13 18:38:37 +08:00
Zhichang Yu
0784bef5b0 Port dataset-level structure merge for timeline/graph/mindmap (#18201) 
Ports Python dataset-level structure aggregation (timeline, graph, mindmap) to Go. Mindmap emits entity/relation rows and merges like graph. Adds dataset_merge guard, engine gate, resolveDatasetStructureKind, kind-required structure graph GET/DELETE API, per-index task-id fields.
 2026-08-13 18:37:47 +08:00
Haruko386
a6f2eac496 fix: unable to pass Temperature and MaxTokens in agent (#18223) 
### Summary

As title
 2026-08-13 18:31:08 +08:00
wenqian
c589a5d050 feat: add UCloud agent sandbox provider (#18088) 2026-08-13 15:33:03 +08:00
euvre
f240568a78 fix(agent): accept 1D files wire shape in chat/completions (#18197) 2026-08-13 14:26:54 +08:00
jay77721
0a4cc65db0 refactor(common): unify </think>-strip into shared StripThinkTrailing helper (#18179) 
Collapse the nine identical **"cut through the last `</think>`"**
implementations — mirroring Python's `re.sub(r"^.*</think>", "", s,
re.DOTALL)` — into one shared helper `common.StripThinkTrailing`,
preventing future behavior drift between copies.
 2026-08-13 13:56:56 +08:00
jay77721
e1b8366ecc refactor: remove resume (CV) workflow DSL template and Go handling code (#18177) 2026-08-13 10:47:37 +08:00
euvre
b1de7e8136 Fix: allow datasets sharing a base embedding model to be searched together (#18166) 2026-08-13 09:51:44 +08:00
Zhichang Yu
6677f14bdf Port dataset nav and structure graph fixes to Go, merge agents list (#18183) 
Fix compilation template config validation for JSONMap; merge template groups into agents list ordered by category/name; install nav service in ingestor; write readable nav cluster/doc names and emit nav_doc leaves;
port tree-to-graph projection and full document structure graph endpoint parity.
 2026-08-12 22:46:24 +08:00
Jack
b505437db5 fix(chunker): honor bare (non-backtick) delimiters in TokenChunker (#17723) (#18182) 
Go's `TokenChunker` previously ignored **bare (non-backtick)
delimiters** such as `::`, `.`, or `；`. `compileDelimPattern` compiled
them with `keepBare=false`, so a bare delimiter produced a `nil` pattern
and the payload was routed to the single-section merge that never split
on it — diverging from Python's `naive_merge`, which splits on bare
delimiters and then merges by token size.
 2026-08-12 22:36:11 +08:00
EthanZhang
1d8b9c7452 feat(agent): add Querit Contents tool (#18156) 
### Summary

Add Querit Contents as a built-in page content tool for RAGFlow Agents
and Canvas workflows.

Querit Contents crawls one or more known URLs and returns their page
content and optional metadata. It complements the existing Querit Search
tool: Search discovers relevant pages, while Contents retrieves pages
already selected by an Agent or workflow.

This integration supports two usage modes:

- A standalone `QueritContents` node in Canvas workflows.
- An embedded content tool available to RAGFlow Agents.
 2026-08-12 21:03:21 +08:00
Jack
c7db20b8d3 test(parser): pin figure-aware reading order in PDF post-process (#18178) 
Adds a characterization / regression test
(`pdf_postprocess_measure_test.go`) that pins the **figure-aware reading
order** produced by the PDF adapter layer
(`internal/parser/parser/pdf_postprocess.go`).
 2026-08-12 20:28:23 +08:00
jay77721
207e2eaf3b fix(ingestion): strip mid-text think blocks in extractor metadata path (#18175) 2026-08-12 19:39:49 +08:00
jay77721
91fd114783 fix(dao): honor tenant-configured context window override (D22/D23) (#18171) 
Make `ResolveModelContentLength` honor the per-model custom **context window length** (`content_length`) — stored in the Python-legacy `tenant_model.extra["max_tokens"]` field, whose semantic meaning is the context window, NOT the generation cap — **before** any provider-catalog read, and remove the parallel service-layer implementation so every consumer shares one resolution path.
 2026-08-12 19:32:37 +08:00
euvre
3d64f8d044 fix(ingestion): remove broken built-in Resume pipeline (#18173) 2026-08-12 19:11:49 +08:00
Jack
8c5c870470 fix(parser): emit a single structured table item in document order (#18168) 2026-08-12 18:36:21 +08:00
Jack
7646afab1b fix(pdf/layout): recover title-bridged & gutter-less double columns (2D rescue + L3) (#18150) 
Incremental follow-up to #18023 (gap + balance-gate hybrid). Adds two
complementary column detectors to `AssignColumn` that run **only after**
the gap detector and the balance gate both fail, so already-correct
pages are never touched.
 2026-08-12 17:59:50 +08:00
Jack
a4dbd898bb docs(ingestion): correct stale parser-component scope comment (#18147) 2026-08-12 17:58:05 +08:00
Zhichang Yu
c677e9af36 Port dataset-level knowledge compile to Go with variant dispatch (#18161) 
Ports dataset-level knowledge compilation (tree/structure/wiki) to Go:
add compile-type variants to backlog events, route per-variant
dataset-level paths, move dataset-nav to the consumer, add structure
merge and per-variant clean, plus rebuild variant recovery.
 2026-08-12 17:24:12 +08:00
jay77721
d7682b5b92 fix(agent): fit LLM prompts to model content_length (#18093) 
Use the chat model's context window (`content_length`) as the message-fitting budget instead of the canvas `max_tokens` parameter.
 2026-08-12 17:23:43 +08:00
jay77721
86b6f64abb fix(ingestion): fit extractor and tagger prompts to model context (#18095) 
Trim Extractor call prompts and the automatic tagger prompt to the chat model's context window (`content_length`) before sending, so oversized chunks or tag files are trimmed instead of rejected by the provider with a context-length error.
 2026-08-12 15:45:01 +08:00
jay77721
341e913b7f refactor(agent): use shared message-fit in LLM component (#18092) 
Switch the agent LLM component's `fitMessages` from the local map-based `messageFitInRaw`/`countAllTokens`/`stringContent`/`setContent` helpers to the new shared `internal/component/messagefit` package (#18091).
 2026-08-12 15:43:46 +08:00
euvre
247c56fb4d fix: enable preview for legacy .doc documents in Go server (#17767) 2026-08-12 15:43:10 +08:00
euvre
da8cbca024 Go: clear stale task cancel flag so first re-parse after cancel runs (#17708) 2026-08-12 14:55:46 +08:00
euvre
27edad8a76 Go: implement ?type=filter branch for GET /agents (#18111) 2026-08-12 14:54:48 +08:00
euvre
f8b2b1248c fix: drop static catalog check when resolving tenant default models (#17714) 2026-08-12 14:46:37 +08:00
euvre
fdb6b5fdd2 fix: search files recursively within current folder subtree (#17779) 2026-08-12 14:43:06 +08:00
mkaaad
1b02abd487 Honor explicit model URL overrides for NVIDIA endpoints (#18152) 
Some NVIDIA hosted models (e.g. meta/llama-3.2-11b-vision-instruct )
expose a full
endpoint URL per model that does not follow the normal base_url +
url_suffix
assembly. Previously the Go driver always called {base}/chat/completions
, so chat
requests for these vision models hit the wrong endpoint and failed.

This PR adds an optional per-model url field in conf/models/nvidia.json
. When
present, every NVIDIA driver request (chat, streaming chat, embedding,
rerank, model
listing) uses it directly; otherwise the standard assembly is unchanged.
 2026-08-12 14:17:13 +08:00
Jack
7b2d052f8a refactor(parser): align TextParser with Python _code delimiter split (#18128) 2026-08-12 13:07:23 +08:00
jay77721
a9123368c5 feat(dao): add shared ResolveModelContentLength helper (#18106) 
Add a shared `dao.ResolveModelContentLength` that resolves a chat model's context window (`content_length`) from a `tenant_model` UUID or a composite `model@provider` reference, with an optional `driver + modelName` catalog fallback for the no-database path.
 2026-08-12 13:04:40 +08:00
jay77721
fb08a6c5ee feat(component): add shared message-fit package (#18091) 
The agent LLM component and the ingestion Extractor component both need
to trim prompts to the model's context window before calling the
provider. Each previously did (or would do) this with its own copy of
the logic. This PR adds the shared primitive; follow-up PRs wire it into
the agent LLM component (#18092) and the ingestion Extractor/tagger
(#18095).
 2026-08-12 13:03:45 +08:00