### What problem does this PR solve?
Closes#15872
On pages that contain both a textless equation and a figure, the layout
recognizer could merge the two unrelated regions into a single cropped
image with concatenated captions. This shows up most often on scientific
and technical PDFs.
The cause is a `layoutno` namespace collision in
`deepdoc/vision/layout_recognizer.py`. Text-overlapping boxes are tagged
per type by `findLayout`: figures become `figure-{ii}` using the
figure-only index, and equations become `equation-{ii}` using the
equation-only index. The fallback loop that handles textless regions,
however, indexes the combined figure plus equation list and always uses
a `figure` prefix.
Because the two paths use different index spaces and prefixes, a page
laid out as `[textless equation, figure with text]` produces two boxes
tagged `figure-0`. `_extract_table_figure` in
`deepdoc/parser/pdf_parser.py` buckets boxes by `f"{page}-{layoutno}"`,
so both fall into the same `page-figure-0` bucket, and `cropout`
stitches the disjoint regions into one image.
**Fix**
The fallback loop now iterates per type, indexes within each type's own
list, and reuses the type as the `layoutno` prefix (`figure-{i}` or
`equation-{i}`). This matches the namespace that `findLayout` already
assigns to text-overlapping boxes. Since the `visited` flag is shared by
reference, each layout is tagged by exactly one path, so per-type
indices stay collision free. Textless equations now land under
`equation-N`, consistent with text-overlapping equations, instead of the
old `figure-N`.
The same fix is applied to `AscendLayoutRecognizer`, which shared the
identical defect.
Downstream consumers of `layoutno` were checked and all treat it as an
opaque equality or bucketing key, so no other code paths are affected.
When a page has no equations, the combined index equals the figure-only
index and the output is unchanged.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
## Summary
- Register `TavilySearch` and `TavilyExtract` Canvas component names in
the Go Agent tool registry.
- Add regression coverage for building both tools by their Python
component names.
## Testing
- `bash build.sh --test ./internal/agent/tool/...`
- `bash build.sh --test ./internal/agent/component/...`
## Summary
Old `zhipuai==2.0.1` SDK (MetaGLM/zhipuai-sdk-python-v4) requires
`pyjwt~=2.8.0`, which blocks upgrading pyjwt past 2.8.0 to address
active CVEs. The upstream project now recommends the maintained
successor [`zai-sdk`](https://github.com/zai-org/z-ai-sdk-python).
`zai-sdk` relaxes the pyjwt constraint to `>=2.9.0,<3.0.0`, allowing
pyjwt 2.13.0+. The embedding API surface (`client.embeddings.create`) is
identical, no functional changes required.
Related CVE's:
- CVE-2026-48522
- CVE-2026-48524
- CVE-2026-48525
- CVE-2026-48526
- CVE-2026-32597
### Summary
As title:
Some operations on datasets have been fixed as they previously lacked
authentication, preventing team members from proper or correct usage.
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
### Summary
As title
verif ied from CLI
```
RAGFlow(api/default)> asr with 'paraformer@test@funasr' audio './internal/test.wav' param '{"language": "en"}'
+----------------------------------------------------------------------------------------------------------------------+
| text |
+----------------------------------------------------------------------------------------------------------------------+
| The examination and testimony of the experts enabled the commission to conclude that five shots may have been fired. |
+----------------------------------------------------------------------------------------------------------------------+
```
---
<img width="2876" height="1377" alt="image"
src="https://github.com/user-attachments/assets/4cee62c6-1c71-43ce-b398-4d9fbff33c3c"
/>
```bash
INFO: 127.0.0.1:51910 - "POST /v1/audio/transcriptions HTTP/1.1" 200 OK██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 2.19it/s]
INFO: 127.0.0.1:60934 - "GET /v1/models HTTP/1.1" 200 OK%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 2.10it/s]
```
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
## Summary
Three groups of changes across the Go ingestion pipeline:
### 1. DOCX parsing improvements
- **docx_parser.go**: Enhanced DOCX parsing with better structure
extraction and media handling
- **docx_parser_cgo_test.go**, **docx_parser_test.go**: Companion tests
- **office_parsers_no_cgo.go**: Stub sync for non-CGO builds
### 2. Email (.eml) parsing: base64 Content-Transfer-Encoding decoding
- **email_parser.go** (`decodeCTE`): Added Content-Transfer-Encoding
decoding for base64 and quoted-printable. Go's `mime/multipart.Reader`
does not decode Content-Transfer-Encoding automatically, so attachments
with `Content-Transfer-Encoding: base64` remained base64-encoded in the
output. The new `decodeCTE` helper is called after reading each
multipart part's raw bytes in `readMailBody`, mirroring Python's
`part.get_payload(decode=True)`.
- **email_parser_test.go**: Two new tests — simple base64 attachment and
nested multipart/alternative with base64 attachment.
### 3. Extractor LLM driver fix + ModelDriver consolidation
- **extractor.go**: Fixed a bug where the Extractor component used
`ModelFactory.CreateModelDriver()`, which creates bare model instances
without API keys or provider configuration. Switched to
`models.GetPreconfiguredDriver()` which resolves the actual
pre-configured driver from `ProviderManager`, matching the codepath used
by `llm.go`. This fixes auto keyword/question extraction in DSL
pipelines that require LLM calls.
- **get_driver.go** (new): Extracted shared `GetPreconfiguredDriver()`
from `llm.go:newChatModelDriver()` so both `llm.go` and `extractor.go`
use the same codepath.
- **get_driver_test.go** (new): Tests for the shared driver resolution.
- **llm.go**: Replaced inline driver resolution with
`models.GetPreconfiguredDriver()`.
### 4. Chunker fixes and observability
- **group.go** (`extractLineRecords`): Fixed to also read `markdown` and
`html` payload keys — previously it only read `text`/`content`, causing
GroupTitleChunker to silently return empty results for markdown-format
parser output.
- **common.go** (`compileDelimPattern`): Aligned with Python's
`_compile_delimiter_pattern` — only backtick-wrapped delimiters produce
an active regex pattern; plain delimiters are not compiled into the
split regex.
- **token.go** (`applyChildrenDelim`): Set `DocType` and `CKType` to
`"text"` on created ChunkDocs so the token-size merge path correctly
identifies and merges text segments.
- **parser.go**, **extractor.go**, **tokenizer.go**, **group.go**,
**hierarchy.go**: Added debug-level logging for pipeline diagnostics.
- **parser_dispatch_test.go**, **group_test.go**: New tests.
## Verification
- All Go tests pass: `bash build.sh --test ./internal/parser/parser/...`
and `bash build.sh --test ./internal/ingestion/component/...`
- Build succeeds: `bash build.sh --go`
