Files
ragflow/internal/deepdoc/parser/pdf/mock_engine.go

46 lines
1.2 KiB
Go
Raw Normal View History

package pdf
import (
"image"
pdf "ragflow/internal/deepdoc/parser/pdf/type"
)
// MockEngine is a minimal pdf.PDFEngine stub for unit/integration tests.
type MockEngine struct {
feat(agent): Go ingestion pipeline progress mirroring and DeepDOC parser hardening (#16795) feat(ingestion): mirror Go pipeline progress into the document table; harden resume guards - pipeline: bind the owning document via WithDocumentID; after each TrackProgress event aggregate ingestion_task_log progress and mirror progress/run/progress_msg back into the document table, so GET /api/v1/datasets/{dataset_id}/documents reflects live Go pipeline progress without a bespoke endpoint. - canvas: extend the S3 resume guard to reject legacy no-op nodes (e.g. ExitLoop) so component_total equals the count of progress-reporting components and the aggregate percent can reach 100%. - runtime/canvas: route progress through TrackProgress; add interrupt test coverage (r3_interrupt_test.go). - dao/entity: add IngestionTask.DocumentID column and AggregateProgress support used by the mirror; IngestionTaskLog keeps a Checkpoint column alongside the progress fields. feat(deepdoc): cache DocAnalyzer inference results in Redis (1h TTL) - Redis-backed DocAnalyzerCache decorator over inference.Client; cache key = "ddoc:cache:<method>:" + sha256 of the JPEG-encoded image bytes (deterministic). - TTL = 1h; hits skip the inner HTTP call and return cached JSON; inner errors are not cached. refactor(deepdoc): align figure cropping with Python cropout + bounded page caches - CropSectionByDLA mirrors Python cropout: best-overlap DLA figure/equation region, fallback to section bbox per page, vertical concat on gray background. - sliding-window page-image cache bounds peak memory to the recent window instead of the whole PDF. - rename DLADebug -> DLARegions across parser/chunker/tests. refactor(parser): drop lib_type selector; align NewXxxParser with NewPDFParser - remove config["lib_type"] lookup and the libType param/field/switch from all nine constructors; surface the CGO-required error at ParseWithResult time instead of construction time; drop resolveLibType, its test, and the four lib_type constants. feat(utility): add a reusable workerpool for bounded concurrent execution - internal/utility/workerpool.go (+ tests). refactor: translate Chinese prose comments to English in non-harness Go files. chore: upgrade github.com/cloudwego/eino from v0.9.9 to v0.9.12.
2026-07-10 10:36:10 +08:00
Chars map[int][]pdf.TextChar
NumPages int
RenderW int
RenderH int
RenderPageImageFunc func(pg int, dpi float64) (image.Image, error)
}
func (m *MockEngine) ExtractChars(pg int) ([]pdf.TextChar, error) {
return m.Chars[pg], nil
}
func (m *MockEngine) RenderPage(pg int, dpi float64) ([]byte, error) {
return nil, ErrNoPDFData
}
func (m *MockEngine) RenderPageImage(pg int, dpi float64) (image.Image, error) {
feat(agent): Go ingestion pipeline progress mirroring and DeepDOC parser hardening (#16795) feat(ingestion): mirror Go pipeline progress into the document table; harden resume guards - pipeline: bind the owning document via WithDocumentID; after each TrackProgress event aggregate ingestion_task_log progress and mirror progress/run/progress_msg back into the document table, so GET /api/v1/datasets/{dataset_id}/documents reflects live Go pipeline progress without a bespoke endpoint. - canvas: extend the S3 resume guard to reject legacy no-op nodes (e.g. ExitLoop) so component_total equals the count of progress-reporting components and the aggregate percent can reach 100%. - runtime/canvas: route progress through TrackProgress; add interrupt test coverage (r3_interrupt_test.go). - dao/entity: add IngestionTask.DocumentID column and AggregateProgress support used by the mirror; IngestionTaskLog keeps a Checkpoint column alongside the progress fields. feat(deepdoc): cache DocAnalyzer inference results in Redis (1h TTL) - Redis-backed DocAnalyzerCache decorator over inference.Client; cache key = "ddoc:cache:<method>:" + sha256 of the JPEG-encoded image bytes (deterministic). - TTL = 1h; hits skip the inner HTTP call and return cached JSON; inner errors are not cached. refactor(deepdoc): align figure cropping with Python cropout + bounded page caches - CropSectionByDLA mirrors Python cropout: best-overlap DLA figure/equation region, fallback to section bbox per page, vertical concat on gray background. - sliding-window page-image cache bounds peak memory to the recent window instead of the whole PDF. - rename DLADebug -> DLARegions across parser/chunker/tests. refactor(parser): drop lib_type selector; align NewXxxParser with NewPDFParser - remove config["lib_type"] lookup and the libType param/field/switch from all nine constructors; surface the CGO-required error at ParseWithResult time instead of construction time; drop resolveLibType, its test, and the four lib_type constants. feat(utility): add a reusable workerpool for bounded concurrent execution - internal/utility/workerpool.go (+ tests). refactor: translate Chinese prose comments to English in non-harness Go files. chore: upgrade github.com/cloudwego/eino from v0.9.9 to v0.9.12.
2026-07-10 10:36:10 +08:00
if m.RenderPageImageFunc != nil {
return m.RenderPageImageFunc(pg, dpi)
}
w, h := m.RenderW, m.RenderH
if w <= 0 {
w = 100
}
if h <= 0 {
h = 100
}
return image.NewRGBA(image.Rect(0, 0, w, h)), nil
}
func (m *MockEngine) PageCount() (int, error) {
if m.NumPages <= 0 {
return 1, nil
}
return m.NumPages, nil
}
func (m *MockEngine) RawData() []byte { return nil }
func (m *MockEngine) Close() error { return nil }
func (m *MockEngine) Outlines() ([]pdf.Outline, error) { return nil, nil }