Commit Graph

11 Commits

Author SHA1 Message Date
Jack
965590ccbe Refactor: dataset/document/file service (#17071)
### Summary

Refactor dataset.go document.do file.go file2document.go in
internal/service.
2026-07-20 09:48:24 +08:00
Jack
10c00a9614 Feat: add built in DSL file API (#17003)
### Summary

1. list and get by id API for builtin DSL
2. add DSL default component param values validation
3. remove all hard code keys for parser config
2026-07-18 11:00:08 +08:00
Jack
047c714ffb Feat: ingestion cleanup (#16953)
### Summary

1. Remove dead code (replaced by builtin ingestion pipeline)
2. Refactor (move document parsing progress from http api into ingestion
executor)
2026-07-16 11:05:26 +08:00
Jack
e543ff02c4 Feat: nats message processing refactor and support built in DSL (#16873) 2026-07-15 14:53:16 +08:00
Jack
cfacaccad7 Refactor: message processing (#16852)
### Summary

1. refactor message processing
2. delete un-used componentIndexMap
3. unfold (delete) internal/ingestion/task/task_handler.go
2026-07-13 16:32:34 +08:00
Jack
3c0f8fc192 Refactor: refactor dataflow_service.go and guard nats message re-delivery (#16826)
### Summary

1. refactor dataflow_service.go 
2. guard nats message re-delivery
3. support document parse cancelling & re-run
2026-07-13 11:08:04 +08:00
Jack
5e60fcec9f Refactor: Refine ingestion task state transitions (#16814)
### Summary

Refine ingestion task state transitions
2026-07-10 22:47:51 +08:00
Jack
7db39822db Feature: user select pipeline support (#16788)
### Summary

Feature: user select pipeline support
2026-07-10 14:30:28 +08:00
Jack
8bd62573eb Fix: delete tasklet (#16778)
### Summary

1. Fix: delete tasklet
2. Remove panic in text processing
2026-07-09 17:31:28 +08:00
Jack
8ac80284c4 Feat: add embedding support (#16769)
### Summary

Feat: add embedding support
2026-07-09 16:24:39 +08:00
Jack
0dd0ac06f8 Feature: task executor migration to go (#16549)
### Summary

Feature: Integrate parser
2026-07-08 19:08:11 +08:00