zlei6/ragflow
1
0
Fork 0
mirror of https://github.com/infiniflow/ragflow.git synced 2026-07-31 04:59:24 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix tag CSV parser ignoring the documented TAB delimiter (#17496)

Browse Source
This commit is contained in:
Yash Raj Pandey
2026-07-29 05:01:07 -04:00
committed by GitHub
parent e5a7bee9db
commit c3aa8053c0
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
rag/app/tag.py
View File

@@ -92,11 +92,12 @@ def chunk(filename, binary=None, lang="Chinese", callback=None, **kwargs):
callback(0.1, "Start to parse.")
txt = get_text(filename, binary)
lines = txt.split("\n")
delimiter = "\t" if any("\t" in line for line in lines) else ","
fails = []
content = ""
res = []
reader = csv.reader((line + "\n" for line in lines))
reader = csv.reader((line + "\n" for line in lines), delimiter=delimiter)
prev_line_num = 0
# line_num tracks the physical span when quoted fields cross lines.