Fix tag_feas code injection in retrieval ranking (#13923)

## Summary
- remove eval-based parsing from retrieval rank feature scoring
- validate `tag_feas` at write time in chunk APIs and SDK routes
- add regression tests for safe parsing and malicious payload rejection

## Details
`tag_feas` is intended to be structured rank-feature data, but the
retrieval ranking path was evaluating stored values as Python
expressions. This change treats `tag_feas` strictly as data.

### What changed
- replace `eval()` in `rag/nlp/search.py` with safe parsing via
`json.loads()` and optional `ast.literal_eval()` compatibility for
legacy Python-dict strings
- strictly filter parsed values down to `dict[str, finite number]`
- reject invalid `tag_feas` payloads at write time in web chunk routes
and SDK document chunk routes
- add focused regression tests to prove executable strings are ignored
and invalid payloads are rejected

## Validation
- `python -m pytest test/unit_test/common/test_tag_feature_utils.py
test/unit_test/rag/test_rank_feature_scores.py -q`

---------

Co-authored-by: unknown <zhenglinkai@CCN.Local>
Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
This commit is contained in:
Ea001
2026-04-15 16:31:11 +08:00
committed by GitHub
parent 1f33ca1099
commit 38cefd88e2
8 changed files with 259 additions and 8 deletions

View File

@@ -0,0 +1,32 @@
import pytest
from common.tag_feature_utils import parse_tag_features, validate_tag_features
def test_validate_tag_features_accepts_numeric_dict():
assert validate_tag_features({"apple": 1, "banana": 2.5}) == {
"apple": 1.0,
"banana": 2.5,
}
def test_validate_tag_features_rejects_string_payload():
with pytest.raises(ValueError, match="object mapping string tags"):
validate_tag_features('{"apple": 1.0}')
def test_validate_tag_features_rejects_non_finite_or_non_numeric_values():
with pytest.raises(ValueError, match="finite numbers"):
validate_tag_features({"apple": float("inf")})
with pytest.raises(ValueError, match="finite numbers"):
validate_tag_features({"apple": "1.0"})
def test_parse_tag_features_supports_legacy_python_literal_strings():
assert parse_tag_features("{'apple': 2.0}", allow_python_literal=True) == {"apple": 2.0}
def test_parse_tag_features_ignores_executable_strings():
payload = '{"apple": (__import__("time").sleep(1) or 1.0)}'
assert parse_tag_features(payload, allow_python_literal=True) == {}