mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-08-19 14:55:40 +08:00
45 lines
3.2 KiB
Markdown
45 lines
3.2 KiB
Markdown
You are an information retrieval evaluation expert. Determine whether the retrieved content is sufficient to answer the user's question(s), following the "Sufficient Context" criterion:
|
|
|
|
The CONTEXT is sufficient to answer the question if and only if a PLAUSIBLE answer can be inferred from it — that is, the retrieved content either directly contains or logically entails an answer to the question. The answer does NOT need to be proven correct; it only needs to be a reasonable, supportable answer. If the context cannot be used to infer any plausible answer, it is INSUFFICIENT.
|
|
|
|
Each retrieved chunk is labeled with an integer ID on a line like `ID: 3`.
|
|
|
|
User question(s):
|
|
{{ question }}
|
|
|
|
Retrieved content:
|
|
{{ retrieved_docs }}
|
|
|
|
Reasoning procedure (do this step-by-step before answering):
|
|
1. Identify the REQUIRED ENTITIES or key facts that a plausible answer to the question must involve.
|
|
2. For each required entity, check whether the retrieved content provides evidence about it. Record this in "coverage".
|
|
3. Check for multi-hop inference: if answering requires combining facts not present in the context, or inferring a connection the context does not state, that is NOT inferable from the context.
|
|
4. Check whether the context is ambiguous: if it could support multiple mutually exclusive plausible answers and nothing in the context lets you distinguish them, mark it insufficient.
|
|
5. Note any internally conflicting figures/statements in the context ("contradictions").
|
|
6. Decide whether a plausible answer can be inferred; give your confidence in that decision.
|
|
|
|
Output format (JSON):
|
|
```json
|
|
{
|
|
"Sufficient Context": true/false,
|
|
"is_sufficient": true/false,
|
|
"required_entities": ["Entity 1", "Entity 2"],
|
|
"coverage": {"Entity 1": true, "Entity 2": false},
|
|
"missing_information": ["Missing information 1", "Missing information 2"],
|
|
"contradictions": ["conflicting figures/statements if any"],
|
|
"confidence": 0.0,
|
|
"reasoning": "Step-by-step reasoning for the judgment",
|
|
"useful_chunk_ids": [0, 3, 7]
|
|
}
|
|
```
|
|
|
|
Requirements:
|
|
1. `Sufficient Context` / `is_sufficient` must be true if and only if a plausible answer can be inferred from the context (per the definition above). A missing detail that a reasonable answer would still require makes it false.
|
|
2. If not sufficient, list the concrete `missing_information`.
|
|
3. `coverage` must mark, for each required entity, whether the context provides evidence about it. Missing required entities belong in `missing_information`.
|
|
4. `confidence` (0-1): how confident you are in your sufficiency decision. 0.9-1.0 if the context clearly supports or clearly fails a plausible answer; 0.5-0.7 if evidence is partial or ambiguous; below 0.5 if you cannot tell.
|
|
5. `contradictions`: list any internally conflicting figures/statements that would make a single answer ambiguous. Empty array when none.
|
|
6. `useful_chunk_ids` must contain ONLY the integer IDs (taken from the `ID:` labels above) of chunks that provide information useful for answering the question(s). Exclude irrelevant or redundant chunks. Use an empty array when none are useful.
|
|
7. The `missing_information` should only be filled when insufficient, otherwise an empty array.
|
|
8. The `reasoning` should be concise and clear.
|