F
Member
【Editorial Discussion】
A fluent RAG answer with a confident citation is worse than a refusal if the cited chunk does not actually support the claim. Users will trust the footnote. Prompting the model to “only use the context” is not a test. You need a check that can fail a build or a canary when the support is missing.
1) Define support at the claim level. Split the answer into checkable statements and ask whether each one is entailed, contradicted, or not present in the retrieved (or cited) text. A single “faithfulness score” on the whole paragraph will hide a wrong number next to a correct sentence.
2) The checker can be programmatic for numbers, dates, and identifiers, and a judge or a human for the rest — but the judge must see the chunk, not only the answer. If the checker can pass without the citation, it is not a citation check.
3) Run this on a frozen set of questions, including items where the corpus is silent and the model tends to fill in. Silence should produce a refusal or a clearly marked lack of support, not a invented detail with a nearby footnote.
4) Put the failing claim, the cited chunk, and the question into the CI artifact. A red score without those three is hard to fix.
Open question: do you already fail releases on unsupported citations, and is the check claim-level or document-level? What error is still the hardest to catch automatically? Examples of checks, not of customer data, are welcome.
A fluent RAG answer with a confident citation is worse than a refusal if the cited chunk does not actually support the claim. Users will trust the footnote. Prompting the model to “only use the context” is not a test. You need a check that can fail a build or a canary when the support is missing.
1) Define support at the claim level. Split the answer into checkable statements and ask whether each one is entailed, contradicted, or not present in the retrieved (or cited) text. A single “faithfulness score” on the whole paragraph will hide a wrong number next to a correct sentence.
2) The checker can be programmatic for numbers, dates, and identifiers, and a judge or a human for the rest — but the judge must see the chunk, not only the answer. If the checker can pass without the citation, it is not a citation check.
3) Run this on a frozen set of questions, including items where the corpus is silent and the model tends to fill in. Silence should produce a refusal or a clearly marked lack of support, not a invented detail with a nearby footnote.
4) Put the failing claim, the cited chunk, and the question into the CI artifact. A red score without those three is hard to fix.
Open question: do you already fail releases on unsupported citations, and is the check claim-level or document-level? What error is still the hardest to catch automatically? Examples of checks, not of customer data, are welcome.

