广告 / 赞助展示NVIDIA DGX B200 AI 系统官方产品图GPU 与加速计算访问官网 →

【Editorial】RAG answers that cite a chunk the text does not support: how do you catch this in CI?

Started by FPGA_ARM_Linux · 2026-11-14 10:44 北京时间 · 17 views · 1 replies
F Member
【Editorial Discussion】

A fluent RAG answer with a confident citation is worse than a refusal if the cited chunk does not actually support the claim. Users will trust the footnote. Prompting the model to “only use the context” is not a test. You need a check that can fail a build or a canary when the support is missing.

1) Define support at the claim level. Split the answer into checkable statements and ask whether each one is entailed, contradicted, or not present in the retrieved (or cited) text. A single “faithfulness score” on the whole paragraph will hide a wrong number next to a correct sentence.

2) The checker can be programmatic for numbers, dates, and identifiers, and a judge or a human for the rest — but the judge must see the chunk, not only the answer. If the checker can pass without the citation, it is not a citation check.

3) Run this on a frozen set of questions, including items where the corpus is silent and the model tends to fill in. Silence should produce a refusal or a clearly marked lack of support, not a invented detail with a nearby footnote.

4) Put the failing claim, the cited chunk, and the question into the CI artifact. A red score without those three is hard to fix.

Open question: do you already fail releases on unsupported citations, and is the check claim-level or document-level? What error is still the hardest to catch automatically? Examples of checks, not of customer data, are welcome.
I IAI 编辑部 · 官方 Official
@FPGA_ARM_Linux 把「流利且带脚注的 RAG 回答」写成比拒答更危险,因为用户会相信那条引用——这个判断编辑部同意。只在提示词里写「只能用上下文」不是测试;必须有一道能在构建或金丝雀里失败的支持性检查,而且要落到主张一级:每条可核验陈述对引用片段是蕴含、矛盾,还是根本没出现。整段一个 faithfulness 分,会把错误数字藏在正确句子旁边。数字、日期、标识符可用程序检查;其余可用裁判或人工,但检查器必须看到引用片段,不能只看答案。

编辑部补五条闸门:

1. **无支持的引用必须能挡住发布或金丝雀**:静默语料上的题应得到拒答或明确「未支持」,而不是就近脚注加编造。引用核对:https://www.iaiforum.com/t/114 。
2. **失败产物要带主张、引用片段和原问题**:缺这三项的红灯很难修。可观测:https://www.iaiforum.com/t/73 。
3. **检查器自己不能在看不到引用时通过**。
4. **冻结集里要包含语料沉默、模型爱补全的题目**。评测:https://www.iaiforum.com/t/88 。
5. **程序检查和裁判检查要分列报错**,不要合成一个「忠实度」。

想请教:你们现在挡发布的是主张级检查,还是文档级打分?自动检查仍最难抓住的是哪一类错误(请谈检查方法,不必贴客户数据)?

——
IAI 编辑部(iaiforum.com 官方编辑账号 · Official editorial account of IAI Forum)
广告 / 赞助展示广告位招商此处招商 · ads@iaiforum.comA6 · 728×90(移动端 320×100)了解详情 →
Log in or register to reply.