【Editorial Discussion】

A preference dataset labels complete answers, but a training pipeline may optimize over shortened versions. If truncation removes the correction that made the chosen answer preferable, the retained pair no longer has the same evidential basis. In the extreme case, both answers become identical token sequences while retaining opposite labels. The dataset can pass ordinary format validation and still stop expressing the preference it was collected to teach.

This is not an attention-boundary problem from packing independent examples. It can occur with one isolated preference pair. Nor does it imply that every shortened pair is invalid: some preferences are visible early, while others depend entirely on a later sentence, a final result, or a missing qualification.

**Audit the actual training view**

Decode the prompt and both completions after the same template, tokenization, truncation, and masking steps used by training. Show reviewers only that retained view first, without revealing which answer was chosen. Then compare their judgment with the full pair and its original label. Separate “still preferred,” “now indistinguishable,” “preference reversed,” and “not enough context.” The purpose is not to manufacture a new automatic judge score but to locate transformations that destroy label support.

Check especially whether both completions retain the same prompt. A length policy that truncates prompt tokens differently depending on completion length can introduce a second confound: the model is no longer comparing answers under identical conditions.

**A diagnostic intervention before another training run**

Construct synthetic pairs with a deliberately localized distinguishing span. In one group the correction is near the start; in another it is near the end. Move the cutoff across that span while keeping the full-pair preference unchanged. Record whether the distinguishing tokens actually contribute to the loss, rather than merely appearing in a debug string. Inspect equality of retained completion tokens and the supervised-token count for each side.

For an otherwise identical retained pair under a deterministic shared forward computation, the pair itself supplies no answer-content contrast. That is a stronger warning than a disappointing aggregate training loss. For nonidentical pairs, however, the sign of a training margin cannot certify that the original human preference survived preprocessing.

Possible responses include recollecting shorter complete answers, excluding unsupported pairs, or explicitly relabeling the retained versions. Each changes the training distribution. Simply increasing the limit may shift the cutoff without fixing prompt loss or label masking. Compare removed and retained data by task type so a cleanup does not silently discard every task whose justification appears late.

Human re-review remains costly and can disagree, and localized synthetic examples do not establish the prevalence of the defect. What retained-view audit would you require before trusting preference labels? How do you handle a pair whose decisive evidence is outside the training window without converting brevity into an unintended training objective?