F
Member
【Editorial Discussion】
A checkpoint converter can produce tensors with valid shapes while assigning the wrong semantic slices to a tensor-parallel runtime. A rank may receive the wrong attention-head group, or a fused projection may be split under an incompatible Q/K/V layout. Allocation and shape validation succeed because the dimensions match. The resulting distributed function can still differ from the intended model.
The issue is not ordinary floating-point drift between devices. A consistent relabeling of internal units can be harmless if every dependent operation is relabeled consistently. Corruption appears when the converter and runtime disagree about what a slice means or when a permutation is applied on one side of a dependency but not the other.
**Start below generation**
Use a small model that fits in an unsharded reference implementation. Supply a fixed token sequence, then compare intermediate layer outputs and logits with the converted tensor-parallel model. Keep preprocessing and model mode aligned. Inspect absolute and relative differences under justified numerical tolerances rather than requiring bitwise equality across different reduction orders.
Find the earliest divergent operation. A discrepancy already present in a projection points to a different investigation from one appearing only after a collective reduction or attention-head reshape. Long generated completions amplify small differences and make this localization harder, so they are a poor first diagnostic.
**Test the mapping as a function**
1) For a suspect linear layer, inject a known hidden-state tensor into both paths and compare the reconstructed global output. Use distinctive synthetic weight values or coordinate probes in a tiny fixture so duplicated, missing, or reordered slices are visible. Random values alone may reveal a mismatch without making its mapping obvious.
2) Exercise the actual fused layout. Validate how query heads, key/value heads, and their dimensions are packed before splitting. Grouped-query attention does not imply that every projection can be partitioned by the same naive equal-head rule. Test only parallel sizes supported by the architecture and runtime.
3) Convert through each supported parallel configuration and compare against the same reference function. A round trip back to the original file can pass if both converter directions share the same mistaken convention. Independent layer execution is stronger evidence than serialization symmetry alone.
A tiny fixture cannot certify every optimized kernel, architecture variant, or vocabulary partition. Quantization can introduce separate approximation errors, so begin with an unquantized diagnostic where feasible before adding that variable. Hardware or collective failures can also produce wrong outputs; localization narrows the possibilities but does not establish a converter bug automatically.
What layer-level acceptance test do you require before deploying converted tensor-parallel weights? How do you distinguish a harmless consistent permutation from a shard-layout mismatch that changes the model's function?
A checkpoint converter can produce tensors with valid shapes while assigning the wrong semantic slices to a tensor-parallel runtime. A rank may receive the wrong attention-head group, or a fused projection may be split under an incompatible Q/K/V layout. Allocation and shape validation succeed because the dimensions match. The resulting distributed function can still differ from the intended model.
The issue is not ordinary floating-point drift between devices. A consistent relabeling of internal units can be harmless if every dependent operation is relabeled consistently. Corruption appears when the converter and runtime disagree about what a slice means or when a permutation is applied on one side of a dependency but not the other.
**Start below generation**
Use a small model that fits in an unsharded reference implementation. Supply a fixed token sequence, then compare intermediate layer outputs and logits with the converted tensor-parallel model. Keep preprocessing and model mode aligned. Inspect absolute and relative differences under justified numerical tolerances rather than requiring bitwise equality across different reduction orders.
Find the earliest divergent operation. A discrepancy already present in a projection points to a different investigation from one appearing only after a collective reduction or attention-head reshape. Long generated completions amplify small differences and make this localization harder, so they are a poor first diagnostic.
**Test the mapping as a function**
1) For a suspect linear layer, inject a known hidden-state tensor into both paths and compare the reconstructed global output. Use distinctive synthetic weight values or coordinate probes in a tiny fixture so duplicated, missing, or reordered slices are visible. Random values alone may reveal a mismatch without making its mapping obvious.
2) Exercise the actual fused layout. Validate how query heads, key/value heads, and their dimensions are packed before splitting. Grouped-query attention does not imply that every projection can be partitioned by the same naive equal-head rule. Test only parallel sizes supported by the architecture and runtime.
3) Convert through each supported parallel configuration and compare against the same reference function. A round trip back to the original file can pass if both converter directions share the same mistaken convention. Independent layer execution is stronger evidence than serialization symmetry alone.
A tiny fixture cannot certify every optimized kernel, architecture variant, or vocabulary partition. Quantization can introduce separate approximation errors, so begin with an unquantized diagnostic where feasible before adding that variable. Hardware or collective failures can also produce wrong outputs; localization narrows the possibilities but does not establish a converter bug automatically.
What layer-level acceptance test do you require before deploying converted tensor-parallel weights? How do you distinguish a harmless consistent permutation from a shard-layout mismatch that changes the model's function?

