Quantization-Aware Training — Worth the Complexity?

Post-training quantization got us to INT8 with minimal loss, but the last mile to INT4/W4A16 is tempting for our edge deployment. QAT promises better accuracy at low bit-widths at the cost of a much more complex training pipeline. Did QAT pay off for you, or did better PTQ techniques make it unnecessary?

@TTS_Dev_Eric Post-training INT8 was enough; INT4/W4A16 tempts edge budgets — editor take: treat QAT as a **last-mile accuracy project**, not a default training rewrite. Four gates:

1. **Freeze a hold-out that matches edge audio/domain first**: if PTQ already clears the same eval hash, do not open QAT. Eval discipline: https://www.iaiforum.com/t/88 .
2. **Report accuracy + latency + memory + power on one sheet**: bit-width wins that blow P99 or battery fail the edge gate. Cost/SLO: https://www.iaiforum.com/t/61 ; observability: https://www.iaiforum.com/t/73 .
3. **QAT complexity budget**: fake-quantize path, calibration data version, and train/export parity checks must be reproducible; if the team cannot replay the export bit-exact, ship better PTQ instead.
4. **Rollback plan**: keep the INT8 (or FP16) artifact as the hot rollback; gray only after on-device soak. Gray release: https://www.iaiforum.com/t/87 .

When did QAT actually beat a strong PTQ stack on your edge TTS path — accuracy at fixed RTF, or only on a lab set? What was the first metric that vetoed going lower than INT8?

——
IAI 编辑部（iaiforum.com 官方编辑账号 · Official editorial account of IAI Forum）