【Editorial Discussion】

An inference pipeline may stage inputs in a reusable pinned host buffer and enqueue an asynchronous host-to-device copy. If a producer overwrites that buffer before the transfer completes, the GPU may receive data from the next request or a mixture of states. The model can then return a wrong answer even though its weights and sampling settings have not changed.

The critical lifetime is the time until the copy has finished reading the source, not the time until the enqueue call returns. Pinned memory enables transfer behavior that makes this distinction important. A host-side queue slot marked “free” immediately after enqueue can therefore create an input-data race.

**Remove the model from the first test**

Build a diagnostic pipeline that alternates distinguishable synthetic input patterns through the staging buffers. After each copy, validate the received device contents against the pattern expected for that request before invoking any model. Use a device-side comparison or a properly ordered readback. Record request identity alongside buffer ownership so an incorrect tensor can be associated with a specific reuse event.

Compare the normal reuse path with a path that waits for copy completion before returning the source buffer to the producer. Keep preprocessing and input shapes unchanged. If mismatches disappear only with protected ownership, investigate the lifetime boundary rather than immediately blaming numerical nondeterminism. A fresh source buffer per in-flight request provides another diagnostic control, though its allocation behavior may alter timing.

**Separate the dependencies**

- Source lifetime: the CPU producer must not mutate or recycle the host region while the transfer may still read it. Record completion in the copy's stream and make buffer ownership depend on that completion. A stream wait orders device work; by itself it does not stop an unrelated CPU writer.

- Destination readiness: a consumer kernel on another stream must wait for the copy to complete before reading the device tensor. Fixing source ownership does not automatically order the consumer.

- Device lifetime: the destination must remain valid until its consumers finish. A pooled allocation reused too early is a separate race with a similar symptom. Inspect which resource is being recycled instead of applying one global synchronization as the final design.

A fully synchronized path is useful for diagnosis but can conceal the original race by changing timing. No mismatch in a short run proves little; vary in-flight concurrency and deliberately delay the relevant work in the test environment. Hardware faults, preprocessing errors, and incorrect request association can also corrupt inputs, so retain independent content checks.

What ownership rule do you use for asynchronous staging buffers? Which experiment has separated a host-source reuse race from a missing copy-to-compute dependency without leaving the entire inference service globally synchronized?