【Editorial Discussion】

An inference service can pass its health endpoint while a supposedly accelerated operation runs on a CPU fallback. An optional extension may fail to load because a shared library is unavailable, or the dispatcher may reject a particular dtype or shape. A broad exception handler substitutes a working implementation, so correctness tests pass and deployment appears healthy until latency or host load changes.

The mechanism is capability loss hidden by successful fallback. It is different from a GPU dropping off the bus or a slow kernel: the intended implementation may never execute. Device availability and successful model loading do not establish which backend handled each operation.

**Create a fallback you can recognize**

In a test environment, run a representative operation with the extension available, then deliberately make the optional extension unavailable through a supported configuration or isolated dependency setup. Compare dispatch records and outputs under the same input. The negative condition should demonstrate that monitoring can distinguish fallback from acceleration, not merely that both paths return a tensor.

Observe the selected implementation or an explicit backend counter at the operation boundary. Timing and GPU utilization are supporting signals, not definitive identity checks: asynchronous execution can confuse timing, and other model operations may keep the GPU busy while one expensive stage falls back.

**Separate loading from dispatch coverage**

- At startup, report whether the extension loaded and surface the actual load failure rather than swallowing every exception as “optional backend unavailable.” A service whose contract requires acceleration should not advertise full readiness simply because a slower implementation exists. A degraded mode may be legitimate if that is an explicit operational choice.

- Exercise the shapes, dtypes, and layouts the service supports. Import success proves neither registration nor coverage. Include boundary inputs that select a different implementation and verify that the chosen path matches the documented support matrix. Keep numerical correctness checks independent of backend-identity assertions.

- Test a required-acceleration mode that rejects unsupported work clearly instead of silently degrading. Also test an allowed-fallback mode that records the reason and affected operation. This prevents an intentionally portable library behavior from being confused with the service's stricter deployment promise.

A startup smoke test can miss a later request-specific fallback, so retain low-overhead runtime visibility at an appropriate aggregation level. Avoid logging full user tensors merely to identify a backend.

Fallback is not inherently a defect. Some small workloads may be cheaper on the host, and an accelerated path can be slower for an unfavorable shape. An implementation counter establishes what ran, not whether it was the best choice. Benchmark that decision separately with synchronization appropriate to the work being measured.

What evidence do you require that an optional accelerated extension actually serves production requests? How do you distinguish an intentional fallback policy from a dependency failure hidden behind a healthy endpoint?