The layered default already on the thread — logs for prompt hash, model version, latency, tokens and error code; traces that split retrieval, rerank and generation; a trace id from the edge; and redaction plus a sampling policy written into the release — is the right gate. I would not treat “we have logs” as observability, and I would not pick a vendor until that id actually lands in every span. The two questions in the staff note are the parts I would decide first.
On sampling, I would not trigger only on error rate. A small continuous baseline, plus extra capture on errors and on the latency tail, is what keeps a healthy shape visible; error-only sampling hides the slow-but-successful request, and a manual switch is usually off during the incident you later want to read. On redaction, do it at write, not only at query or display. Display-only masking still leaves raw prompts on disk, in backups and in the next support export. Retention and the sampling rule belong on the same change ticket as the tool itself.
What logs and traces still miss, even with a consistent id, is a quality signal and a clock you can trust. A span that is fast and green can still have retrieved the wrong chunk or accepted a tool result that failed a schema check; those belong on the waterfall next to time and token counts, together with the prompt, model and tool versions. If the retriever host and the inference host do not share a clock, the waterfall will blame the wrong span. I would also propagate the same trace id into the machine journal, because the failure is sometimes the process, the container or the accelerator, not the prompt.
Cost and evidence need a place to live. Default storage can be hashes, sizes and timings, with full payloads only on the sampled traces and with a short TTL. On a self-hosted ARM inference box I would export those sampled traces off the box, or onto a second partition: the same disk that is dying will otherwise take the incident with it. A change to sampling, redaction or the tool chain is a configuration change and should go out with a canary, as the staff note already requires.
If I cannot follow one id from the gateway to retrieval, generation and the host journal, and I cannot say what was sampled and what was redacted at write, I would not call the system observable yet.