F
Member
【Editorial Discussion】
Rolling back a prompt or a model version is not always a rollback of the system. Embeddings may have been reindexed, a tool schema may have gained a required field, or a cache key may still point at the new template. The old prompt then reads the new index, or the old model calls a tool it no longer matches. A rollback unit has to name every artifact that must move together.
1) Package the prompt, model identifier, tool schema, embedding model and index pointer, and output validator as one versioned bundle with a single rollback command. If any piece can be pinned independently, the runbook should say which combinations are supported.
2) Before a forward change that rebuilds an index or tightens a schema, record a restore path: a snapshot of the old index, a dual-write window, or a schema that stays backward compatible until the prompt is stable. A rollback that cannot restore retrieval is only a partial undo.
3) In staging, practice the rollback, then run a fixture that would fail if the old prompt hit the new schema or the new embeddings. Reverting the prompt file is not evidence that the product moved back.
4) Cache keys, and any flag that selects the bundle, should include the bundle version. A cache filled by the new prompt and served to the old one will look like a mysterious regression after the rollback.
Open question: what do you include in a compatible rollback unit for an LLM feature, and how do you test that the unit actually restores retrieval and tools? Have you rolled back a prompt and still been on the new index or the new schema? Please share the packaging rule you use.
Rolling back a prompt or a model version is not always a rollback of the system. Embeddings may have been reindexed, a tool schema may have gained a required field, or a cache key may still point at the new template. The old prompt then reads the new index, or the old model calls a tool it no longer matches. A rollback unit has to name every artifact that must move together.
1) Package the prompt, model identifier, tool schema, embedding model and index pointer, and output validator as one versioned bundle with a single rollback command. If any piece can be pinned independently, the runbook should say which combinations are supported.
2) Before a forward change that rebuilds an index or tightens a schema, record a restore path: a snapshot of the old index, a dual-write window, or a schema that stays backward compatible until the prompt is stable. A rollback that cannot restore retrieval is only a partial undo.
3) In staging, practice the rollback, then run a fixture that would fail if the old prompt hit the new schema or the new embeddings. Reverting the prompt file is not evidence that the product moved back.
4) Cache keys, and any flag that selects the bundle, should include the bundle version. A cache filled by the new prompt and served to the old one will look like a mysterious regression after the rollback.
Open question: what do you include in a compatible rollback unit for an LLM feature, and how do you test that the unit actually restores retrieval and tools? Have you rolled back a prompt and still been on the new index or the new schema? Please share the packaging rule you use.

