F
Member
【Editorial Discussion】
A voice pipeline that transcribes first and then lets an LLM plan or act will compound errors: a wrong name, a missed negation, or a cut-off sentence becomes a confident tool call. The product question is whether the user (or a downstream check) sees and can correct the transcript before side effects, or whether you only try to make the ASR better.
1) Measure the pipeline as a whole, with errors tagged at ASR, at interpretation, and at the action. A better model in the middle will not fix a systematic miss on domain terms if you never look at the transcript. Domain lexicons and audio quality belong in the same review as the LLM prompt.
2) High-impact actions should have a confirmation that includes what was heard, not only a paraphrase of the plan. A paraphrase can drop the same negation the ASR dropped. For low-impact actions, a visible transcript after the fact may be enough; say which is which.
3) End-of-utterance and partial transcripts are a separate source of premature actions. If the LLM is invoked on a partial, you need a rule for revision when the rest of the sentence arrives.
4) Test with expected ASR mistakes, not only with clean studio audio: names, punctuation that changes meaning, and noisy rooms. The LLM’s job includes handling a dirty transcript, or refusing, not only answering a perfect one.
Open question: do you require a visible or confirmable transcript before tool calls, and for which actions? Where have compounded errors actually hurt you — ASR, turn-taking, or the LLM filling a gap? Practical patterns welcome.
A voice pipeline that transcribes first and then lets an LLM plan or act will compound errors: a wrong name, a missed negation, or a cut-off sentence becomes a confident tool call. The product question is whether the user (or a downstream check) sees and can correct the transcript before side effects, or whether you only try to make the ASR better.
1) Measure the pipeline as a whole, with errors tagged at ASR, at interpretation, and at the action. A better model in the middle will not fix a systematic miss on domain terms if you never look at the transcript. Domain lexicons and audio quality belong in the same review as the LLM prompt.
2) High-impact actions should have a confirmation that includes what was heard, not only a paraphrase of the plan. A paraphrase can drop the same negation the ASR dropped. For low-impact actions, a visible transcript after the fact may be enough; say which is which.
3) End-of-utterance and partial transcripts are a separate source of premature actions. If the LLM is invoked on a partial, you need a rule for revision when the rest of the sentence arrives.
4) Test with expected ASR mistakes, not only with clean studio audio: names, punctuation that changes meaning, and noisy rooms. The LLM’s job includes handling a dirty transcript, or refusing, not only answering a perfect one.
Open question: do you require a visible or confirmable transcript before tool calls, and for which actions? Where have compounded errors actually hurt you — ASR, turn-taking, or the LLM filling a gap? Practical patterns welcome.

