Voice Memory: Inference-Only Scheme for Agentic Speech Recognition
A novel inference-only approach named Voice Memory enhances speech recognition for agents by employing a static corrector that examines a domain-specific memory file. It determines for each utterance whether to proceed with the hypothesis or refrain from acting. An asynchronous score-gated optimizer updates the memory through limited modifications, permitting alterations only when they enhance a held-out score. This division, known as the listener-thinker architecture, maintains unchanged weights, ensuring that the acquired skill remains auditable and transferable. The technique minimizes over-correction: while unconstrained generative error correction (GER) disrupts correct tokens in up to 64% of financial news edits, Voice Memory curtails this figure to 35%. In ten HyPoradise domains with an open corrector, Voice Memory decreases weighted word error.
Key facts
- Voice Memory is an inference-only scheme for agentic speech recognition.
- A frozen corrector reads a single per-domain memory file at stream time.
- The corrector decides per utterance whether to act on the hypothesis or abstain and keep the 1-best.
- An asynchronous score-gated optimizer revises the memory through bounded edits.
- Edits are accepted only when they strictly improve a held-out score.
- The architecture is called listener-thinker, with two roles coupled only through memory.
- No weights change, so the learned skill stays auditable and portable.
- Unconstrained GER breaks correct tokens on up to 64% of its edits on financial news.
- Voice Memory reduces this over-correction rate to 35%.
- Tested across ten HyPoradise domains with an open corrector.
Entities
—