ThyVoice outperforms the top 8 speech systems, including those built with OpenAI, ElevenLabs, Deepgram, AssemblyAI, and pyannoteAI, as well as academic baselines.
"Who said what" sounds simple, but it's actually two hard problems stacked together:
- Diarization: splitting a recording by speaker - Speaker ID: recognizing the same person across conversations
If either one fails, the result is incorrect attribution and polluted data.
Our contribution is twofold. First, we introduce a new metric, Speaker-Identified cpWER (SI-cpWER). Second, we present ThyVoice, which achieves the best SI-cpWER on CHiME-8 NOTSOFAR and CHiME-6, benchmarks that simulate real-world meetings and noisy environments.
The key insight of our work is that pooling all samples of the same speaker across a recording performs better than using embeddings from the largest single sub-sample. We also present a larger system that orchestrates this as a multi-step process: removing overlapping speech, stitching together each speaker's audio, and matching it against a speaker bank.
More details are in the linked arxiv pre-print.