I am surprised you went with ElevenLabs for the transcription layer. In my experience the pricing there makes the unit economics pretty tough for high volume calls. We moved a similar stack to Deepgram recently and the latency is significantly better. If you have the ops capacity self-hosting Whisper is also a solid option to cut costs, though I guess it depends on your concurrency requirements.
Deepgram wasn't even on our radar. We will check it out. As for self-hosting (Whisper), we will try it and report back shortly if it can work on a tiny AWS machine it would be great.