Voice agents that speak Uzbek and Russian: how the stack handles the mix

Listen to twenty real customer calls in Tashkent and you will hear something no clean demo prepares you for: language as it is actually spoken. A sentence starts in Uzbek, borrows a Russian clause for the technical part, and ends with a price in whichever language the number came to mind. Any voice agent that assumes one language per call is planning to fail politely.

The three problems in the mix

Recognition: the agent must transcribe speech that switches language between phrases without dropping the switched fragment. Modern multilingual speech models handle Russian excellently and Uzbek increasingly well, but code-switching is where cheap stacks crack: the recognizer locks onto one language and mangles the other.

Understanding: even when transcription is right, intent must survive the blend. 'Zavtra priyom bormi?' is a routine sentence for a Tashkent clinic and a nightmare for a monolingual NLU pipeline. The language model behind the agent has to treat mixed input as normal, because it is.

Synthesis: the response should come back in the customer's dominant language with names, addresses, and numbers pronounced naturally. Uzbek TTS has improved fast; the remaining craft is in prosody around numbers and proper names, which is exactly what booking confirmations are full of.

What a production setup looks like

We run recognition in multilingual mode rather than forcing a language, and we keep the model's language guess per utterance, not per call. The dialog manager tracks which language the customer used most and answers in it, while staying ready to flip if the customer flips.

Numbers get special treatment in both directions. Prices, dates, and phone numbers are confirmed back explicitly ('to'qqiz yuz ming so'm, to'g'rimi?') because a misheard digit in a booking is worse than a slightly robotic confirmation.

Escalation stays language-aware: if the agent hands the call to a human, it hands over the language too, so the customer is not greeted in the wrong one by the very person meant to rescue the call.

Latency, the silent killer

On a call, a delay that would be fine in chat feels like a dropped line. The pipeline (recognize, think, synthesize) must fit inside roughly a second to feel natural. That budget shapes everything: retrieval must be pre-warmed, prompts lean, and synthesis streamed so the first syllable starts before the sentence is fully formed.

This is also why we test agents on real telephony, not on laptop microphones. Codec compression and network jitter eat into both recognition accuracy and the latency budget, and a demo that ignores them is a demo of a different product.

What to check before you sign with anyone

Ask for test calls on your scripts, on a real phone line, in both languages and in the mix. Ask how the agent confirms numbers. Ask what happens on the third misunderstanding (the right answer involves a human). And ask to hear the agent say your actual street address in Uzbek, because that one sentence exposes more about the stack than any slide deck.