Voice agent vendors are competing on language count right now. Deepgram Flux Multilingual supports 10 languages. GPT-Realtime-Translate supports 70+. Neither number tells you what accuracy you’ll see in production at the dialect and accent level — which is where deployments actually succeed or fail.
The production challenges of voice AI at scale don’t start with vendor selection. They start with knowing what to measure. So in this article we’re going to frame the comparison around that: what the accuracy numbers actually tell you, where they go quiet, and what multilingual deployment requires beyond picking a model.
Let’s get into it.
Why Is Language Coverage the Wrong Way to Evaluate Multilingual Voice AI?
The 70-versus-10 comparison is a marketing number. It tells you what languages each model claims to support. It does not tell you how accurately those languages are recognised across the actual range of speakers your callers represent.
A model that “supports Spanish” might perform adequately on Castilian Spanish while producing materially worse results on Mexican, Colombian, or Argentine Spanish. That’s the dialect variation problem — and it’s distinct from language-level coverage.
Benchmark Word Error Rate scores make this worse. They’re run on curated test sets under vendor-controlled conditions. They don’t reflect accent distribution in your actual caller population, audio quality from telephony compression, or how often your callers switch between languages mid-sentence. A model that transcribes English at 3% WER but produces 45% WER on Spanish utterances will show a blended score of 8–12%. That looks fine until you check your Spanish user churn rate.
The meaningful evaluation dimension is dialect-level and accent-level accuracy: how does the model behave on the specific regional speech patterns your callers actually use?
What Do Deepgram Flux Multilingual and GPT-Realtime-Translate Actually Offer?
These are architecturally different products solving different problems. It’s worth being clear on that before comparing them.
Deepgram Flux Multilingual went generally available on 29 April 2026. It’s a single streaming conversational ASR model covering 10 languages: English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian, and Dutch. Code-switching, end-of-turn detection, and interruption handling are all native. For existing Deepgram users, migration from the English-only model is a one-line API change.
GPT-Realtime-Translate launched on 8 May 2026. It’s a live speech-to-speech translation model: 70+ input languages, 13 output languages, $0.034/minute. It is not a conversational AI. It converts speech in one language to speech in another. You wouldn’t run a Q&A conversation through it — you’d use it to let a Spanish-speaking customer talk to an English-speaking support agent.
GPT-Realtime-Translate and GPT-Realtime-2 are separate models. GPT-Realtime-Translate’s 70-language coverage is a translation layer; GPT-Realtime-2 is the conversational reasoning engine. For a voice agent that listens, understands, and responds in the caller’s language, you need a conversational ASR model — that’s Flux Multilingual.
BolnaAI, an OpenAI launch partner, reported 12.5% lower Word Error Rate on Hindi, Tamil, and Telugu using GPT-Realtime-Translate. It’s vendor-reported data — not an independent audit — but it’s directional evidence that language-level accuracy differences are real. All Deepgram Flux Multilingual accuracy claims are also vendor-reported. No independent third-party benchmarks for either model exist as of this writing.
Where Do Benchmark Scores Break Down in Multilingual Production?
Both products come with vendor-reported accuracy numbers, and both have the same problem: aggregate WER is a blunt instrument that hides the information that matters most.
A blended 8% WER across 10 languages can coexist with 25%+ WER on a specific regional accent. The average looks fine; the affected user segment has a poor experience.
The metrics that matter are switch-point WER (accuracy at the 2–3 word window around each language transition), per-language WER breakdown (blended scores mask language-specific regressions), and hallucination rate at transitions (phonetically plausible but semantically wrong substitutions that WER alone doesn’t catch).
The questions to ask vendors: What accent distribution did your test set reflect? What was your switch-point WER? What was your hallucination rate at code-switch boundaries? If they can’t answer, the aggregate score is directional at best.
What Is Code-Switching and Why Does It Break Voice Agents?
💡 Code-switching is when a speaker alternates between two or more languages within a conversation or a single sentence — a common pattern in multilingual communities, not an edge case.
Intra-sentential code-switching — switching mid-sentence — is the hard problem. Monolingual ASR models bias toward their dominant training language and produce phonetically plausible but semantically incorrect words at the switch point. WER spikes of 30–50% at switch points are documented.
The traditional fix — cascade pipeline architecture — adds latency without solving accuracy. A Language Identification (LID) classifier assigns one language label and routes to the appropriate monolingual model. This adds 70–200ms of LID overhead per segment and fails on intra-sentential switches, because the utterance contains two languages and one label has to win.
Deepgram Flux Multilingual handles code-switching natively within a single model, eliminating the LID routing step entirely. If your LLM needs 200ms and your ASR pipeline needs 400ms due to LID overhead, the voice agent is at 600ms before network latency — budget gone.
A typical Indian caller might say: “Sir, aapka loan amount sanctioned ho gaya hai, but documentation pending hai, can you please share the salary slip by tomorrow?” — Hindi structure, English nouns, Hinglish connectors, one sentence. Your users aren’t switching languages to break your model — they’re just talking naturally. Build for that.
How Does Accent Recognition Bias Create Differential Service Quality?
Supporting a language is not the same as supporting the full range of accents within it.
STT models trained predominantly on standard accent data perform measurably worse on regional and non-standard accents. Hindi alone carries substantial dialect variation: Bhojpuri, Awadhi, Haryanvi-influenced, Mumbai-Hindi. A vendor’s “Hindi support” claim covers none of that explicitly.
Dominant ASR models were trained on web-crawled audio from YouTube, podcasts, and broadcast. Indian-language presence in those corpora is materially lower than English or Mandarin. That accuracy gap — documented at 12–25% WER for global models on Indian telephony — maps directly onto demographic segments.
The operational consequence is differential service quality: some customer segments experience longer handling times, more failed self-service interactions, and more escalations. In regulated industries — FinTech, HealthTech — this creates emerging compliance exposure under anti-discrimination frameworks. Voice AI differential accuracy hasn’t been directly litigated yet. But in regulated contexts, that framing is worth taking seriously at deployment design time, not after go-live.
What Does Multilingual Production Deployment Actually Require?
Beyond model selection, production-readiness requires a testing and evaluation methodology calibrated to your caller population — not the vendor’s benchmark conditions. Here’s how to structure it.
Switch-point WER evaluation. Test ASR accuracy at the 2–3 word window around language transitions. Run per-language WER breakdown rather than blended scores. Measure hallucination rate at code-switch boundaries. Pull recordings from your own contact centre and run evaluation against ground-truth transcripts you produce.
Dialect-specific testing. Stratify evaluation audio by accent and dialect within each supported language. If your callers include Mexican Spanish and Argentine Spanish speakers, both must appear in your pre-launch test set. “Spanish support” tells you nothing useful about those two groups.
Vendor accuracy claim validation. Ask: What accent distribution did your test set reflect? What is your switch-point WER? If they can’t answer, treat aggregate WER claims as directional.
Escalation path design. Design fallback workflows for three scenarios: a caller speaking an unsupported language (route to human with a language identification note); consistently low ASR confidence suggesting extreme accent variation (escalate with a transcript-for-review flag); dense code-switching producing frequent low-confidence outputs (language selection prompt or bilingual agent). This is a deployment requirement, not a nice-to-have.
For European deployments, Deepgram Flux Multilingual supports self-hosted EU regional endpoints for data residency compliance. Deutsche Telekom‘s partnership with GPT-Realtime-Translate confirms enterprise-scale European multilingual deployment is live. For context on how these requirements sit within the wider enterprise voice AI deployment picture, the cluster overview covers everything from latency benchmarking to compliance architecture.
The multilingual requirements in the deployment framework don’t change based on which model you choose — they’re calibrated to your callers. Get that right before selecting a vendor and the selection becomes considerably easier, because you know exactly what to ask for.
Frequently Asked Questions
What languages does Deepgram Flux Multilingual support?
Deepgram Flux Multilingual (GA: 29 April 2026) supports 10 languages: English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian, and Dutch. Code-switching between these languages is handled natively without separate routing.
How is GPT-Realtime-Translate different from GPT-Realtime-2?
Architecturally separate models. GPT-Realtime-Translate converts speech in one language to speech in another — 70+ input languages, 13 output languages, $0.034/minute — optimised for translation, not conversation. GPT-Realtime-2 is the conversational reasoning engine. Don’t use them interchangeably.
What is code-switching in voice AI and why does it matter?
Code-switching is when a speaker alternates between languages within a conversation or sentence. Mid-sentence switching is the hardest case: monolingual ASR models spike to 30–50% higher WER at switch points because they can’t hold two phonological systems in context simultaneously. In Indian markets, Hinglish code-switching is the conversational norm, not an edge case.
Is the BolnaAI 12.5% WER improvement for GPT-Realtime-Translate independently verified?
No. It originates in OpenAI’s launch documentation, where BolnaAI is cited as a launch partner reporting 12.5% lower WER on Hindi, Tamil, and Telugu. Vendor-reported, not an independent audit. Treat it as directional.
Can Deepgram Flux Multilingual handle Hinglish or Spanglish?
Deepgram Flux Multilingual handles code-switching between its supported languages natively, including Hindi-English and Spanish-English. That’s an architectural advantage over cascade pipelines, which fail at intra-sentential switch boundaries. Dialect variation within each language — regional Hindi accents, for example — is a separate accuracy dimension not addressed in Deepgram’s published benchmarks.
How do I evaluate multilingual ASR accuracy before selecting a vendor?
Request per-language WER breakdown and switch-point WER — not aggregate WER. Test on audio that reflects your actual caller population’s accent distribution. Measure hallucination rate at code-switch boundaries. If a vendor can’t provide switch-point WER data, treat their aggregate claims as directional.
What is the cascade pipeline problem in multilingual voice AI?
A cascade pipeline routes audio through a Language Identification (LID) classifier, which directs it to the appropriate per-language monolingual model. This adds 70–200ms of LID latency per segment, fails on intra-sentential code-switches, and requires a separate model per language. End-to-end multilingual models like Deepgram Flux Multilingual eliminate the routing layer entirely.
Should I use Deepgram Flux Multilingual or GPT-Realtime-Translate for a multilingual voice agent?
Different tools for different jobs. Flux Multilingual is a conversational ASR model — use it when you need a voice agent that listens and understands in multiple languages. GPT-Realtime-Translate is a translation layer — use it for live speech-to-speech translation across a wide language range. For a voice agent that reasons and responds in the caller’s language, you need Flux, not GPT-Realtime-Translate.
What are the data residency options for multilingual voice agents in the EU?
Deepgram Flux Multilingual offers self-hosted EU regional endpoints for data residency compliance. GPT-Realtime-Translate is used by Deutsche Telekom at enterprise scale, indicating it’s viable for European regulated deployments — confirm data residency specifics with OpenAI for your use case.
What is differential service quality in voice AI, and why is it a compliance risk?
Differential service quality occurs when an ASR model performs materially worse for speakers of non-standard accents or regional dialects — longer handling times, more failed self-service interactions, more escalations for those segments. In regulated industries, this creates emerging compliance exposure under anti-discrimination frameworks. Not yet directly litigated for voice AI, but the framing is consistent with how algorithmic bias in other AI systems has been treated.
What should an escalation path look like for multilingual voice agents?
Design fallback workflows for three failure scenarios: unsupported language (route to human with a language identification note); consistently low ASR confidence suggesting extreme accent variation (escalate with a transcript-for-review flag); dense code-switching producing frequent low-confidence outputs (offer a language selection prompt or escalate to a bilingual agent). An escalation path is a deployment requirement.
