No model on the shelf hears Latvian on a phone line. We trained one.

We trained our own Latvian↔Russian detection model on real telephony audio, at the bandwidth and codec a phone line actually delivers, with ground truth labelled by native speakers. Part one showed what happens to models trained on anything cleaner. This one heard the line from the first example.

Latvian becomes an answer the model can actually give. On real Latvian speech our model is about 90% sure it is hearing Latvian; whisper-small is under 0.1% sure. It labels roughly nine fragments in ten of Latvian speech Latvian, and when it is wrong it says Russian, the other language actually on the call. It is the only model we measured that weighs Latvian against Russian. Every off-the-shelf model only scores how Russian the audio sounds; their Latvian scores are measurably above zero and nowhere near the scale a decision needs, and the bigger Whisper tiers get better at spotting Russian without recognising Latvian any more often.

It is also fast: about 21 milliseconds per fragment, against about 0.7 seconds for the Whisper tier a live call can afford and 5.3 seconds for the most accurate one.

Every number in this piece comes from one test. Recorded real calls, Latvian and Russian speakers, are replayed through each build exactly as they arrived on the line, and every decision is scored against what native speakers say was spoken.

Two units appear. Per fragment, a second or so of speech is scored on its own, as in part one. Per caller, a whole call is scored on whether a Russian-speaking caller would have been switched at any point. Both are held to a strict ceiling on Latvian-speaking callers wrongly switched, every decision rule was set on calls the reported rates never saw, and where two builds are compared they run under the same switch rule. The two error rates are not the same number: a fragment wrongly flagged does not mean a caller wrongly switched, so fragment error rates run higher than caller ones for every build.

Latvia has its own speech models, and the obvious objection to part one is that we should have used them. The Institute of Mathematics and Computer Science at the University of Latvia, AiLab, publishes openly licensed Latvian speech models: AiLab-IMCS-UL/whisper-large-v3-lv-late-cv17 and AiLab-IMCS-UL/whisper-large-v3-lv-phono.

The specialist does the hard part well. It is the only third-party model in this work that recognises accented Latvian as Latvian, and measured per call, that recognition covers roughly seven in ten of the wrongful switches on this line. The result held when we widened from a small sample to full coverage.

Scored as bilingual detectors, on clean synthetic audio they can tell the two languages apart: the stronger one catches two Russian speakers in five. On real telephony audio both answer "Latvian" to almost everything, Russian speech included. The stronger catches about one Russian speaker in 75, the other about one in 100. A synthetic benchmark overstates the general models by 2× to 4.6×. It overstates these two by 23× and 31×.

These models were built to transcribe Latvian, and they are good at it; a Latvian transcriber that answers "Latvian" on a Latvian line is behaving correctly. Measuring them as bilingual detectors measures them at a job they never claimed. We do it because someone evaluating voice AI for this market will be told to repurpose them into one, and the benchmark will say it works while the phone line says otherwise. The ability is still in the models; clean audio proves it. The telephony channel is what removes it.

Put our model through exactly part one's test, the same real telephony audio, the same native-speaker ground truth, the same strict ceiling on wrongful switches, and per fragment it catches 37% of Russian speech, the same as whisper-small, the Whisper tier a live call can afford. It does that in 21 milliseconds against 0.7 seconds, and with about 90% confidence in Latvian where whisper-small has under 0.1%. Whisper's larger tiers catch more per fragment and cannot answer inside a live turn.

Per-fragment accuracy is where every model on this channel tops out, ours included. That number was the ceiling for the model. It was not the ceiling for the system, and the speed is what let us move past it.

A caller is not a second of audio. A caller is half a minute of stops, starts, background noise and half-words, and the decision that matters is about the person.

At 21 milliseconds a fragment there is room to score every fragment a caller produces and still answer inside a live turn. The Whisper tier a live call can afford keeps up on a typical fragment and stalled for nearly thirty seconds on its worst one. So we moved the decision from the fragment to the caller.

Every bar below comes from the same replay of recorded real calls, with native-speaker ground truth and the wrongful-switch rate on Latvian callers stated per bar.

Our model alone, judged fragment by fragment, catches 57% of Russian-speaking callers at a wrongful-switch rate of about one Latvian caller in 140. The off-the-shelf Whisper build catches 67% at about one in 70, so those two bars sit at different error rates and the gap between them is not a like-for-like result. Our detector, deciding per caller, catches 82% at the off-the-shelf build's own rate of about one in 70. That recovers roughly half of the callers the off-the-shelf build leaves in the wrong language, without one extra wrongful switch.

Part one sharpened the questions buyers already ask. This measurement adds the ones specific to adaptation claims, because "tuned for your language, your channel, your industry" is now a standard line, and it is measurable.

These are measurements on a replay of recorded calls: one line, one window. A live field rate is a different number and is not claimed here. Per-caller and per-fragment numbers are different units and stay labelled as such throughout. The off-the-shelf comparison is one configuration of one Whisper tier under our switch rule; the larger tiers were not run per caller because they cannot answer inside a live turn. Nothing here says our model is a better speech model in general: whisper-large-v3 remains the most accurate model we measured per fragment, undeployable on a live call. Our model and our detector appear here as measured outcomes only. And nothing here covers commercial cloud APIs, which neither piece in this series publishes.