Why Translation Apps Lag — And What We Found When We Measured Our Own

If you've used a real-time translation app, you know the feeling. You say something. Then you wait. Then the other person hears it, replies, and you wait again. A two-second gap doesn't sound like much written down, but in a conversation it's brutal. People start talking over each other. They give up on jokes. They stop having a conversation and start taking turns making announcements.

Almost everyone blames the AI.

We spent the last month instrumenting every stage of our own pipeline, and the AI turned out to be the fastest part of it. Here's what's actually happening in that gap, and what we found when we finally measured instead of guessed.

What has to happen between you speaking and them hearing

Live translation isn't one thing. It's a relay race, and every runner has to finish before the next one starts.

1. Your phone has to decide you've stopped talking. Software can't translate half a sentence — it needs to know where the thought ends. So it waits for a pause. Too short a pause and it cuts you off mid-sentence; too long and everything feels sluggish. This is a design choice, not a technical limit, and it's the first few hundred milliseconds.

2. Your speech becomes text. Speech recognition, running on a server somewhere.

3. The text gets translated. This is the part everyone calls "the AI."

4. The translated text becomes a voice. Text-to-speech synthesis generates an audio file.

5. That audio travels to the other person's phone and plays.

Five stages, each one waiting on the last. And here's the thing that makes it hard: you can't parallelize a conversation. You can't translate a sentence before you've heard it. Every optimization has to come out of one of those five stages, because the order can't change.

We were measuring the wrong thing

Our app had one timer that wrapped the middle of that chain. We watched it for months. It read about two seconds, and since we knew translation sat inside it, we concluded translation was slow.

That was wrong, and the mistake is embarrassing in hindsight. The timer wrapped several stages at once — translation, voice generation, and a lot of our own bookkeeping in between. We'd built a measurement that could tell us the total but never the parts. So we blamed the piece we could name and stayed blind to the piece we couldn't.

In September we finally broke it apart and recorded each stage separately. The first real data came from a 26-minute call between a user in Greece and a user in Brazil, speaking Italian and Portuguese.

Stage	Measured
Translation	200–400 milliseconds
Voice generation (whole stage)	2.5–2.8 seconds

So the thing we'd been blaming was fine. But the second number came with a twist. When we measured how long the actual voice synthesis took — the part an outside company does for us — it was well under a second.

Which meant roughly two seconds of that stage was us. Our own code. Not the AI, not the voice provider, not the network. Our own software, being slow in a way we'd never been able to see.

The culprit was geography

Here's what was happening. Every time our system translated a sentence, it needed to check a handful of things — is this call still active, has this sentence already been handled, does this user have minutes left. Reasonable checks. But it made them one at a time, waiting for each answer before asking the next question. Fifteen separate trips to our database for a single translated sentence. One of those checks ran four times.

When the user and the database are both in North America, each trip is quick and fifteen of them go unnoticed. When the user is in Brazil and the database is in California, every trip crosses a continent and an equator. Fifteen round trips becomes seconds.

Which explains something that had been confusing us for weeks. Our latency numbers had been getting steadily worse — from about 1.5 seconds in early September to nearly 4 seconds a week later. We assumed something had broken.

Nothing had broken. Our users had moved. Early calls were between Americans. Recent ones were Brazilian, Greek, Hungarian, Italian, Bangladeshi. The pipeline didn't degrade — the user base got further from the database. We could never have seen that without recording which country each request came from, which is the entire argument for measuring things you think you already understand.

What happened when we fixed it

We rewrote the path to ask its questions in batches instead of one at a time. Fifteen round trips became seven.

Then we got a rare gift: the same two people, on the same phones, in the same two countries, made a call before the fix and another after. Same route, same languages. A genuine before-and-after, which almost never happens outside a lab.

Listener	Before	After	Change
Chile	3638 ms	1572 ms	−57%
United States	1491 ms	1026 ms	−31%

The user in Chile got back more than two seconds. The user in the United States, on the identical change, got back half a second.

That asymmetry is the proof. If we'd simply made the code more efficient, both users would have improved by similar amounts. Instead the saving scaled with distance from the database — exactly what you'd predict if the problem was long-haul trips, and hard to explain any other way.

We also checked that our voice provider hadn't just gotten faster on its own. Its timings were flat across both calls: 297 ms before, 294 ms after. The improvement was ours.

The safety net that was making things worse

This is the finding that surprised us most, and it's the one we'd want another team to take away.

Our app generates translated audio in the cloud, which sounds better than what a phone can produce on its own. But cloud audio has to download, and downloads can be slow. So we built what seemed obviously correct: a deadline. If the good audio hadn't arrived within a set time, we'd give up on it and use the phone's built-in voice instead. A safety net.

We tried it at 700 milliseconds. Then at 1.5 seconds. We measured both.

At 700 ms, we lost 4 sentences out of 31. But when we looked at what actually happened on those four, the downloads had finished — at 643, 655 and 686 milliseconds. They were 40 to 90 milliseconds away from playing. We threw away audio that was essentially ready, and the phone's own voice then took between 1.5 and 2.4 seconds to say the same thing.

At 1.5 seconds, on a recent call, we lost 8 sentences out of 122 for the listener in Chile. Seven of those downloads completed between 1482 and 1493 milliseconds — again, about 50 milliseconds short. The replacement then averaged 3.1 seconds against the 843 milliseconds of the audio we'd just discarded.

The listener in the United States on that same call lost nothing at all, out of 182 sentences.

So the safety net was firing almost exclusively on slow connections — and on slow connections it was making things three times worse. Every sentence it "saved" arrived later than if we'd done nothing.

There's no third number that fixes this. Falling back to the phone's own voice costs 1.5 to 3.6 seconds from cold, so a deadline only helps if it fires well under a second — and firing that early is precisely what kills downloads that were about to succeed. Every possible deadline is either too early to be safe or too late to be worth taking.

We deleted it. No deadline, no fallback timer. The app now waits for the good audio and only gives up if the download genuinely fails.

What we still can't fix

We're not done, and some of it isn't ours to finish.

Distance is physics. Our database lives in North America and there's no version of it available in South America today. A user in Santiago is paying for the geography of the internet, and no amount of clever code removes a continent.

The last mile belongs to the user's carrier. On that Chile–US call, the Chilean user's connection added 300 milliseconds before our software did anything at all. The American's added 30. That ten-fold gap, paid on every sentence in both directions, is most of what still separates their experiences.

Some languages are just slower to synthesize. At the same sentence length, Italian took 821 milliseconds to generate while English, Spanish and Portuguese took around 300. Same provider, same settings. That's a property of the voice model, and it means "how fast is your app" has a different answer depending on what you speak.

What we'd tell anyone building this

Measure the stages, not the total. A single timer that wraps several steps will let you blame the wrong one for months. Ours did.

Record where your users are. Our worst-looking week was actually a healthy week with a more international user base. Without country on each measurement, that reads as a regression and you go hunting for a bug that doesn't exist.

Be suspicious of fallbacks. A safety net that fires on slow connections is worse than useless if the thing it falls back to is slower than what it interrupted. We ran ours for months at two different settings before we checked whether it was helping.

And the AI probably isn't your problem. Translation was 200 milliseconds in a two-and-a-half second stage. The rest was our own plumbing and the distance between a user and a database.

We're still not where we want to be. A conversation feels natural somewhere under a second, and we're above that for users far from our infrastructure. But we know what the remaining time is made of now, which is a considerably better position than guessing.