What I learned building a real-time AI conversation practice tool

The live conversation loop

The browser connects to an ElevenLabs Agent for the live call. ElevenLabs handles speech recognition, the language-model exchange, and the returning voice audio. The app supplies the situation, the character's role, the opening line, and the voice selected for that session.

Keeping that work in one streaming system matters. A chain of separate speech-to-text, text-generation, and text-to-speech requests is easier to inspect, but each handoff adds delay. For a rehearsal tool, a short pause can feel much longer than it measures because the user is waiting for another person to answer.

The user still has to grant microphone access, and network conditions still matter. The site can reduce its own overhead, but it cannot promise a fixed response time on every device and connection.

Latency is not one number

A useful latency check separates the delay into parts: detecting that the user has stopped speaking, producing a response, generating the first audio, and playing it in the browser. Changing a language model may help one part while making another part slower.

The end-of-turn decision is especially noticeable. If the system waits too long, every response feels sluggish. If it decides too quickly, it interrupts natural pauses. The best setting depends on the conversation. An upset parent may answer in short bursts, while a job candidate may need a moment to think.

The current prototype favors a responsive exchange rather than a long, polished answer. Character instructions ask for concise spoken replies. That usually helps more than asking a faster model to generate a paragraph the user did not need.

• Measure time to first returned audio, not only the total model request.
• Test interruptions and short acknowledgements, not only complete sentences.
• Keep character replies short enough to sound like a conversation.
• Repeat tests on a phone and an ordinary network connection.

A stable character needs more than a voice setting

Voice controls such as stability, similarity, and speaking style affect the sound. They do not, by themselves, keep the character consistent. The prompt also has to define what the person wants, why they are upset, what they know, and what would make them move toward a next step.

The ready-made scenarios therefore include hidden context as well as an opening line. The character is told not to calm down just because the user says one polite phrase. It should respond to the substance of the exchange: whether the user recognized the concern, asked a useful question, checked their understanding, and proposed a concrete next step.

This is still a simulation. The same scenario can develop differently across runs, and a generated character can misunderstand a good response. That variation can be useful for practice, but it also means the tool should not present its result as an objective assessment of the user.

Making a conversation possible to finish

The first prototype made the other character angry, but it did not give the model a clear way to become constructive. A user could handle the conversation well and still see no change. The fix was not to add a magic calming phrase. It was to define an ordered set of observable jobs for the conversation.

Each guided scenario now follows four broad skills: recognize the concern, ask a relevant question, check the user's understanding, and agree on a next step. The character can soften as those jobs are handled. The post-call review uses the same structure so the conversation and the feedback are at least aiming at the same target.

A rigid order would be frustrating, because real conversations move back and forth. The state is guidance for the character, not a script the user has to recite. The review should point to evidence from the exchange and leave room for more than one good way to respond.

Cost controls are part of the product

Live voice calls cost money while they run. Say It Ahead uses a shared community pool, so starting a call has to reserve enough capacity before the provider begins charging. A guided session is capped at five minutes. The app sets aside a five-minute estimate, then replaces that estimate with the reported duration and cost after the call.

This avoids two bad outcomes. Without a reservation, several people could start at once and spend more than the pool holds. Without reconciliation, someone who practices for 30 seconds would be charged as if they used the full five minutes.

There are several ways a call can end. The normal browser flow releases or reconciles the reservation. An ElevenLabs completion webhook provides a second path. A scheduled cleanup checks old active sessions in case the browser closed and the webhook did not arrive. Billing work is not glamorous, but a public demo is unreliable without it.

Privacy changes what the product should ask for

A custom scenario is more useful when it has context, but the site should not encourage people to paste private records into it. The interface asks for fictional labels and warns against entering student records, medical details, customer information, or confidential work material.

Audio and transcripts have to be processed by ElevenLabs and its model provider to run the conversation and produce feedback. Say It Ahead does not store raw audio or full transcripts in its own database. It does store operational records such as session status, duration, cost, and result data needed for account limits and the community pool.

That boundary needs to stay visible in the product. A privacy page alone is not enough if the custom-scenario form invites sensitive detail without a warning at the point of entry.

What still needs work

The largest open question is whether the practice transfers to a real conversation. A believable call and a tidy score do not prove that. The immediate product goal is narrower: give someone a low-pressure place to say the words out loud, hear pushback, and try again.

The next useful work is measurement, not more decorative features. I want to compare latency across devices, review where characters change tone too early or too late, and see whether users understand the four conversation skills without extra explanation. I also want to learn which situations people repeat, because repeat practice is a stronger signal than a page view.

Real-time AI voice is good enough to make this experiment worth running. It is not good enough to hide its limits. The product is more useful when it states those limits plainly and lets people judge the practice for themselves.