LiveKit voice agents, self-hosted or on LiveKit Cloud
LiveKit carries live audio between a person and an AI agent. We self-host it for our voice agent work, and this page covers the choices that matter before an agent takes its first real call.
Reviewed by: Rahul Jaiswar, founder
What LiveKit is
LiveKit is an open-source server for real-time audio and video, with an Agents framework for building AI voice agents. You can run the server yourself or use LiveKit Cloud, the company’s hosted service.
How we use it
We run self-hosted LiveKit for real-time voice in our voice agents (AI voice agents).
Self-hosted or LiveKit Cloud
| Self-hosted | LiveKit Cloud | |
|---|---|---|
| Where audio runs | A server in the location you choose | LiveKit’s infrastructure |
| Who keeps it running | We deploy and host it in an account you own; cover after launch follows the scoped support levels (running it) | LiveKit |
| Features | The open-source server | Extra hosted features; some, such as its noise cancellation, may not run self-hosted |
| Bill | Server cost you control | Usage-based |
| Phone calls | Needs LiveKit’s SIP service running alongside the server | Telephony offered as part of the hosted service |
When it fits
- A voice agent in a browser or on phone calls, for enquiries, bookings or follow-ups.
- Audio, recordings and transcripts that must stay on servers you pick. Self-hosting LiveKit achieves that only if speech-to-text, text-to-speech and the model also run on your servers.
- Calls that may pass from the agent to a person partway through.
When it doesn’t
- A fixed menu (“press 1 for sales”) needs a phone system, not an AI voice stack.
- Recording calls, nothing more. Telephony providers can usually do that on their own. For phone lines, see Telnyx and Plivo.
Which language the agent speaks is a project decision. We test a model in the language you need, including languages outside India, before committing to it.
Five questions for your build team
- Self-hosted or Cloud, and why for our case?
- Where will the server sit relative to our callers?
- Where are recordings and transcripts stored, and who can open them?
- What happens to a caller if the agent process crashes mid-call?
- How does a call pass to a person, and what does that person see?
Before a real call
- Transcripts can silently stop saving. Some conversation items are not speech, and code that treats them as speech can fail without a visible error. Filter conversation items to messages before saving the transcript, and check the transcript of a real test call, not only the logs.
- Idle memory. Each warm agent process loads speech models, so set how many stay warm.
- Microphones need a secure page. Browsers grant the mic only over HTTPS, and that page then needs a secure WebSocket to the server.
- Media ports. Audio uses its own ports, which have to be opened and tested from outside the server.
- Strict firewalls. Some office and mobile networks block those ports. Configure TURN over TLS on port 443 as a fallback so those users still connect.
Planning the handover to a person
Decide before building what the agent passes on when a caller asks for a human: a live transfer, a callback, or a written summary for your team. Then decide what the person sees on screen when the call reaches them, so the caller doesn’t have to repeat themselves. Test the handover path on a real call as carefully as the agent’s answers.
Related
Planning a voice agent on LiveKit?
Start with the calls it should take, and whether you already have a LiveKit account.