Hosted or self-hosted: choosing where your AI model runs
Where an AI model runs decides where your business data ends up.
Reviewed by: Rahul Jaiswar, founder
Side by side
| Hosted API | Hosted model in your cloud account | Open model, one machine | Open model, GPU servers | |
|---|---|---|---|---|
| Our pages | OpenAI, Anthropic | Claude through a cloud platform (Anthropic) | Ollama | Self-hosted LLM deployment |
| Where requests go | The provider’s servers | A provider’s model offered inside your cloud account (for example Microsoft Foundry, Amazon Bedrock or Google Vertex AI), in a region you choose where it is available | Your machine | GPU servers or serverless GPUs you rent |
| Bill | Per use, in foreign currency | Per use, on your cloud bill | Hardware you buy | GPU time; serverless scales down but cold-starts |
| Upkeep | Provider runs the model | Cloud provider runs the model | You run one machine | You run a serving setup |
| Suits | Quick starts, hard language tasks | Hosted model quality, processed by your cloud provider in a region you choose, under your cloud contract | Private data, small teams | Many users on one open model |
Four questions that decide it
- May the data leave your systems? If contracts or customers rule out a public API, look at a provider’s model inside your own cloud account and region, or at an open model you host. For data that should stay in India, check which models your cloud provider offers in its Indian regions.
- How do the models do on your own examples? Collect real inputs (enquiries, call notes, documents) and compare setups on those. Public benchmarks won’t tell you how a model handles your inputs.
- How many people use it at once? A few staff through the day and a customer-facing feature need different hosting.
- Could you switch later? Keep model calls behind one layer of code, so a provider change touches little.
What running a self-hosted system involves
A self-hosted model or voice server is a system someone must update, monitor and restart when it fails. We deploy and host what we build, in an account you own, and you choose serverless or a managed server at the start. Cover after launch is the scoped, paid option: standard cover in business hours, with round-the-clock cover including emergencies as a paid upgrade, agreed in the scope of work (running it after launch). If nobody will hold that cover, a hosted option is often the better trade even at a higher usage bill.
What the model knows
A model answers from its training unless you give it context. Retrieval adds that context at question time (RAG).
Voice
A voice agent turns speech into input for a model and speaks its reply. We run the real-time voice layer on self-hosted LiveKit (LiveKit), and the voice agent service covers what those agents do (AI voice agents).
Where this meets our services
Model choice sits inside larger automation builds (AI automation), and inside integrations that connect AI steps to your existing systems (integrations). Where the answer is an open model on your own account, we choose it, build it and run it (self-hosted LLM deployment).
Weighing a hosted model against your own hardware?
Send one real example of the task and a line on where your data is allowed to go.