Open AI models on your own machines, with Ollama
When data can’t go to an outside AI provider, a model running on your own machine is often the first thing to try. We can set it up and connect it to the tools your team uses.
Reviewed by: Rahul Jaiswar, founder
What Ollama is
Ollama is open-source software that downloads open models and runs them on a laptop, desktop or server. Applications reach it through a local API, much as they would a hosted provider.
How we use it
- Local models: we run open models locally with Ollama.
- MicroFlow’s copilot can use models from Ollama Cloud (MicroFlow). Ollama Cloud runs on Ollama’s servers, so prompts sent there leave the machine. Local and cloud are separate choices.
When it fits
- Private records such as client files, HR data or internal notes that policy keeps on your hardware.
- A trial before any API bill, to learn whether a feature is worth building.
- A small internal team using a tool through the day.
- A site with patchy internet, where a model on the office server keeps working offline.
When it doesn’t
- Many users at the same moment. Ollama is built for convenience rather than heavy concurrent load; a served setup on GPUs is the step up (self-hosted LLM deployment).
- Tasks that need the strongest models. Some reasoning and writing still goes better on a hosted model (OpenAI).
- No one to maintain the machine. We can host and run it in an account you own, under cover agreed in the scope of work (what running a self-hosted system involves).
Before you install
- Compressed versions trade quality. Quantised models use less memory and can answer worse; compare them on your own examples.
- Licences differ by model. An open model is not automatically free for commercial use.
- Keep it off the public internet. Ollama listens only on localhost by default, so put a proxy with sign-in in front before exposing it.
Related
Need AI that stays on your own hardware?
What should the model do, and which machines do you have today?