Python backend development for data and background workers
When the job is cleaning messy records, scoring them or handling audio, Python usually has the library already. The real work is fitting it into the rest of your system.
Reviewed by: Rahul Jaiswar, founder
Where we use it
MicroLead and our voice agent workers are written in Python with FastAPI. The real-time voice layer those agents use is covered on the LiveKit page.
What we check
Before anyone uses the output, we run the code against a small set where we already know the right answers. Scored or cleaned data looks convincing whatever it says, so a known answer is what tells you it is right.
What Python is
Python is a general-purpose language known for readable code. It has a very large set of libraries for data work, scraping, scientific computing and machine learning.
Python or Node.js for a new service?
Node.js, unless the job is data-shaped or leans on a library that is strongest in Python. Adding Python to a product already written in TypeScript means a second deployment and a wider hiring brief, so a plain create-read-update API is not worth it. Cleaning, scoring or deduplicating records, or working with audio, usually is.
Choosing Python
Choose it when the work is data-shaped (deduplicating, normalising, scoring, reporting), when a provider’s AI or speech SDK is strongest in Python, or when a process runs on its own schedule apart from the web app.
Think twice when the rest of the product is TypeScript and the new service is a plain create-read-update API; Node.js avoids a second language. The same goes for heavy computation that needs many CPU cores in one process: standard Python builds run one thread of Python code at a time, so that work is split across processes. Screens are rarely a Python job.
When the Python code needs an API
FastAPI is a Python framework for building web APIs. It uses Python type hints with Pydantic to validate each request, and generates OpenAPI documentation from the code. It suits Python logic that a Next.js or Node.js app needs to call, and APIs that mostly wait on other APIs. Django is the better start when you need server-rendered pages, an admin panel and user accounts out of the box, and a script on a schedule may need no API at all.
The async trap. FastAPI endpoints can be async def or plain def. A blocking call, such as a synchronous database driver or HTTP client, inside an async def endpoint freezes the event loop for the requests on that worker. Put blocking calls in plain def endpoints, which FastAPI runs in a thread pool, or switch to async clients.
Before launch: switch off or protect the generated docs, which list the endpoints and their inputs; list allowed CORS origins explicitly; and keep slow jobs out of requests, since proxies cut long requests off (Cloudflare’s limit is on its page) and a queue handles them better.
Where Python services trip up
- Duplicates that don’t look alike. One business can appear with different spellings or phone formats. Normalise names and numbers before deduplicating, since an Indian record set mixes address formats and +91 or 0-prefixed phone numbers.
- Environments that drift. Unpinned dependencies make a service work on one machine and fail on another. Lock versions.
- Third-party limits. Data and AI APIs have rate limits and per-call costs. Size batches, cache results and make runs resumable.
- Row-by-row writes. Updating thousands of rows one query at a time over a network is slow. Write in batches.
For automation work that uses Python behind the scenes, see AI automation.
Data to clean, score or put behind an API
Tell us where it comes from today and what you want at the end.