API vs Self-Hosted Models: Cost Breakdown

The "build vs buy" question for LLM inference comes down to economics. API providers charge per token. Self-hosting charges per GPU-hour. At low volumes, APIs win. At high volumes, self-hosting wins. The crossover point is where the decision gets interesting.

The API Cost Model

With providers like OpenAI and Anthropic, you pay per input and output token. Pricing varies by model tier. At low volumes this is ideal — zero infrastructure overhead, instant scaling, no DevOps burden.

The problem emerges at scale. A system processing 10 million tokens per day at Claude Sonnet pricing costs roughly $900/month on input alone. At 100 million tokens per day, you are looking at $9,000/month — and that scales linearly with no volume discount on most tiers.

The Self-Hosted Cost Model

Self-hosting an open-source model (Llama, Mistral, DeepSeek) on dedicated GPUs converts per-token costs into fixed infrastructure costs. A single NVIDIA A10G on AWS costs roughly $500–800/month. Running vLLM with continuous batching, that GPU can serve approximately 50–100 million tokens per day depending on model size and quantisation.

The break-even point for most workloads falls somewhere between 20 and 50 million tokens per day. Below that, APIs are cheaper when you account for engineering time. Above that, self-hosting delivers 3–10x cost savings.

The Hidden Costs of Self-Hosting

Self-hosting is not just GPU rental. You need engineers who understand model serving, monitoring for quality regression, redundancy and failover, and ongoing model updates. Budget an additional 20–40% above raw compute costs for operational overhead.

Our Recommendation

Most production systems benefit from a hybrid approach: API calls for complex reasoning tasks (where frontier model quality matters) and self-hosted inference for high-volume, simpler tasks (classification, extraction, formatting). The routing layer between them is where the real savings happen.