The other half of the price question
Self-host or buy?
67 of the models tracked here are permissively licensed and download without asking, so paying per token is a choice. A rented GPU costs the same whether you serve one token or a billion, and the API has no fixed cost — so the question is simply whether your bill clears the rent. Set your volume and see.
At 100M tokens a month, buying the API wins for every model here. A rented GPU costs more than your whole bill would.
Not your decision to make
Licence says API only
These publish their weights, but under a licence that forbids commercial use. However large you get, buying the API is the only route for a product.
- Aya Vision 32Bcc-by-nc-4.0
- Aya Expanse 32Bcc-by-nc-4.0
- Aya Vision 8Bcc-by-nc-4.0
- Aya Expanse 8Bcc-by-nc-4.0
Bigger than one box
Beyond a single node
At bf16 these need more memory than eight GPUs hold, so a meaningful rent figure would mean pricing a cluster and the networking around it. Quantising to 8- or 4-bit roughly halves or quarters the requirement and can bring some of them back into range — at some cost to quality.
- Kimi K32780B · ~6950 GB
- DeepSeek V4 Pro 08131650B · ~4126 GB
- DeepSeek V4 Pro1599B · ~3997 GB
- Kimi K2.51027B · ~2567 GB
- Kimi K2.61027B · ~2567 GB
- Kimi K2.7 Code1027B · ~2567 GB
- Kimi K2 Thinking1026B · ~2566 GB
- MiMo-V2.5-Pro1023B · ~2558 GB
How this is worked out
Method
Throughput is deliberately not modelled. Tokens per second depends on hardware, batch size, quantisation, framework and sequence length. Any single figure would be invented precision, so instead of guessing we compare the one cost that is certain: the rent.
Memory is parameters × 2 bytes (bf16) plus 25% for KV cache, activations and framework overhead. Long contexts and large batches need more. We then pick the cheapest configuration of up to eight GPUs whose combined VRAM holds that.
GPU rates are representative on-demand list prices as of 2026-08 — A100 40GB $1.1/hr, A100 80GB $1.8/hr, H100 80GB $3/hr, H200 141GB $4/hr — billed at 730 hours a month. Reserved and spot capacity run materially cheaper.
The comparison flatters self-hosting. It assumes a permanently saturated GPU and counts no engineering time, no redundancy, no idle capacity and no traffic peaks. Real deployments are worse on every one of those. So treat a positive difference as permission to price it out properly — not as a saving.
licence and parameter counts from huggingface.co model cards · API prices are the cheapest listed blended rate across every provider we track