AI hardware · running cost
What does a million tokens cost in electricity?
Watts while generating, tokens per second and your price per kWh in; cost per million tokens, energy and time out.
Inputs
Starting values are illustrative, not current prices or specifications. Replace each with your own.
Result
Answer
Electricity per million tokens is power in kilowatts, multiplied by the hours it takes to generate a million tokens, multiplied by your price per kWh. A million tokens at 40 tokens per second takes 25,000 seconds, about 6.9 hours.
Worked example
At 300 W and 40 tokens per second, one million tokens takes 1,000,000 ÷ 40 = 25,000 s = 6.94 h. Energy is 0.3 kW × 6.94 h = 2.083 kWh. At 0.15 per kWh that costs 0.3125 per million tokens (0.000313 per thousand).
Formulas and notes
- Hours per million tokens = 1,000,000 ÷ tokens per second ÷ 3,600.
- kWh per million tokens = watts ÷ 1,000 × hours.
- Cost = kWh × price per kWh.
- Use the draw of the whole machine while generating, not the card's rated maximum, and note idle power is not included.
- Batching several requests raises total throughput (Databricks calls batching critical for it), so a shared server's cost per token can differ from this single-stream figure.
Planning estimate only, not a benchmark: real memory use and speed depend on the model file, the runtime, the driver and the hardware.
Sources, opened 2026-10-08: Databricks, LLM Inference Performance Engineering: Best Practices (12 Oct 2023)
FAQ
How do I calculate the electricity cost of running an LLM?
Divide one million by your tokens per second to get seconds, convert to hours, multiply by the draw in kilowatts to get kWh, then multiply by your price per kWh.
What wattage should I enter?
The power the whole computer pulls from the wall while it generates, ideally from a plug-in meter. A card's rated maximum may differ from the real draw during generation.
Is the electricity price I should use the same as my bill rate?
Use the marginal rate per kWh from your bill. If you pay time-of-use rates, use the rate for the hours you run.
Why is the result so small compared with API prices?
It covers electricity only. Hardware cost, your time and cooling are not included; the local vs API break-even calculator adds the hardware.
Does this include idle power?
No. Add idle watts yourself by raising the draw or the hours if the machine stays on between jobs.