LLM tokens and cost
Tokens per second to response time.
Enter the output length and the generation speed you measured. Optionally set a target time to see the speed or token budget it needs.
Response time = time to first token + output tokens / tokens per second. An 800-token reply at 50 tokens per second with a 0.8 s first token takes 16.8 s (16.0 s of generation). To finish in 10 s it needs 87.0 tokens per second, or a reply of 460 tokens or fewer at 50.
Your numbers
Result
Worked example
An 800-token reply at 50 tokens per second with a 0.8 s time to first token. Generation: 800 / 50 = 16.0 s. Total: 0.8 + 16.0 = 16.8 s. Each token takes 1,000 / 50 = 20 ms. For a 10 s target, the generation budget is 10 - 0.8 = 9.2 s, so the speed needed is 800 / 9.2 = 87.0 tokens per second, or at 50 tokens per second a reply of at most 9.2 x 50 = 460 tokens. Halving the output to 400 tokens gives 8.8 s.
How it works
OpenAI's latency optimization guide says generating tokens is almost always the highest-latency step when using an LLM and gives the heuristic that cutting 50% of your output tokens may cut about 50% of your latency. This page applies that as plain arithmetic: once the first token arrives, the rest of the reply streams at the generation speed.
The speed and first-token delay are inputs because they differ by model, provider, load and prompt length, and neither is a fixed property. Measure them on your own traffic, ideally the median and a slow percentile. The model ignores network time, queueing, tool calls, any reasoning tokens generated before the visible answer (count them in the output tokens) and speed changes during a response. Convert a word count to tokens with the token estimator.
Sources and basis (opened 2026-10-08): Latency optimization | OpenAI API. Everything else is arithmetic on your inputs.
All prices on this page are examples you can overwrite, not any provider's current prices. Check your provider's pricing page before you rely on a number. Results are planning estimates, not a quote or financial advice. Everything runs in your browser. Nothing you type is sent or stored.
FAQ
How long will my LLM response take?
Add the time to first token to the output tokens divided by tokens per second. An 800-token reply at 50 tokens per second with a 0.8 s first token takes 16.8 s.
What is a good tokens-per-second figure?
It varies by model, provider and load, so measure it on your own requests instead of relying on a published number.
How do I make responses faster?
Generate fewer tokens. OpenAI's latency guide says cutting 50% of output tokens may cut about 50% of latency. Here halving the reply to 400 tokens takes 8.8 s instead of 16.8 s.
Does reasoning time count?
Yes. If the model generates reasoning tokens before the visible answer, add them to the output tokens or to the time to first token.