Thread pool size calculator.

Worked example

8 cores at an 80% target, with tasks that wait 200 ms and compute 50 ms: 8 x 0.8 x (1 + 200 / 50) = 32 threads. Each thread handles one task per 250 ms, so 32 threads sustain about 128 tasks per second. The Zalando example is 2 cores with a 50 ms wait and 5 ms of compute: 2 x (1 + 50 / 5) = 22. To serve 100 requests per second you need 100 x 0.25 = 25 threads in flight (Little's law).

How it works

The sizing rule credited to Brian Goetz (Java Concurrency in Practice) is threads = cores x target utilization x (1 + W / C), where W is the average wait and C the average compute time per task. Its logic: while one thread waits, other threads use the core. The throughput ceiling and the threads-for-a-rate lines use Little's law: threads busy = tasks per second x total time per task.

It is a starting point. The rule uses averages and ignores memory, database connection limits, downstream capacity and contention. In containers, check the CPU count your runtime actually sees. Measure under load and adjust.

FAQ

How many threads per CPU core should I use?

For pure compute work, about one per core. For work that waits on I/O, cores x (1 + wait / compute). A 10:1 wait-to-compute ratio suggests around 11 threads per core.

What is the thread pool sizing formula?

Threads = cores x target CPU utilization x (1 + wait time / compute time), attributed to Goetz's Java Concurrency in Practice.

Why use a target CPU utilization below 100%?

It leaves room for other work on the same machine. The utilization term is the variant of the rule used when several thread pools share the cores.

Do database connection limits matter?

Yes. If tasks share a database connection pool or call a rate-limited service, that smaller limit caps useful threads. Check each limit and measure under load.