Requirements & Design Decisions

Let's define what our Thread Pool must do. Each requirement is specific enough to drive a class, method, or synchronization decision in the final design.

R1: Fixed Worker Count. The pool shall be constructed with a configurable number of worker threads (pool_size). All workers are created and started at construction time. The pool size does not change after creation.

R2: Task Submission. The pool shall accept callable tasks via a submit(task) method. Each submitted task is a unit of work that returns a result (or raises an exception). Submission returns immediately — it does not wait for the task to execute.

R3: Future-Based Results. Each call to submit() shall return a Future object. The caller can call future.get() to block until the result is available. If the task raised an exception, future.get() shall re-raise it.

R4: Bounded Task Queue. The internal task queue shall have a configurable maximum capacity (queue_capacity). When the queue is full, the submission behavior is governed by the rejection policy (R6).

R5: FIFO Task Ordering. Tasks shall be executed in the order they were submitted (first-in, first-out). No priority-based reordering.

R6: Rejection Policy. When the queue is full and a new task is submitted, the pool shall apply one of three configurable policies:

• ABORT: Raise an exception to the caller
• CALLER_RUNS: Execute the task on the submitting thread (back-pressure)
• DISCARD: Silently drop the task

R7: Graceful Shutdown. Calling shutdown() shall:

• Stop accepting new task submissions (subsequent submit() calls raise an exception)
• Allow all currently queued tasks to complete
• Allow all in-flight tasks (being executed by workers) to finish
• Terminate all worker threads after the queue is drained

R8: Immediate Shutdown. Calling shutdown_now() shall:

• Stop accepting new task submissions
• Clear the task queue and return the discarded tasks to the caller
• Allow in-flight tasks to finish (we don't forcibly kill threads — that's unsafe)
• Terminate all worker threads after in-flight tasks complete

R9: Await Termination. The pool shall provide an await_termination(timeout) method that blocks the calling thread until all workers have exited or the timeout expires. Returns True if the pool terminated within the timeout.

R10: Worker Lifecycle. Each worker thread shall run an infinite loop: pull a task from the queue (blocking if empty), execute it, store the result in the corresponding Future. The loop exits when shutdown is signaled and the queue is empty (graceful) or immediately after the current task (immediate).

Thread pools are inherently concurrent — every operation touches shared mutable state. These requirements define the thread-safety guarantees:

C1: Thread-Safe Queue. The internal task queue must be safe for concurrent access by multiple submitter threads and multiple worker threads simultaneously. This is the classic multi-producer, multi-consumer problem (Fundamentals S15).

C2: Atomic Shutdown Flag. The shutdown state must be visible to all threads immediately after it's set. A worker must never miss a shutdown signal and block forever waiting for a task.

C3: No Deadlocks. The design must be deadlock-free. Workers waiting on an empty queue must be wake-able by shutdown signals. The future.get() call must not deadlock if the worker executing that task crashes.

C4: No Lost Wake-Ups. When a task is enqueued, at least one waiting worker must be woken. When shutdown is signaled, all waiting workers must be woken. Lost wake-ups cause workers to sleep forever.

C5: Thread-Safe Future. future.get() may be called from any thread. The result must be safely published from the worker thread that computed it to the caller thread that reads it. This requires proper synchronization (lock + condition or volatile in Java).

Several non-obvious decisions shape the design. Let's justify each one:

D1: Fixed vs. Dynamic Pool Size

We chose a fixed pool. Java's ThreadPoolExecutor supports dynamic sizing (core pool → max pool with keep-alive timeout), but the dynamic logic adds complexity that obscures the core concurrency concepts. A fixed pool is what Executors.newFixedThreadPool() creates — the most commonly used configuration in practice.

D2: Bounded vs. Unbounded Queue

We chose a bounded queue with rejection policies. An unbounded queue never rejects tasks but risks OutOfMemoryError if tasks arrive faster than workers can process them. A bounded queue forces the designer to think about back-pressure — what happens when the system is overloaded? This is a critical production concern.

D3: Workers Created at Construction vs. On-Demand

We create all workers eagerly at construction. An alternative is lazy creation (start a new worker when a task arrives, up to the max). Eager creation is simpler, avoids synchronization around "should I create a new worker?", and matches the "pool" metaphor — the threads are pre-allocated.

D4: Separate Python and Java Implementations

Python's GIL (Global Interpreter Lock) means threads don't truly run in parallel for CPU-bound work — but the pool is still useful for I/O-bound tasks (HTTP calls, file I/O, database queries) where threads block on I/O and release the GIL. Java has true multi-core parallelism. The design is identical; the primitives differ enough that combining them in one tutorial would be confusing.

D5: Why Not Use the Built-In Implementations?

Python has concurrent.futures.ThreadPoolExecutor and Java has java.util.concurrent.ThreadPoolExecutor. We're building from scratch to understand the internals: where locks go, why condition variables exist, how shutdown propagates. After building it manually, you'll understand what the standard library does and why.

In Scope:

• Fixed-size thread pool with configurable worker count
• Bounded task queue with configurable capacity
• Callable tasks that return results via Futures
• Three rejection policies (Abort, CallerRuns, Discard)
• Graceful shutdown (drain queue) and immediate shutdown (discard queue)
• await_termination() with timeout

Out of Scope:

• Dynamic pool resizing (min/max threads with keep-alive)
• Scheduled / delayed task execution (ScheduledThreadPoolExecutor)
• Work stealing across multiple queues
• Task priorities or dependency graphs
• Distributed task queues (Celery, RabbitMQ)
• Process pools (Python multiprocessing)

Each requirement maps directly to a class or method in our final design:

Requirement	Class / Method	Concurrency Concern
R1: Fixed workers	ThreadPool.__init__(pool_size)	Workers are Thread objects
R2: Task submission	ThreadPool.submit(task)	Must check shutdown flag + enqueue atomically
R3: Future results	TaskFuture class	Cross-thread result publication
R4: Bounded queue	Internal BlockingQueue	Multi-producer, multi-consumer
R5: FIFO ordering	Queue's natural ordering	—
R6: Rejection policy	RejectionPolicy enum + handler	Checked inside submit() under lock
R7: Graceful shutdown	ThreadPool.shutdown()	Sets flag, wakes all workers
R8: Immediate shutdown	ThreadPool.shutdown_now()	Clears queue, wakes all workers
R9: Await termination	ThreadPool.await_termination()	Joins all worker threads
R10: Worker loop	Worker.run()	Queue poll + shutdown check loop
C1-C5	Lock + Condition on queue and Future	See code tutorials

In a concurrency interview, spend the first 3-5 minutes establishing the scope:

• "Should the pool size be fixed or dynamic?" — Usually fixed for the interview.
• "Bounded or unbounded queue?" — Bounded reveals more about your understanding.
• "Do we need to return results, or is fire-and-forget acceptable?" — Futures are the richer design.
• "What should happen on overload?" — Opens the rejection policy discussion.
• "What are the shutdown semantics?" — Graceful vs. immediate is a critical distinction.

Asking these questions tells the interviewer you understand the design space. The answers constrain the problem to a tractable scope and prevent you from building unnecessary complexity.