Concurrency & Parallelism 🧵
In modern system design, the terms Concurrency and Parallelism are often used interchangeably, but they represent two distinct ways of managing multiple tasks. Understanding these concepts is essential for building responsive and high-performance applications.
This content is adapted from Mastering System Design from Basics to Cracking Interviews (Udemy). It has been curated and organized for educational purposes on this portfolio. No copyright infringement is intended.
🚀 Concurrency vs. Parallelism
- Concurrency: The ability of a system to handle multiple tasks at once by managing context switching. Even on a single-core CPU, a system can be concurrent by interleaving task progress.
- Goal: Responsiveness.
- Parallelism: Executing multiple tasks simultaneously, typically across multiple CPU cores.
- Goal: Speed and Throughput.
🏗️ Processes vs. Threads
A Process is an isolated execution environment offered by the OS, while a Thread is a unit of execution within a process.
| Feature | Processes | Threads |
|---|---|---|
| Memory | Isolated (Own space) | Shared within the process |
| Creation | Heavier/Slower | Lightweight/Faster |
| Risk | Safer (Crashes are isolated) | Higher risk (Race conditions) |
🛠️ Thread Pools & Worker Models
Instead of creating a new thread for every task (which is expensive), we use a Thread Pool.
- Thread Pool: A collection of pre-created, reusable threads.
- Worker Model: Tasks are queued, and idle threads (workers) pick them up for processing. This improves scalability and prevents resource exhaustion.
⏳ Asynchronous Processing
Asynchronous programming avoids blocking threads during I/O operations (like database calls or network requests).
- Techniques:
async/await, Promises, Futures. - Benefit: Improved throughput by allowing a single thread to handle many concurrent connections while waiting for I/O.
🖥️ Concurrency in Web Servers
- Traditional (e.g., Apache): Spawns a new thread/process per request. Hard to scale for thousands of concurrent users.
- Modern (e.g., Node.js, Nginx): Uses an Event Loop and non-blocking I/O. A small thread pool handles thousands of connections efficiently.
⚠️ Common Pitfalls
- Race Conditions: Occur when multiple threads access and modify shared data concurrently, leading to unpredictable results.
- Fix: Use locks, mutexes, or atomic operations.
- Deadlocks: A situation where two or more threads are stuck waiting for each other to release resources.
- Fix: Acquire locks in a consistent order or use timeouts.
Interview Questions - Concurrency & Parallelism 💡
1. What is the difference between concurrency and parallelism?
Answer: Concurrency is about managing multiple tasks (even by interleaving them), focusing on responsiveness. Parallelism is about executing tasks simultaneously, typically using multiple cores for speed.
2. Why use a thread pool instead of raw threads?
Answer: Thread pools reduce the overhead of creating/destroying threads and prevent system exhaustion by limiting the total number of concurrent threads.
3. How do you prevent race conditions?
Answer: Use synchronization mechanisms like locks, mutexes, or semaphores. Alternatively, use thread-safe data structures or keep operations atomic.
4. How would you handle thousands of concurrent requests in a web server?
Answer: Use Asynchronous non-blocking I/O (like Node.js event loop or .NET async/await), connection pooling, and a load balancer to distribute traffic.
5. What is a Deadlock and how can it be resolved?
Answer: A deadlock is a circular wait where threads are stuck waiting for each other's resources. Resolve it by ensuring a strict global lock ordering, using timeouts, or minimizing lock scope.
Summary: Concurrency is for responsiveness; parallelism is for speed. Prefer async I/O for I/O-bound tasks and thread pools for CPU-bound tasks. Always guard your shared data!
Next up? Deep dive into performance at scale — Performance Measurement: SLAs & SLOs