++++In modern system design, Concurrency and Parallelism are distinct ways of managing multiple tasks. Understanding these is essential for building responsive and high-performance applications.
Concurrency & Parallelism 🧵
This content is adapted from Mastering System Design from Basics to Cracking Interviews (Udemy). It has been curated and organized for educational purposes on this portfolio. No copyright infringement is intended.
🚀 Concurrency vs. Parallelism
- Concurrency: The ability of a system to handle multiple tasks at once by managing context switching. Even on a single-core CPU, a system can be concurrent by interleaving task progress.
- Goal: Responsiveness.
- Parallelism: Executing multiple tasks simultaneously, typically across multiple CPU cores.
- Goal: Speed and Throughput.
🏗️ Processes vs. Threads
A Process is an isolated execution environment offered by the OS, while a Thread is a unit of execution within a process.
| Feature | Processes | Threads |
|---|---|---|
| Memory | Isolated (Own space) | Shared within the process |
| Creation | Heavier/Slower | Lightweight/Faster |
| Risk | Safer (Crashes are isolated) | Higher risk (Race conditions) |
🛠️ Thread Pools & Worker Models
Instead of creating a new thread for every task (which is expensive), we use a Thread Pool.
- Thread Pool: A collection of pre-created, reusable threads.
- Worker Model: Tasks are queued, and idle threads (workers) pick them up for processing. This improves scalability and prevents resource exhaustion.
⏳ Asynchronous Processing
Asynchronous programming avoids blocking threads during I/O operations (like database calls or network requests).
- Techniques:
async/await, Promises, Futures. - Benefit: Improved throughput by allowing a single thread to handle many concurrent connections while waiting for I/O.
🖥️ Concurrency in Web Servers
- Traditional (e.g., Apache): Spawns a new thread/process per request. Hard to scale for thousands of concurrent users.
- Modern (e.g., Node.js, Nginx): Uses an Event Loop and non-blocking I/O. A small thread pool handles thousands of connections efficiently.
⚠️ Common Pitfalls
- Race Conditions: Occur when multiple threads access and modify shared data concurrently, leading to unpredictable results.
- Fix: Use locks, mutexes, or atomic operations.
- Deadlocks: A situation where two or more threads are stuck waiting for each other to release resources.
- Fix: Acquire locks in a consistent order or use timeouts.
Interview Questions & Answers 💡
1. What is the difference between concurrency and parallelism?
Concurrency is about managing multiple tasks (even by interleaving them), focusing on responsiveness. Parallelism is about executing tasks simultaneously, typically using multiple cores for speed.
- Concurrency: Dealing with many things at once. - Parallelism: Doing many things at once.
2. Why use a thread pool instead of raw threads?
Thread pools reduce the overhead of creating and destroying threads and prevent system exhaustion by limiting the total number of concurrent threads.
3. How do you prevent race conditions?
Race conditions occur when threads compete for shared resources without coordination.
- Synchronization Mechanisms: Locks, mutexes, or semaphores. - Atomic Operations: Operations that complete as a single unit. - Thread-safe Data Structures: Built-in collections that handle concurrency.
4. How would you handle thousands of concurrent requests in a web server?
Handling massive concurrency requires moving away from the "thread-per-request" model.
- Non-blocking I/O: Use an event loop (Node.js) or async/await (.NET). - Connection Pooling: Reuse existing database connections. - Load Balancer: Distribute traffic across a pool of servers.
5. What is a Deadlock and how can it be resolved?
A deadlock is a circular wait where threads are stuck waiting for each other's resources.
- Global Lock Ordering: Always acquire locks in the same order. - Timeouts: Don't wait indefinitely for a resource. - Lock Scope: Keep synchronized blocks as small as possible.
Final Thoughts
Concurrency is for responsiveness; parallelism is for speed. Prefer async I/O for I/O-bound tasks and thread pools for CPU-bound tasks. Always guard your shared data!
Next up? Deep dive into performance at scale — Performance Measurement: SLAs & SLOs