Synchronization, Race Conditions & Deadlocks

Why synchronization matters#

Threads in a process share memory. When two threads read and write the same data at the same time, the result can depend on the exact timing — a race condition. These bugs are rare in testing and common in production, which makes them some of the hardest to find.

A race condition you can see#

Each deposit reads the balance, does some work, then writes the new value. If another thread runs in between, one update overwrites the other — a lost update. Real races depend on luck, so this demo uses a Barrier to force the unlucky timing every time:

import threading

balance = 0
everyone_has_read = threading.Barrier(10)

def deposit_unsafe(amount):
    global balance
    current = balance              # 1. read
    everyone_has_read.wait()       # 2. force the worst case: all threads read before any writes
    balance = current + amount     # 3. write — overwrites everyone else's update

threads = [threading.Thread(target=deposit_unsafe, args=(100,)) for _ in range(10)]
for t in threads: t.start()
for t in threads: t.join()
print("expected 1000, got", balance)

Output:

expected 1000, got 100

All ten threads read 0 before any of them wrote, so nine deposits vanished. The code between the read and the write is a critical section — only one thread should be inside it at a time.

Mutex (lock)#

A mutex (mutual exclusion lock) allows one thread at a time into a critical section.

import threading
import time

balance = 0
lock = threading.Lock()

def deposit_safe(amount):
    global balance
    with lock:                       # only one thread inside at a time
        current = balance
        time.sleep(0.001)
        balance = current + amount

threads = [threading.Thread(target=deposit_safe, args=(100,)) for _ in range(10)]
for t in threads: t.start()
for t in threads: t.join()
print("expected 1000, got", balance)

Output:

expected 1000, got 1000

Keep critical sections short — while one thread holds the lock, others wait.

The same problem appears in databases: two requests read a stock count of 1 and both sell the item. Fixes include SELECT … FOR UPDATE, atomic updates (UPDATE products SET stock = stock - 1 WHERE id = ? AND stock > 0) or optimistic locking with a version column.

Semaphores#

A semaphore holds a counter of available permits. acquire() takes a permit (waiting if none are left); release() returns it. A mutex is like a semaphore with one permit. Semaphores are perfect for limiting concurrency — for example, at most 3 simultaneous database connections:

import threading
import time

pool = threading.Semaphore(3)
active = 0
peak = 0
counter_lock = threading.Lock()

def query():
    global active, peak
    with pool:                          # wait for a free "connection"
        with counter_lock:
            active += 1
            peak = max(peak, active)
        time.sleep(0.05)                # pretend to run a query
        with counter_lock:
            active -= 1

threads = [threading.Thread(target=query) for _ in range(10)]
for t in threads: t.start()
for t in threads: t.join()
print("max concurrent queries:", peak)

Output:

max concurrent queries: 3

The producer–consumer problem#

Producers create work and consumers process it through a shared bounded buffer. Producers must wait when the buffer is full; consumers must wait when it's empty. Python's queue.Queue implements this with locks and condition variables internally:

import queue
import threading

buffer = queue.Queue(maxsize=2)       # bounded: producer blocks when full

def producer():
    for i in range(1, 6):
        buffer.put(f"job-{i}")        # waits while the buffer is full
    buffer.put(None)                  # sentinel: no more work

def consumer():
    while (job := buffer.get()) is not None:
        print("processed", job)

threading.Thread(target=producer).start()
consumer()

Output:

processed job-1
processed job-2
processed job-3
processed job-4
processed job-5

This is the same pattern as a job queue (Redis, RabbitMQ, Laravel queues) feeding background workers.

Deadlocks#

A deadlock happens when threads wait for each other forever. Classic example: thread A holds lock 1 and wants lock 2, while thread B holds lock 2 and wants lock 1.

Thread A: lock(accounts[1]) ──▶ wants accounts[2] ──┐
                                                     │  each waits for the other forever
Thread B: lock(accounts[2]) ──▶ wants accounts[1] ──┘

The four Coffman conditions#

A deadlock can occur only if all four hold:

Break any one condition and deadlock becomes impossible.

Preventing deadlocks in practice#

1. Lock ordering — always acquire locks in a global order (e.g. by account ID). This breaks circular wait and is the most common fix.
2. Timeouts — lock.acquire(timeout=1); back off and retry if it fails.
3. Acquire everything at once or hold fewer locks.
4. Let the database detect it — PostgreSQL and MySQL detect deadlocks and abort one transaction; your code should retry.

import threading

locks = {1: threading.Lock(), 2: threading.Lock()}
balances = {1: 500, 2: 500}

def transfer(src, dst, amount):
    first, second = sorted((src, dst))       # same order for every thread → no circular wait
    with locks[first], locks[second]:
        balances[src] -= amount
        balances[dst] += amount

threads = [threading.Thread(target=transfer, args=(1, 2, 10)) for _ in range(50)]
threads += [threading.Thread(target=transfer, args=(2, 1, 10)) for _ in range(50)]
for t in threads: t.start()
for t in threads: t.join()
print(balances, "total:", sum(balances.values()))

Output:

{1: 500, 2: 500} total: 1000

One hundred concurrent transfers in opposite directions, no deadlock, and no money created or lost.

Related problems#

• Livelock — threads keep reacting to each other and retrying, but make no progress.
• Starvation — a thread never gets the resource because others keep winning.
• Priority inversion — a high-priority thread waits for a lock held by a low-priority thread.

Common mistakes#

• Check-then-act without a lock (if not exists: create) — use atomic operations or unique constraints.
• Holding locks during slow I/O such as network calls.
• Inconsistent lock ordering across code paths.
• Assuming Python's GIL makes code thread-safe — it doesn't protect multi-step operations.

Key takeaways#

• Race conditions come from unsynchronised access to shared state.
• Mutexes protect critical sections; semaphores limit concurrency.
• Producer–consumer with a bounded queue is the basis of job queues.
• Deadlock needs all four Coffman conditions; consistent lock ordering prevents it.