Concurrency: threads, processes, and the GIL

Why this matters

Python has four concurrency primitives — threads, processes, asyncio, and the experimental free-threaded build. The right choice depends on whether your work is I/O-bound or CPU-bound, and how much state you share. Knowing this stops "why isn't my parallel code faster?" hour-long debugging sessions.

Learning objectives

1. Apply the threads vs processes vs asyncio decision rule.
2. Use concurrent.futures.ThreadPoolExecutor and ProcessPoolExecutor.
3. Use threading primitives (Lock, Event, Queue).
4. Use multiprocessing correctly (start methods, pickling).
5. Understand the GIL and the 3.13 free-threaded build.

1. The decision rule

                              Is your work I/O-bound or CPU-bound?

           I/O-bound (network, disk, DB)              CPU-bound (numbers, parsing, compression)
                       │                                          │
                       ▼                                          ▼
        Lots of concurrent requests?              In NumPy/PyTorch/native already?
              /              \                          /                  \
            yes              no                       yes                   no
            ▼                ▼                        ▼                     ▼
         asyncio         threads               threads OK              ProcessPool
                                          (GIL releases in C)         or free-threaded build

2. The GIL in one paragraph

CPython holds a global lock around the interpreter; only one thread executes Python bytecode at a time. The lock is released:

• Around blocking I/O (read/write/recv/sleep).
• Inside C extensions that explicitly release it (NumPy, PyTorch, requests via httpx → libcurl, hashlib).

So threads parallelise I/O and C-level CPU work but not pure-Python CPU work. The free-threaded build (3.13+) removes the GIL at a small single-threaded cost; it's behind a --disable-gil compile flag for now.

3. concurrent.futures — your default concurrency API

Two pools, same API:

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

# I/O-bound: threads
def fetch(url): ...

with ThreadPoolExecutor(max_workers=20) as ex:
    results = list(ex.map(fetch, urls))

# CPU-bound: processes
def heavy(x): ...

with ProcessPoolExecutor() as ex:
    results = list(ex.map(heavy, items, chunksize=100))

submit + as_completed

from concurrent.futures import as_completed

with ThreadPoolExecutor(max_workers=10) as ex:
    futures = {ex.submit(fetch, url): url for url in urls}
    for fut in as_completed(futures):
        url = futures[fut]
        try:
            data = fut.result()
        except Exception as e:
            log.warning("%s failed: %s", url, e)
        else:
            process(data)

Use submit when:

• Tasks are heterogeneous.
• You want to handle exceptions per-task.
• You need results as they arrive.

map for simple "apply f to each, collect results in order."

chunksize for ProcessPool

Pickling overhead matters. With many small tasks, batch them:

ex.map(heavy, items, chunksize=100)

4. threading — primitives

import threading

# Start a thread
t = threading.Thread(target=worker, args=(arg,), daemon=True)
t.start()
t.join(timeout=10)
t.is_alive()

# Lock — mutual exclusion
lock = threading.Lock()
with lock:
    shared_counter += 1

# RLock — reentrant; same thread can acquire multiple times
rlock = threading.RLock()

# Event — flag with wait/set
ready = threading.Event()
ready.wait(timeout=5)         # block until set
ready.set(); ready.clear()

# Condition — wait/notify
cv = threading.Condition()
with cv:
    cv.wait_for(predicate)
    cv.notify_all()

# Semaphore — limit concurrent access
sem = threading.Semaphore(value=5)
with sem:
    do_thing()

# Thread-local storage
local = threading.local()
local.x = 1                   # only this thread sees this

For most code use concurrent.futures. Drop to threading when you need fine-grained control.

queue.Queue — producer/consumer

import queue
from threading import Thread

q: queue.Queue[int] = queue.Queue(maxsize=100)

def producer():
    for i in range(1000):
        q.put(i)
    q.put(None)              # sentinel

def consumer():
    while True:
        item = q.get()
        if item is None: break
        process(item)
        q.task_done()

t1 = Thread(target=producer); t2 = Thread(target=consumer)
t1.start(); t2.start()
t1.join(); t2.join()
q.join()                     # wait until all items processed

Queue is thread-safe — the canonical hand-off between threads.

5. multiprocessing — for CPU parallelism

import multiprocessing as mp

def heavy(x): return x * x

if __name__ == "__main__":
    with mp.Pool(processes=mp.cpu_count()) as p:
        results = p.map(heavy, range(1_000_000), chunksize=10_000)

The if __name__ == "__main__": guard is mandatory on Windows/macOS — the workers re-import the module, and without the guard you'd recursively spawn processes.

Start methods

mp.set_start_method("spawn")    # default on Windows + macOS (since 3.14 also Linux)
mp.set_start_method("fork")     # Linux default historically; faster but unsafe with threads
mp.set_start_method("forkserver")    # middle ground

spawn re-imports your module — slow but safe. fork clones memory — fast but breaks if the parent had threads or sockets.

Sharing data

Processes don't share memory. Options:

• mp.Queue, mp.Pipe — pickled messages.
• mp.Manager — proxied dicts/lists (slow).
• mp.shared_memory (3.8+) — raw shared bytes (fast, manual).
• numpy arrays in shared memory.
• Ray / Dask for higher-level distributed objects.

For most "map this function over a million items," ProcessPoolExecutor.map(..., chunksize=N) is enough.

Pickling caveats

Anything sent to a worker process must be picklable. Common offenders:

• Lambdas (use named functions or functools.partial).
• Open file/socket handles.
• Locks.
• Closures over un-picklable values.

If you get TypeError: cannot pickle 'X', refactor.

6. Threads vs Processes — measurement, not vibes

import time
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

def cpu(n):
    s = 0
    for i in range(n):
        s += i * i
    return s

N = 10_000_000
items = [N] * 8

t0 = time.perf_counter()
[cpu(N) for _ in items]
print("sequential:", time.perf_counter() - t0)

t0 = time.perf_counter()
with ThreadPoolExecutor(max_workers=8) as ex:
    list(ex.map(cpu, items))
print("threads:", time.perf_counter() - t0)            # not faster (GIL)

t0 = time.perf_counter()
with ProcessPoolExecutor(max_workers=8) as ex:
    list(ex.map(cpu, items))
print("processes:", time.perf_counter() - t0)          # ~4-8x faster

I/O-bound? Swap cpu for time.sleep(0.5) — threads win.

7. Free-threaded CPython (3.13+)

Compile with --disable-gil; binaries marked *t. Single-threaded code is ~5-10% slower; pure-Python multithreaded CPU code can scale near-linearly with cores.

import sys
sys._is_gil_enabled()                      # False on free-threaded

State in 2026:

• Many top libraries (NumPy, PyTorch, requests, FastAPI) work or have patches.
• Some C extensions may crash; some fall back to a per-extension GIL.
• Slowly being made the default; in 3.15 expected as opt-out, not opt-in.

For production today: still niche. Use processes or asyncio for parallelism.

8. Patterns

Pool of workers consuming a queue

from queue import Queue
from threading import Thread

q = Queue(maxsize=200)

def worker():
    while True:
        item = q.get()
        try:
            process(item)
        finally:
            q.task_done()

for _ in range(8):
    Thread(target=worker, daemon=True).start()

for item in produce():
    q.put(item)
q.join()                         # block until all items processed

Producer/consumer with backpressure

Queue(maxsize=N) blocks the producer when full → automatic backpressure.

"Fire and forget" with futures

futures = []
with ThreadPoolExecutor() as ex:
    for url in urls:
        futures.append(ex.submit(fetch, url))
# exiting `with` waits for all futures

Timeout on a future

fut = ex.submit(slow)
try:
    fut.result(timeout=5)
except TimeoutError:
    log.warning("slow")

fut.cancel() only works if the task hasn't started yet — Python can't interrupt running threads.

9. Common bugs in concurrent code

1. Race condition: two threads update a shared variable; result depends on timing. Use a lock or atomic operation.
2. Deadlock: thread A holds lock 1, waits for lock 2; thread B vice versa. Always acquire locks in the same order; use RLock when reentrancy is needed.
3. Forgotten daemon flag: non-daemon threads keep the program alive.
4. Shared mutable state without locks → corruption.
5. Pickling failure in multiprocessing.
6. CPU-bound code in ThreadPool → no speedup, surprise.
7. asyncio and threads mixed naively → callbacks scheduled on wrong loops.

10. Logging in concurrent code

import logging
logging.basicConfig(format="%(asctime)s %(threadName)s %(levelname)s %(message)s")

%(threadName)s / %(processName)s add the worker identity to every line — invaluable for debugging.

For processes, use logging.handlers.QueueHandler + QueueListener to centralise logs to one writer.

11. contextvars — thread/async-safe context

Per-task (or per-thread) variables that propagate through async tasks:

import contextvars

request_id: contextvars.ContextVar[str] = contextvars.ContextVar("request_id", default="")

def log_event(msg):
    print(f"[{request_id.get()}] {msg}")

request_id.set("req-42")
log_event("hi")          # [req-42] hi

Used internally by asyncio, FastAPI, structlog to carry context across awaits without globals.

12. Worked example: download a list of URLs

from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path
import httpx
import logging

log = logging.getLogger(__name__)

def download(url: str, out_dir: Path) -> Path:
    name = url.split("/")[-1] or "index.html"
    target = out_dir / name
    with httpx.Client(timeout=30) as client:
        r = client.get(url)
        r.raise_for_status()
    target.write_bytes(r.content)
    return target

def download_many(urls: list[str], out_dir: Path, *, workers: int = 16) -> list[Path]:
    out_dir.mkdir(parents=True, exist_ok=True)
    saved: list[Path] = []
    with ThreadPoolExecutor(max_workers=workers) as ex:
        futures = {ex.submit(download, u, out_dir): u for u in urls}
        for fut in as_completed(futures):
            url = futures[fut]
            try:
                saved.append(fut.result())
            except Exception as e:
                log.warning("failed %s: %s", url, e)
    return saved

For thousands of concurrent fetches, switch to asyncio + httpx.AsyncClient (Phase 4.4).

Hands-on lab (2 hours)

1. Time time.sleep(0.1) × 20 sequential, threaded, processed; compare.
2. Time a pure-Python CPU loop the same way; observe threads don't help.
3. Convert your download_many to use ProcessPoolExecutor; observe slowness from pickling.
4. Write a producer/consumer with queue.Queue and 4 worker threads; print stats.
5. Trigger a deadlock with two locks; fix by acquiring in consistent order.
6. Use mp.Pool with chunksize=100 vs chunksize=1; measure overhead.
7. Bonus: install python3.13t (free-threaded build) via uv python install 3.13t; rerun the CPU-loop test on threads.

Common pitfalls

1. CPU-bound code on threads → no speedup.
2. No if __name__ == "__main__": for multiprocessing on Windows/macOS.
3. Sharing mutable state across processes (it isn't shared).
4. Pickling lambdas / closures.
5. Catching Exception in worker but not re-raising; failures get swallowed.
6. Holding the GIL via tight pure-Python loops in I/O-bound paths (block other threads).

Self-check

1. Threads vs processes vs asyncio — pick one for each: 5000 HTTP fetches, image resizing of 1000 photos, parsing 100 large JSON files.
2. What does the GIL prevent?
3. Why does multiprocessing require if __name__ == "__main__": on Windows?
4. State two threading primitives besides Lock.
5. Why does ProcessPoolExecutor need chunksize?

References

• Effective Python, Slatkin — Items on concurrency.
• Python Concurrency with asyncio, Matthew Fowler.
• David Beazley, "Concurrency from the Ground Up" (PyCon talk).
• PEP 703 — Making the Global Interpreter Lock Optional.
• concurrent.futures docs.