Errors, exceptions, and the logging module

Why this matters

Real software fails — networks drop, disks fill, users send garbage. Python's exception model is the cleanest in any mainstream language; mastering try / except / else / finally, exception chaining, custom exception types, and the logging module is the difference between a tool that prints "oops" and one that ships with confidence.

Learning objectives

Use the try / except / else / finally block correctly.
Define custom exceptions with hierarchies.
Chain exceptions with raise ... from ....
Use exception groups (3.11+).
Configure logging for libraries and applications.

1. The `try` block

python

try:
    risky()
except FileNotFoundError:
    handle_missing()
except (PermissionError, IsADirectoryError) as e:
    log(e)
except OSError:
    fallback()
else:
    # ran only if no exception
    commit()
finally:
    # ALWAYS runs (cleanup)
    close_handle()

Rules of thumb:

Catch the narrowest exception that matches your handling.
except Exception: is rarely right. Never except: (catches even KeyboardInterrupt).
Put as little code as possible inside try so you know exactly which line could throw.
else is for "the successful path." It separates "what might fail" from "what runs after success."
finally is for cleanup that must run — but prefer with (context managers) where possible.

2. The exception hierarchy

BaseException
 ├── SystemExit
 ├── KeyboardInterrupt
 └── Exception                  <-- you almost always catch from here down
      ├── ArithmeticError
      │    ├── ZeroDivisionError
      │    └── OverflowError
      ├── LookupError
      │    ├── IndexError
      │    └── KeyError
      ├── OSError               <-- IOError merged into this
      │    ├── FileNotFoundError
      │    ├── PermissionError
      │    └── TimeoutError
      ├── TypeError
      ├── ValueError
      ├── AttributeError
      ├── RuntimeError
      │    └── RecursionError
      └── StopIteration

BaseException includes things you almost never want to catch (e.g., KeyboardInterrupt). Always catch Exception or a more specific subclass.

3. Raising exceptions

python

raise ValueError("expected positive number")
raise ValueError(f"got {x!r}")
raise FileNotFoundError(path)

# Re-raise unchanged
try:
    op()
except ValueError:
    log()
    raise                       # bare raise — preserves traceback

Don't use exceptions for normal control flow (it's slow and confusing) — except for StopIteration (built into iteration), and for EAFP lookups where the "miss" case is rare.

4. Custom exceptions

Always inherit from Exception (or a more specific one):

python

class AppError(Exception):
    """Base for all application errors."""

class ConfigError(AppError):
    """Bad or missing configuration."""

class APIError(AppError):
    def __init__(self, status: int, body: str):
        super().__init__(f"API {status}: {body}")
        self.status = status
        self.body = body

Benefits:

Callers can except APIError: without catching every error.
You attach structured data (e.status, e.body).
Logs and metrics can group by type.

Group your exceptions under a single base class per package so users can except mypkg.MyPkgError and catch everything from your code.

5. Exception chaining (`raise ... from ...`)

When you catch one exception and raise another, preserve the cause:

python

try:
    parse(raw)
except json.JSONDecodeError as e:
    raise ConfigError("bad config file") from e

Output:

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

ConfigError: bad config file

Use from None to suppress the cause when it's noise:

python

raise ConfigError("bad config file") from None

6. Exception groups (PEP 654, 3.11+)

For concurrent / batch operations, you may want to raise multiple exceptions at once:

python

errs = []
for item in items:
    try:
        process(item)
    except Exception as e:
        errs.append(e)
if errs:
    raise ExceptionGroup("processing failed", errs)

Handle with except*:

python

try:
    do_concurrent_work()
except* APIError as eg:
    for e in eg.exceptions:
        log(e)
except* (TimeoutError, ConnectionError) as eg:
    retry(eg.exceptions)

asyncio.TaskGroup (Phase 4.4) raises an ExceptionGroup when any of its tasks fail — except* is how you handle them.

7. Context managers as cleanup tools

finally is fine, but with is cleaner:

python

# Bad:
f = open("x.txt")
try:
    data = f.read()
finally:
    f.close()

# Good:
with open("x.txt") as f:
    data = f.read()

contextlib.suppress for "ignore this specific exception":

python

from contextlib import suppress
with suppress(FileNotFoundError):
    Path("tmp.txt").unlink()

Phase 3.3 covers context managers in depth.

8. Logging — never `print` in production

print is fine for scripts and notebooks. For libraries and services, use logging.

Setup (in your application's entry point)

python

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)-8s %(name)s: %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S",
)

Use a logger per module

python

# anywhere in your code
import logging
log = logging.getLogger(__name__)        # name = module path

log.debug("internal: x=%s", x)
log.info("started")
log.warning("retrying %d/%d", attempt, max_attempts)
log.error("failed: %s", err)
log.exception("crashed")                  # includes traceback (use inside except)
log.critical("dying")

Use %s placeholders, not f-strings — values are formatted lazily, only if the level is enabled.

Levels (in order)

DEBUG < INFO < WARNING < ERROR < CRITICAL

Default level is WARNING. Loggers filter by their effective level.

Library vs application loggers

Libraries: never call basicConfig. Just log = logging.getLogger(__name__) and emit. Let the application configure.

Applications: call basicConfig (or use logging.config.dictConfig for fine control) once at startup.

Structured / JSON logging

For services, use structured logs so they're greppable / queryable by tools (Datadog, Loki, ELK):

python

# Using structlog (third-party)
import structlog
log = structlog.get_logger()
log.info("request_completed", user_id=42, latency_ms=120, status=200)

Output (JSON renderer):

json

{"event": "request_completed", "user_id": 42, "latency_ms": 120, "status": 200, "level": "info", "timestamp": "..."}

structlog (uv add structlog) is the standard in 2026 for production Python apps.

Don't log secrets

Mask Authorization, API keys, passwords, PII. Have a "redactor" middleware. Failures here become CVEs.

9. Warnings — for deprecation and "you should know"

warnings.warn is the right channel for things that aren't errors but the user should hear:

python

import warnings

def old_api():
    warnings.warn("old_api() is deprecated; use new_api()", DeprecationWarning, stacklevel=2)

# As library author
warnings.warn("expensive default; pass explicit value", UserWarning)

Run with python -W error to turn warnings into exceptions (helpful in tests / CI).

10. `assert` — for invariants, not validation

python

def divide(a, b):
    assert b != 0, "divisor must be non-zero"
    return a / b

assert is stripped when Python runs with -O. So:

Use for invariants you believe should be true ("if this is false, the code is broken").
Don't use for input validation — raise ValueError instead.
Don't use in tests (use pytest's assert, which rewrites the message — Phase 6.1).

11. Putting it together — robust function pattern

python

import logging
import httpx

log = logging.getLogger(__name__)

class FetchError(Exception):
    def __init__(self, url: str, cause: Exception):
        super().__init__(f"failed to fetch {url}: {cause}")
        self.url = url
        self.cause = cause

def fetch_json(url: str, *, timeout: float = 10) -> dict:
    try:
        r = httpx.get(url, timeout=timeout)
        r.raise_for_status()
        return r.json()
    except httpx.HTTPStatusError as e:
        log.warning("non-2xx from %s: %s", url, e.response.status_code)
        raise FetchError(url, e) from e
    except httpx.HTTPError as e:
        log.warning("network error fetching %s: %s", url, e)
        raise FetchError(url, e) from e
    except ValueError as e:                   # invalid JSON
        log.warning("invalid JSON from %s", url)
        raise FetchError(url, e) from e

Note:

Custom exception with structured data.
Chained with from e (preserves traceback).
Logged with appropriate level before re-raising (caller may not log).
Narrow except clauses; broad Exception would mask bugs.

Hands-on lab (1.5 hours)

Define a small exception hierarchy for a fake MyAPI client.
Wrap json.loads in a function that re-raises as your ConfigError with chaining.
Build a safe_divide(a, b) that returns None on ZeroDivisionError, logs at WARNING.
Configure logging.basicConfig; emit logs at every level; observe what shows.
Write a small async function that gathers 10 fetches and raises an ExceptionGroup for failures.
Add structlog to a script; emit one event with three fields; pipe through | jq (or Get-Content on Windows) to filter.
Use warnings.warn to deprecate a function; verify with python -W error.

Common pitfalls

except Exception as e: pass — silently swallows bugs. Always log.
except: (bare) — catches KeyboardInterrupt; never use.
Using assertions for input validation.
f-strings inside log calls (log.info(f"...")): formats eagerly. Use log.info("x=%s", x).
Calling logging.basicConfig in a library.
Raising naked strings: raise "bad" is a TypeError since Python 3. Raise instances.
Forgetting from e and losing the cause.

Self-check

What does else: mean in a try block?
Why prefer with over try/finally?
raise X from Y — what does it do?
What is an ExceptionGroup?
Why use %s placeholders in logs, not f-strings?

References

Effective Python, Slatkin — Items on errors and logging.
PEP 654 — Exception Groups and except*.
Python docs, logging HOWTO.
structlog documentation: https://www.structlog.org/.
Hynek Schlawack, "Structured logging in Python."

Why this matters

Learning objectives

1. The try block

2. The exception hierarchy

3. Raising exceptions

4. Custom exceptions

5. Exception chaining (raise ... from ...)

6. Exception groups (PEP 654, 3.11+)

7. Context managers as cleanup tools

8. Logging — never print in production

Setup (in your application's entry point)

Use a logger per module

Levels (in order)

Library vs application loggers

Structured / JSON logging

Don't log secrets

9. Warnings — for deprecation and "you should know"

10. assert — for invariants, not validation

11. Putting it together — robust function pattern

Hands-on lab (1.5 hours)

Common pitfalls

Self-check

References

1. The `try` block

5. Exception chaining (`raise ... from ...`)

8. Logging — never `print` in production

10. `assert` — for invariants, not validation