Descriptors and metaclasses — power features, used sparingly — Python Mastery

Why this matters

Descriptors and metaclasses are Python's two most powerful — and most over-used — meta-programming tools. They power @property, @classmethod, @dataclass, Django ORM fields, SQLAlchemy columns, Pydantic models. You probably won't write your own metaclass in 99% of jobs, but you must read them when they appear, and you'll occasionally write a descriptor.

Learning objectives

Explain what a descriptor is and how it intercepts attribute access.
Write data and non-data descriptors.
Use __set_name__ for clean field-style APIs.
Explain what a metaclass is and how type works.
Know when to use a class decorator or __init_subclass__ instead.

1. Descriptors

A descriptor is an object with at least one of:

__get__(self, instance, owner)
__set__(self, instance, value)
__delete__(self, instance)

It must be assigned at class level (not instance level) to be triggered.

python

class Const:
    def __init__(self, value): self.value = value
    def __get__(self, instance, owner): return self.value

class Demo:
    PI = Const(3.14)

Demo.PI                          # 3.14
Demo().PI                        # 3.14

Data vs non-data

Data descriptor: defines __set__ (or __delete__). Takes priority over instance __dict__.
Non-data descriptor: only __get__. Loses to instance __dict__.

python

class NonData:
    def __get__(self, instance, owner): return "from descriptor"

class D:
    x = NonData()

d = D()
d.x                              # "from descriptor"
d.__dict__["x"] = "from instance"
d.x                              # "from instance"  (instance wins)

For data descriptors, instance assignment never wins — they're the basis of @property.

2. `@property` is a descriptor

python

class C:
    @property
    def x(self): return self._x

property is a class implementing __get__ and __set__. The decorator returns a property instance. That's all the magic.

3. Typed-attribute descriptors

The classic example: enforce types or constraints.

python

class Typed:
    def __init__(self, kind):
        self.kind = kind

    def __set_name__(self, owner, name):
        self.name = name             # remember the attribute name

    def __get__(self, instance, owner):
        return instance.__dict__[self.name]

    def __set__(self, instance, value):
        if not isinstance(value, self.kind):
            raise TypeError(f"{self.name} must be {self.kind.__name__}")
        instance.__dict__[self.name] = value


class Person:
    name = Typed(str)
    age = Typed(int)

    def __init__(self, name, age):
        self.name = name             # triggers Typed.__set__
        self.age = age

p = Person("Ada", 30)
p.age = "thirty"                     # TypeError

`__set_name__` (PEP 487, 3.6+)

Called automatically when the class is created, with the attribute name. Without it, you'd need a metaclass or boilerplate to know what to call the field.

Where to store the value

We stored on instance.__dict__[self.name] — same dict as ordinary attributes. Don't store on self (the descriptor instance) — it's a class attribute, shared by all instances.

4. Why dataclasses / Pydantic still beat hand-rolled descriptors

For 99% of "I want validated fields," don't write descriptors. Use:

@dataclass + __post_init__ for validation.
pydantic.BaseModel for parsed + validated models with rich error messages.

Hand-rolled descriptors are right when you're building a library that exposes field-like APIs to users (Django Fields, SQLAlchemy columns, Marshmallow schemas).

5. The descriptor lookup chain

When you write instance.x, Python:

Looks up x on type(instance)'s MRO. If it's a data descriptor → call __get__.
Looks up x in instance.__dict__. Hit → return value.
Looks up x on type(instance)'s MRO. If it's a non-data descriptor → call __get__.
If it's a regular class attribute → return it.
Otherwise call __getattr__ if defined.
Raise AttributeError.

This is the attribute lookup algorithm. It explains why methods (non-data descriptors via the function type) work, why @property overrides instance assignment, and how __slots__ integrates.

6. Methods are descriptors

Functions implement __get__, which binds self:

python

class C:
    def m(self): return "hi"

C.m                              # <function C.m at 0x...>
c = C()
c.m                              # <bound method C.m of <C object ...>>
c.m()                            # "hi"

c.m triggers function.__get__(c, C) which returns a bound method. This is why a method "knows" its self.

7. Metaclasses

"Metaclasses are deeper magic than 99% of users should ever worry about." — Tim Peters

A class is an instance of a metaclass. The default metaclass is type.

python

class Foo: pass

type(Foo)                        # <class 'type'>
isinstance(Foo, type)            # True

You can create classes dynamically:

python

Foo = type("Foo", (object,), {"x": 1, "greet": lambda self: "hi"})
Foo().greet()                    # "hi"

Custom metaclass

python

class TracingMeta(type):
    def __new__(mcs, name, bases, ns):
        print(f"Creating class {name}")
        return super().__new__(mcs, name, bases, ns)

class Foo(metaclass=TracingMeta):
    pass
# prints "Creating class Foo" at definition time

__new__ runs once per class creation. You can mutate ns (the class dict) before the class is built.

Practical (but rare) use cases

Auto-register all subclasses in a registry (now usually __init_subclass__).
Auto-create accessors / fields (Django ORM, SQLAlchemy declarative).
Enforce class-level invariants.

Stop using metaclasses if...

__init_subclass__ solves it.
A class decorator solves it (@dataclass).
You can express it with composition.

Metaclasses are sticky — they apply to all subclasses transitively and interfere with other metaclasses (multiple inheritance becomes a nightmare).

8. `__init_subclass__` — the modern alternative

Already covered (Lesson 2.1, 2.2). Recap:

python

class Plugin:
    registry = {}
    def __init_subclass__(cls, *, name, **kwargs):
        super().__init_subclass__(**kwargs)
        cls.registry[name] = cls

class CSV(Plugin, name="csv"): ...

90% of metaclass use cases collapse into __init_subclass__ + class decorators. Reach for them first.

9. `getattr` and `getattribute`

__getattr__(self, name): called only when normal lookup fails. Use for lazy / proxy / dispatch.
__getattribute__(self, name): called on every attribute access. Power tool; fragile.

python

class LazyConfig:
    def __init__(self, loader): self._loader = loader; self._cache = {}
    def __getattr__(self, name):                     # only if not found normally
        if name not in self._cache:
            self._cache[name] = self._loader(name)
        return self._cache[name]

__getattr__ doesn't intercept already-bound attributes — safe and common. __getattribute__ does — avoid unless you really need it.

10. Worked example: a tiny ORM field

python

class Field:
    def __init__(self, kind, *, default=None, nullable=False):
        self.kind = kind; self.default = default; self.nullable = nullable

    def __set_name__(self, owner, name):
        self.name = name
        if not hasattr(owner, "_fields"):
            owner._fields = {}
        owner._fields[name] = self

    def __get__(self, instance, owner):
        if instance is None: return self
        return instance.__dict__.get(self.name, self.default)

    def __set__(self, instance, value):
        if value is None:
            if not self.nullable: raise ValueError(f"{self.name} cannot be None")
        elif not isinstance(value, self.kind):
            raise TypeError(f"{self.name} must be {self.kind.__name__}")
        instance.__dict__[self.name] = value


class Model:
    @classmethod
    def from_dict(cls, d):
        obj = cls.__new__(cls)
        for name, field in cls._fields.items():
            setattr(obj, name, d.get(name, field.default))
        return obj

    def to_dict(self):
        return {name: getattr(self, name) for name in self._fields}


class User(Model):
    name = Field(str)
    age = Field(int, default=18)
    email = Field(str, nullable=True)

u = User.from_dict({"name": "Ada"})
u.age                            # 18
u.email                          # None
u.name = 5                       # TypeError

This is the kernel of Django Model / SQLAlchemy declarative / Pydantic v1 internals (Pydantic v2 has a Rust core and doesn't use Python descriptors for fields, but the conceptual API is similar).

Hands-on lab (2 hours)

Write a PositiveInt descriptor that raises on negative values; use it on a class.
Add __set_name__ so the error message names the attribute.
Convert your descriptor to a class decorator: @with_typed_fields(name=str, age=PositiveInt).
Write a tiny metaclass TracingMeta that prints each subclass creation; create three subclasses.
Refactor TracingMeta into __init_subclass__; compare.
Write a LazyAttr non-data descriptor that computes a value once and stores on the instance.
Bonus: implement a Pydantic-like Model.from_dict / to_dict (lighter version of the lesson's ORM example) and write tests.

Common pitfalls

Storing descriptor state on self (the descriptor) — shared across instances. Use instance.__dict__.
Forgetting __set_name__ and hand-encoding names in __init__.
Writing a metaclass when __init_subclass__ would do.
__getattribute__ instead of __getattr__ and breaking everything.
Combining two metaclasses via multiple inheritance — TypeError. Always have a single, deepest metaclass.

Self-check

Difference between data and non-data descriptors.
What does __set_name__ do?
Why is @property a descriptor?
State the attribute lookup chain.
When use a metaclass vs __init_subclass__?

References

Fluent Python, Ramalho — Chapters 23, 24.
PEP 252 — Making Types Look More Like Classes.
PEP 487 — Simpler subclass customisation.
Raymond Hettinger, "Descriptor HowTo Guide" (official Python docs).
David Beazley, "Python 3 Metaprogramming" (PyCon talk).

Descriptors and metaclasses — power features, used sparingly

Why this matters

Learning objectives

1. Descriptors

Data vs non-data

2. @property is a descriptor

3. Typed-attribute descriptors

__set_name__ (PEP 487, 3.6+)

Where to store the value

4. Why dataclasses / Pydantic still beat hand-rolled descriptors

5. The descriptor lookup chain

6. Methods are descriptors

7. Metaclasses

Custom metaclass

Practical (but rare) use cases

Stop using metaclasses if...

8. __init_subclass__ — the modern alternative

9. __getattr__ and __getattribute__

10. Worked example: a tiny ORM field

Hands-on lab (2 hours)

Common pitfalls

Self-check

References

2. `@property` is a descriptor

`__set_name__` (PEP 487, 3.6+)

8. `__init_subclass__` — the modern alternative

9. `getattr` and `getattribute`