polars — the fast alternative to pandas — Python Mastery

Why this matters

Polars is the fast, modern alternative to pandas: 5–30× faster, multicore by default, lazy evaluation, immutable DataFrames, no SettingWithCopyWarning. Written in Rust on top of Apache Arrow. In 2026, many teams have switched to Polars for new analytics work and use pandas only for legacy / library interop.

Learning objectives

Read / write data with Polars.
Use the expression API (pl.col, pl.when, pl.lit).
Use lazy evaluation for huge datasets.
Translate pandas patterns to Polars.
Know when Polars wins and when pandas still rules.

1. Install + basics

powershell

uv add polars pyarrow

python

import polars as pl

df = pl.DataFrame({
    "name": ["Ada", "Bob", "Cara"],
    "age":  [30, 25, 35],
    "city": ["NY", "SF", "LA"],
})

df.head(); df.describe(); df.schema; df.shape
df.columns; df.dtypes

A DataFrame is immutable — every operation returns a new one. No inplace=True, no SettingWithCopyWarning.

2. Reading / writing

python

pl.read_csv("data.csv", try_parse_dates=True)
pl.read_parquet("data.parquet")
pl.read_json("data.json")
pl.read_ndjson("data.jsonl")
pl.read_excel("file.xlsx")
pl.read_database("SELECT * FROM t", conn)

df.write_csv("out.csv")
df.write_parquet("out.parquet")
df.write_ndjson("out.jsonl")

Polars and pandas can interoperate via Arrow:

python

pd_df = pl_df.to_pandas()
pl_df = pl.from_pandas(pd_df)

3. Selection and filtering

python

df.select("name", "age")
df.select(pl.col("name"), pl.col("age") * 2)

df.filter(pl.col("age") > 28)
df.filter((pl.col("age") > 28) & (pl.col("city") == "NY"))

Notice: no df["col"] style for math; you compose expressions with pl.col() inside .select / .filter / .with_columns.

4. Adding / modifying columns

python

df.with_columns(
    age_squared=pl.col("age") ** 2,
    decade=(pl.col("age") // 10) * 10,
    full_name=pl.col("first") + " " + pl.col("last"),
)

df.with_columns(
    age_z=(pl.col("age") - pl.col("age").mean()) / pl.col("age").std(),
)

with_columns is the universal column-add / transform method. Chain it as you'd chain .assign in pandas.

Conditional

python

df.with_columns(
    band=pl.when(pl.col("age") < 30).then(pl.lit("young"))
          .when(pl.col("age") < 60).then(pl.lit("mid"))
          .otherwise(pl.lit("senior"))
)

5. Groupby (called `group_by`)

python

df.group_by("city").agg(
    pl.col("age").mean().alias("avg_age"),
    pl.col("age").max().alias("max_age"),
    pl.len().alias("n"),
)

# Multiple keys
df.group_by(["city", "department"]).agg(...)

# Maintain order
df.group_by("city", maintain_order=True).agg(...)

For "per-group transform" (broadcast a group stat back to each row):

python

df.with_columns(
    pl.col("age").mean().over("city").alias("city_avg")
)

.over("city") is Polars' answer to pandas .groupby().transform(). Much cleaner.

6. Joins

python

left.join(right, on="user_id", how="inner")
left.join(right, left_on="uid", right_on="user_id", how="left")
left.join(right, on="user_id", how="outer")
left.join(right, on="user_id", how="semi")          # rows in left also in right
left.join(right, on="user_id", how="anti")          # rows in left NOT in right

semi / anti joins are powerful for filtering. Pandas has to emulate them.

Asof joins (time series)

python

left.join_asof(right, on="ts", by="key", strategy="backward", tolerance="5m")

Match each row in left to the nearest preceding row in right within 5 minutes. Indispensable for events + state at a moment.

7. Lazy evaluation — the killer feature

Switch any DataFrame to a LazyFrame:

python

lf = (
    pl.scan_csv("huge.csv")          # scan, don't read
    .filter(pl.col("status") == "paid")
    .group_by("day")
    .agg(pl.col("revenue").sum())
)
df = lf.collect()                     # executes the optimised plan

scan_csv / scan_parquet return LazyFrames. The plan is optimised (predicate pushdown, projection pushdown, parallelisation) before any data is read.

python

lf.explain()                          # print the query plan
lf.show_graph()                       # visualise (requires graphviz)

For datasets larger than RAM, use streaming:

python

lf.collect(streaming=True)

This is Polars' answer to pandas chunked CSV. It just works.

8. Dates and time

python

df.with_columns(
    year=pl.col("ts").dt.year(),
    dow=pl.col("ts").dt.weekday(),
    is_weekend=pl.col("ts").dt.weekday() >= 5,
)

df.group_by_dynamic("ts", every="1d").agg(pl.col("revenue").sum())
df.rolling("ts", period="7d").agg(pl.col("revenue").sum())

group_by_dynamic resamples; rolling does windowed reductions. Both handle time-aware semantics natively.

9. String operations

python

df.with_columns(
    upper=pl.col("name").str.to_uppercase(),
    domain=pl.col("email").str.split("@").list.get(1),
    has_num=pl.col("text").str.contains(r"\d+"),
    cleaned=pl.col("text").str.replace_all(r"\s+", " "),
)

Accessed via .str and .list namespaces.

10. Working with nested data

Polars natively handles lists and structs (Arrow types) — pandas struggles here.

python

# List column
df = pl.DataFrame({"tags": [["a", "b"], ["a"], ["b", "c", "d"]]})

df.with_columns(
    tag_count=pl.col("tags").list.len(),
    first_tag=pl.col("tags").list.first(),
    contains_a=pl.col("tags").list.contains("a"),
)

# Explode list to rows
df.explode("tags")

python

# Struct column (multiple fields per cell)
df = pl.DataFrame({"info": [{"x": 1, "y": 2}, {"x": 3, "y": 4}]})
df.with_columns(pl.col("info").struct.field("x").alias("x"))

For JSON pipelines, this is a massive win.

11. Performance notes

Parallelism: Polars uses all cores by default. No multiprocessing boilerplate.
Predicate / projection pushdown: lazy mode pushes filters and column selection into the file reader; reads only what's needed.
Arrow memory: zero-copy interop with pyarrow, DuckDB, Spark, etc.
No SettingWithCopy headaches: immutability removes a category of bugs.

Benchmarks consistently show 3-30× speedups over pandas. Polars' TPC-H benchmark is the canonical reference.

12. Translating pandas → Polars cheat sheet

pandas	Polars
`df["x"] = ...`	`df = df.with_columns(x=...)`
`df[df.age > 30]`	`df.filter(pl.col("age") > 30)`
`df.groupby("a").b.mean()`	`df.group_by("a").agg(pl.col("b").mean())`
`df.groupby("a").b.transform("mean")`	`pl.col("b").mean().over("a")`
`df.merge(o, on="id")`	`df.join(o, on="id")`
`df.pivot_table(...)`	`df.pivot(...)`
`df.melt(...)`	`df.unpivot(...)`
`df.apply(f, axis=1)`	rewrite as expressions, or `df.map_elements(f)` (avoid)
`df["x"].rolling(7).mean()`	`pl.col("x").rolling_mean(window_size=7)`
`df.resample("D").sum()`	`df.group_by_dynamic("ts", every="1d").agg(...)`

13. When Polars vs pandas

Use Polars when	Use pandas when
Building new pipelines	Quick exploration in a notebook + libraries that want a DataFrame
Datasets > 100 MB	Tiny datasets where pandas is "good enough"
Need parallelism / lazy	scikit-learn, statsmodels, seaborn expect pandas (you can `.to_pandas()` at the boundary)
You want predictable performance	You're working with someone else's pandas-heavy code

Many production ETL pipelines now: Polars for transforms → to_pandas() only when handing off to a library.

14. Worked example: e-commerce analytics (same as 5.2)

python

import polars as pl

orders = pl.scan_csv("orders.csv", try_parse_dates=True)
users  = pl.scan_csv("users.csv")

daily = (
    orders.join(users, on="user_id", how="left")
          .filter(pl.col("status") == "paid")
          .with_columns(
              day=pl.col("placed_at").dt.date(),
              aov=pl.col("revenue") / pl.col("units"),
          )
          .group_by("day")
          .agg(
              revenue=pl.col("revenue").sum(),
              orders=pl.len(),
              aov=pl.col("aov").mean(),
          )
          .sort("day")
          .with_columns(
              revenue_7d=pl.col("revenue").rolling_mean(window_size=7),
              orders_7d=pl.col("orders").rolling_mean(window_size=7),
          )
          .collect()
)

daily.write_parquet("daily_revenue.parquet")

Cleaner than the pandas version, runs faster, scales to GBs without changes.

15. Polars + DuckDB

For SQL-heavy workflows, DuckDB (uv add duckdb) is the perfect companion. Both share Arrow memory, so you can hop between them cost-free:

python

import duckdb, polars as pl

result = duckdb.sql("""
    SELECT city, AVG(age) AS avg_age
    FROM df_polars                  -- references the Polars DataFrame directly
    GROUP BY city
""").pl()                            # returns a Polars DataFrame

DuckDB is OLAP-grade SQL on a single machine; Polars is the dataframe API. The combo replaces a surprising amount of what people previously used Spark for.

Hands-on lab (2 hours)

Re-do the pandas lab (Lesson 5.2) in Polars: load, filter, add features, group_by, join.
Convert one of your pandas pipelines to lazy mode; compare .explain() outputs.
Use group_by_dynamic("ts", every="1d") to make daily aggregations.
Use over("group_col") to compute a per-group z-score.
Use join_asof to attach the "last known" price to a stream of trades.
explode a list column; convert struct fields to columns.
Benchmark a 1M-row groupby in Polars vs pandas.

Common pitfalls

Mixing pandas indexing (df["x"]) inside Polars .select / .filter — use pl.col("x").
Forgetting that DataFrame ops are not in-place; always reassign or chain.
Using map_elements (row-by-row) when an expression would work.
Calling .collect() too early in lazy pipelines (loses optimisation benefits).
Expecting pandas-style integer indexing — Polars has no "index"; use with_row_index() if needed.

Self-check

What does pl.col do?
Lazy vs eager evaluation in Polars?
over vs group_by?
When use join_asof?
Trade-off between Polars and pandas in 2026?

References

Polars docs: https://docs.pola.rs/.
"Polars vs Pandas" benchmarks: https://pola.rs/posts/benchmarks/.
Modern Polars (online book): https://kevinheavey.github.io/modern-polars/.
Ritchie Vink, "Polars: a fast DataFrame library" (talk).
DuckDB docs: https://duckdb.org/docs/.

polars — the fast alternative to pandas

Why this matters

Learning objectives

1. Install + basics

2. Reading / writing

3. Selection and filtering

4. Adding / modifying columns

Conditional

5. Groupby (called group_by)

6. Joins

Asof joins (time series)

7. Lazy evaluation — the killer feature

8. Dates and time

9. String operations

10. Working with nested data

11. Performance notes

12. Translating pandas → Polars cheat sheet

13. When Polars vs pandas

14. Worked example: e-commerce analytics (same as 5.2)

15. Polars + DuckDB

Hands-on lab (2 hours)

Common pitfalls

Self-check

References

5. Groupby (called `group_by`)