System-design prompts — Python-focused

Six prompts at the senior+ level, each with a structured sample answer covering: requirements, API, data model, components, scaling, failure modes, and Python-specific choices.

Use these for whiteboard / virtual whiteboard practice. Time-box to 35 min each.

How to approach any prompt

A reliable 7-step structure (about 5 min each):

1. Clarify requirements (functional + non-functional). Quantify: QPS, data volume, latency SLO.
2. Capacity estimation: rough math (users × actions × bytes).
3. API design: REST endpoints / RPC / streams.
4. Data model: tables / collections / events.
5. Architecture diagram: clients → API → services → stores.
6. Scaling + reliability: cache, replicas, sharding, queues, retries.
7. Failure / edge cases + monitoring: what breaks, alerts, runbooks.

For Python-flavoured interviews, weave in specific library/runtime choices (FastAPI, asyncpg, asyncio, Pydantic, uvicorn workers) and explain tradeoffs.

Prompt 1 — URL shortener at scale

Design a URL shortener like bit.ly that serves 100k redirects/sec.

Requirements

Functional

• Create short URL from long URL.
• Optional custom alias.
• Redirect (302).
• Per-link analytics: clicks, referrers, user agents.
• Optional expiry, password protection.

Non-functional

• 100k redirects/sec; 1k creates/sec.
• p99 redirect latency < 50 ms.
• 99.95 % availability.
• Data: 1B links; ~100B click events/year.

Capacity

• Redirects: 100k QPS × 86 400 = 8.6B redirects/day.
• Click events: ~100 GB/day if 100 B each.
• DB writes: 100k/sec at peak. Must not commit synchronously per click.

API

POST /api/v1/links     { url, alias?, expires_at? } → { id, alias, short_url }
GET  /{alias}          → 302
GET  /api/v1/links/{alias}/stats

Data model

CREATE TABLE link (
    id            BIGSERIAL PRIMARY KEY,
    alias         TEXT UNIQUE NOT NULL,
    url           TEXT NOT NULL,
    user_id       UUID,
    created_at    TIMESTAMPTZ DEFAULT now(),
    expires_at    TIMESTAMPTZ
);
CREATE INDEX ON link(alias);

CREATE TABLE click_event (
    id        BIGSERIAL,
    link_id   BIGINT,
    ts        TIMESTAMPTZ,
    referrer  TEXT,
    ua        TEXT,
    country   TEXT
) PARTITION BY RANGE (ts);

Architecture

[Browser] → [CDN/Edge] → [Load Balancer] → [FastAPI redirect pods (40x)]
                                                ↓ cache hit (90%)
                                            [Redis cluster]
                                                ↓ miss
                                            [Postgres primary + 2 read replicas]
                                                ↓ async
                                            [Kafka click_events]
                                                ↓ batch consumer
                                            [ClickHouse / BigQuery]

Python-specific choices

• FastAPI + uvicorn[standard]: 8 workers × 40 pods = 320 workers handling redirects.
• asyncpg + aioredis to keep the event loop non-blocking.
• Pydantic validates the create endpoint only (skip on hot path).
• Background tasks push click events to Kafka; the redirect returns immediately.

Scaling

• Redis cache: alias → url with 1h TTL; 90 %+ hit ratio because hot links dominate.
• Read replicas for stats endpoint; primary handles writes.
• Async writes for click events (Kafka → ClickHouse).
• CDN edge: front the redirects with Cloudflare/Fastly that can issue 302s themselves for hottest aliases.

Failure modes

• Redis down → fall back to Postgres reads (with shortened cache TTLs once recovered).
• Kafka unavailable → buffer in local SQLite, replay on recovery (cap with disk limits).
• Postgres failover → connection pool with retry + circuit breaker.

Monitoring

• Prometheus: redirect QPS, p99 latency, cache hit ratio, replica lag, Kafka lag.
• Logs: structured JSON with alias, request_id, cache_status.
• Alerts: cache hit < 70 %, p99 > 100 ms, replica lag > 5 s.

What interviewers want to hear

• "I'd skip Pydantic on the redirect hot path — it's pure function and Pydantic costs μs we don't have at 100k QPS."
• "I'd choose async because connection-pool-bound workloads are exactly what asyncio is for."
• "I'd batch click events because writing 100k rows/sec to Postgres won't fly."

Prompt 2 — Real-time analytics pipeline

Ingest 1M events/sec, compute per-minute aggregates, serve dashboards.

Requirements

• 1M events/sec sustained ingest, 5M peak.
• Per-minute aggregates: count, sum, p50/p95/p99 per (event_type, country).
• Queryable from a dashboard within 1 min of event time.
• Replay last 7 days (reprocess on schema change).

Architecture

Producers → Kafka (200 partitions, 7-day retention)
   ↓
[Stream processor]
  - Python on Faust / Bytewax / Spark Streaming
  - Or Apache Flink (preferred at this scale)
   ↓
Hot store: ClickHouse (aggregates)
Cold store: S3 + Parquet (raw events)
   ↓
Dashboard API (FastAPI)
   ↓
Grafana / Custom UI (WebSocket for live)

Python-specific choices

• Producers: confluent-kafka-python (C-based, fast). Batch + compression.
• Stream processor: Bytewax for pure-Python streaming, or PyFlink for JVM-grade scale. Avoid pure asyncio at this scale.
• Aggregation: t-digest for percentiles (tdigest lib) instead of materialising all values.
• API layer: FastAPI + ClickHouse async driver (aioch).

Why not asyncio alone?

At 1M events/sec, you need multi-machine processing. asyncio is single-process, single-thread. You'd shard producers across consumers (Kafka does that), and within each consumer Python is fast enough — but use frameworks designed for backpressure + state + checkpoints.

Data model (ClickHouse)

CREATE TABLE events_minute (
    minute     DateTime,
    event_type LowCardinality(String),
    country    LowCardinality(String),
    count      UInt64,
    sum_value  Float64,
    p50        Float32,
    p95        Float32,
    p99        Float32
) ENGINE = AggregatingMergeTree() ORDER BY (minute, event_type, country);

Scaling

• Kafka partitions = throughput unit.
• Bytewax / Flink scale horizontally; one worker per partition.
• ClickHouse: shard by minute; pre-aggregate with materialised views.
• Dashboard API: cache common queries 30 s in Redis.

Failure modes

• Stream lag: alarm at > 60 s; auto-scale workers.
• Schema change: reprocess from S3 cold store (Bytewax replay or Spark batch job).
• ClickHouse write failure: producer retries with idempotent inserts.

Interview win

Explain Python's role honestly: it's the glue language, but for raw throughput you use it as a wrapper around C/JVM systems. "Pure-Python event loops will not do 1M events/sec; we need a framework that handles state, partitioning, and backpressure."

Prompt 3 — LLM chat backend

Design a ChatGPT-like backend: streaming responses, conversation memory, rate limits, auth.

Requirements

• Multi-user chat with auth.
• Streaming token responses (SSE).
• Persistent conversation history.
• Per-user rate limit (e.g., 50 messages/hour free tier).
• Multiple LLM providers behind a router.

Architecture

Web/Mobile → FastAPI (async) → JWT auth → router
                                 ↓
                       LLM provider abstraction
                       (OpenAI / Anthropic / local vLLM)
                                 ↓
                       SSE stream → client
                                 ↓
                       Postgres (messages)
                                 ↓
                       Redis (rate limits + recent context cache)
                                 ↓
                       Vector DB (retrieval)

Endpoints

• POST /conversations → create.
• POST /conversations/{id}/messages → SSE stream of assistant tokens.
• GET /conversations → list.
• GET /conversations/{id} → fetch.

Python-specific

• FastAPI with StreamingResponse for SSE.
• httpx async client for upstream LLM with httpx-sse for incoming streams.
• Pydantic for message validation.
• asyncio.TaskGroup for fan-out (parallel context retrieval + safety check).
• tenacity for retries on transient 5xx from providers.

Streaming

@app.post("/conversations/{cid}/messages")
async def send(cid: int, msg: NewMessage, request: Request):
    async def stream():
        async with llm_client.stream(msg.text, conversation=cid) as upstream:
            assembled = []
            async for token in upstream:
                if await request.is_disconnected():
                    break
                assembled.append(token)
                yield f"data: {json.dumps({'token': token})}\n\n"
            await save_message(cid, role="assistant", text="".join(assembled))
        yield "data: [DONE]\n\n"
    return StreamingResponse(stream(), media_type="text/event-stream")

Rate limiting

• Per-user sliding window in Redis (Lua script or ZADD + ZREMRANGEBYSCORE).
• Return 429 with Retry-After header.

Conversation memory

• Last N messages stored verbatim in Postgres.
• Older context summarised via background job (LLM-generated summary).
• For RAG: pull top-k vector matches before each call.

Failure modes

• Provider down → router fails over to fallback (Anthropic ↔ OpenAI).
• User disconnects mid-stream → save partial response; cancel upstream call.
• Token budget exceeded → truncate history (oldest first); summarise into a system message.

Cost control

• Track tokens in/out per request (Prometheus counter).
• Daily cost dashboard.
• Per-user cost ceiling enforced in middleware.

Prompt 4 — Background job system

Design a job queue + workers in Python, like Celery / RQ / Dramatiq.

Requirements

• Submit job (function + args) from web layer.
• Workers execute reliably.
• Retries on failure.
• Scheduled / delayed jobs.
• Visibility: see queued, running, failed.

Approach

Two main flavours of design:

(A) Build on existing broker (Redis / RabbitMQ). Recommended; don't reinvent reliability.

(B) From scratch. Educational; interviewers may push you here to test depth.

Components

[Web/API]
   ↓ enqueue
[Broker: Redis (or RabbitMQ / SQS)]
   ↓ pop
[Worker pool: Python processes]
   ↓ execute
[Result store: Redis / Postgres]
   ↓
[UI / API for status]

Job representation

@dataclass
class Job:
    id: UUID
    func: str            # "module.function"
    args: list[Any]
    kwargs: dict[str, Any]
    queue: str = "default"
    retry: int = 3
    eta: datetime | None = None     # delayed
    enqueued_at: datetime = field(default_factory=datetime.utcnow)

Serialize to JSON for broker.

Worker loop

def worker_loop(queue: str):
    while True:
        raw = redis.brpop(f"queue:{queue}", timeout=5)
        if raw is None: continue
        job = Job.from_json(raw[1])
        try:
            fn = import_dotted(job.func)
            result = fn(*job.args, **job.kwargs)
            redis.hset(f"result:{job.id}", mapping={"status": "ok", "result": json.dumps(result)})
        except Exception as e:
            if job.retry > 0:
                job.retry -= 1
                schedule_with_backoff(job)
            else:
                redis.hset(f"result:{job.id}", mapping={"status": "failed", "error": str(e)})

Reliability features

• Visibility timeout: worker takes job to "processing" set; if not acked in N seconds, returned to queue (handles crashes).
• Idempotency: jobs should be safe to run twice (caller responsibility, but document).
• Dead letter queue after max retries.
• Graceful shutdown: catch SIGTERM, finish current job, exit.

Scheduling

For delayed jobs: store in a Redis sorted set with score = unix_ts_to_run. A scheduler process polls ZRANGEBYSCORE 0 now, moves due jobs to ready queue.

Why not just asyncio.create_task?

• No persistence — process restart loses jobs.
• No retries / DLQ.
• No multi-machine scaling.
• No isolation — one CPU-bound task starves all others.

For real reliability you need a broker. But also acknowledge: for in-process background work in FastAPI, BackgroundTasks / asyncio.create_task is fine.

Production stack recommendation

• Dramatiq (cleaner than Celery, also Redis/RabbitMQ-backed).
• RQ for simplicity.
• Arq for asyncio-native.
• Temporal (now has Python SDK) for long-running, durable workflows.

Prompt 5 — Multi-tenant SaaS data isolation

Design data isolation for a SaaS where each tenant's data must not leak across requests.

Approaches

1. Database per tenant: hardest isolation, hardest to operate. Small N tenants only.
2. Schema per tenant (Postgres): one DB, separate schemas. Mid-N.
3. Shared schema + tenant_id column: simplest, scales most. Highest leak risk.

For most SaaS, (3) with strong defensive coding.

Architectural enforcement

• Every request has a tenant_id extracted from JWT.
• Pass tenant_id into a request-scoped context (contextvars).
• ORM query helper auto-filters by tenant_id. Never write a raw query without it.

from contextvars import ContextVar
_tenant: ContextVar[UUID] = ContextVar("tenant")

class TenantSession(Session):
    def query(self, *args, **kwargs):
        return super().query(*args, **kwargs).filter_by(tenant_id=_tenant.get())

Belt-and-braces: Postgres RLS

CREATE POLICY tenant_isolation ON orders
   USING (tenant_id = current_setting('app.tenant_id')::uuid);
ALTER TABLE orders ENABLE ROW LEVEL SECURITY;

Set app.tenant_id per connection in middleware. Even a buggy query can't leak rows.

Python middleware

@app.middleware("http")
async def tenant_middleware(request: Request, call_next):
    token = request.headers["Authorization"].split()[1]
    payload = jwt.decode(token, KEY, algorithms=["HS256"])
    tenant_id = payload["tenant_id"]
    token = _tenant.set(tenant_id)
    try:
        async with db_pool.acquire() as conn:
            await conn.execute("SET app.tenant_id = $1", tenant_id)
            request.state.conn = conn
            return await call_next(request)
    finally:
        _tenant.reset(token)

Pitfalls

• Forgetting to set tenant_id in a background task (context lost on thread/process hop). Always rebind explicitly.
• Caches keyed without tenant_id → cross-tenant leaks. Always include in cache keys.
• Search indices (Elasticsearch) must filter by tenant on every query.
• Logs / metrics should anonymise or scope to tenant; never spit raw PII.

Prompt 6 — Distributed scheduled job runner

Design a Cron-like service for thousands of customer-defined jobs running at scheduled times.

Requirements

• Customers schedule jobs via cron expression or interval.
• Run at most once per scheduled time (no duplicates).
• Survive node crashes.
• 10k jobs, average 1 per minute.

Architecture

[Customers API] → store cron jobs in Postgres
                        ↓
              [Scheduler service]
              (computes next run times)
                        ↓
              [Ready queue: Redis sorted set]
                        ↓
              [Worker pool] (subscribes, executes job)
                        ↓
              [Job runs table]
                        ↓
              [Webhooks / receipts / retries]

Data model

CREATE TABLE schedule (
    id UUID PRIMARY KEY,
    customer_id UUID,
    cron TEXT,
    payload JSONB,
    next_run TIMESTAMPTZ,
    enabled BOOLEAN
);
CREATE TABLE job_run (
    id UUID PRIMARY KEY,
    schedule_id UUID,
    scheduled_for TIMESTAMPTZ,
    status TEXT,
    started_at TIMESTAMPTZ,
    finished_at TIMESTAMPTZ,
    output JSONB,
    UNIQUE (schedule_id, scheduled_for)     -- ← idempotency
);

Scheduler

A small Python service that:

• Periodically scans schedule where next_run <= now().
• For each, inserts a job_run row with INSERT ... ON CONFLICT DO NOTHING (idempotency).
• Pushes job to Redis ready queue.
• Updates schedule.next_run to the next cron tick.

Run 2+ scheduler replicas with a leader lock (SELECT pg_try_advisory_lock(...)) so only one is active.

Worker

Pop from Redis, execute, update job_run.status. Retries on failure.

Why Postgres for source of truth?

• Strong durability.
• Unique constraint enforces no-double-fire.
• ACID for the "claim" semantics.

Python tooling

• croniter for parsing/evaluating cron expressions.
• apscheduler if you don't want to roll your own scheduler.
• For >100k scheduled jobs / sec, look at Temporal or AWS Step Functions instead.

Failure modes

• Clock skew across nodes: use a single canonical now() (Postgres now() server-side).
• Scheduler crashes mid-iteration: next leader resumes; idempotency saves you from duplicates.
• Slow jobs blocking workers: timeout + circuit break.
• Customer payload triggers infinite loop: enforce wall-clock + CPU limit (signal.alarm or sandboxed subprocess).

Cross-cutting talking points to volunteer in any system design

These three almost always score bonus points if you weave them in:

1. Observability stack: structured logs (structlog), Prometheus metrics, OpenTelemetry tracing — explain how you'd debug a slow request end-to-end.
2. Deployment story: how you'd ship the service (Dockerfile, blue/green or canary, k8s/Fly.io, secrets via Vault / cloud KMS).
3. Cost awareness: rough cost back-of-envelope ("this design at 1M QPS is ~$X/month; if budget halves, here's what we drop first").

If you're applying to MAANG / top-tier startups, expect at least one prompt above. Spend 30 min per prompt practicing with a friend; you'll be ready for almost anything in Python system design loops.