BPE tokenizer from scratch
Build a byte-level BPE tokenizer in pure Python, train it on a small corpus, and compare against tiktoken.
Open brief →
Portfolio projects · Large Language Models
Build the systems, not just the vocabulary
Start a project as soon as you finish the foundations module, then extend it as later modules land.
nanoGPT++ — a modern decoder-only Transformer
Build, train, and evaluate a ~50-150M parameter Transformer with modern conventions (RMSNorm, SwiGLU, RoPE, GQA, FlashAttention).
Open brief →Domain-specialised LLM via LoRA + LLM-as-judge eval
Fine-tune an open model with LoRA and evaluate it with an LLM-as-judge pipeline.
Open brief →Preference alignment with DPO + synthetic data
Synthesise preference pairs and align a model with DPO, then measure the win rate against the base.
Open brief →Mini inference engine (KV cache + sampling + quantization)
Serve a model in a compact engine: implement the KV cache, sampling, and INT4 quantization end to end.
Open brief →