Academy Pricing

Sign in Get started

← Course overview

Large Language Models

Sign in to track progress

🔢 Foundations 3

Math for LLMs (only the parts you really need)
PyTorch essentials for LLM work
Classic NLP (the part that still matters)
Quiz — Foundations

🧠 Neural foundations 3

Neural networks (the parts that survived)
Sequence models before Transformers (RNN, LSTM, seq2seq)
Attention, intuitively
Quiz — Neural foundations

🕸️ Transformer 5

Transformer anatomy
Attention deep dive (MHA → MQA → GQA → FlashAttention)
Positional encodings (sinusoidal → RoPE → YaRN)
Build a GPT from scratch
Modern transformer variants (what changed since 2017)
Quiz — Transformer

📊 Pretraining 5

Pre-training objectives
The pretraining data pipeline
Scaling laws (Chinchilla and beyond)
Distributed training (DDP, ZeRO, FSDP, TP, PP)
Training infrastructure (mixed precision, optimisers, monitoring)
Quiz — Pretraining

🎯 Post-training 5

Supervised Fine-Tuning (SFT)
RLHF (the InstructGPT recipe)
DPO, ORPO, KTO, IPO, SimPO
RLAIF and Constitutional AI
Reasoning RL (o1, R1, GRPO, RLVR)
Quiz — Post-training

⚡ Inference 5

Sampling and decoding
KV cache and PagedAttention
Speculative decoding
Quantization (INT8 / INT4 / FP8 / AWQ / GPTQ / GGUF)
Long context (engineering 100k+ tokens)
Quiz — Inference

🚀 Frontiers 4

Mixture of Experts (MoE)
State-space models (Mamba, S4, hybrids)
Multimodal LLMs (vision, audio, video)
Agentic / native-tool-use models
Quiz — Frontiers

🔎 Evaluation & interpretability 3

Evaluating LLMs (benchmarks, leaderboards, judges)
Safety, harmfulness, and bias evaluation
Interpretability (mechanistic + behavioural)
Quiz — Evaluation & interpretability

💼 Interview prep 2

Interview prep — LLM system design templates
Interview prep — LLM coding drills

🏆 Final exam

Academy / Large Language Models / Pretraining

Quiz — Pretraining

8 questions drawn at random from this module. Score 70% or higher to earn the Pretraining badge. Unlimited retakes.

📊 Module 3 8 questions Pass mark 70%

Loading quiz…

The fastest way to ask and learn about AI & Machine Learning.

Academy

All courses Python Mastery Large Language Models Agentic AI IAM for AI Agents

Product

Features Pricing FAQ

Account

Sign in Create account Dashboard

© AskAiml. All rights reserved.