STEM · Full roadmap · ~90 min read · 26 steps
🤖How large language models actually work
Understand what ChatGPT and Claude really do under the hood
Activities in this path
Skill tree
0 / 26 steps
Unit 1
Start here
Course overview
The one trick behind every chatbot
An LLM predicts the next chunk of text
Why "predict the next word" can feel like thinking
Prediction at scale produces useful behavior
Tokens: the real units a model reads
Models work in tokens, not whole words
Turning words into numbers with embeddings
Each token becomes a list of numbers
Unit 2
Meaning as geometry
Similar meanings sit near each other in space
What "neural network" actually means
A neural network is a big stack of tunable math
Parameters: the knobs that get tuned
Parameters are the learned numbers inside the model
The transformer, in plain words
The transformer is the design that made modern LLMs work
Attention: deciding what matters
Attention lets each word focus on the relevant other words
Unit 3
Pretraining: reading the internet
Pretraining tunes parameters by predicting text at huge scale
From raw predictor to helpful assistant
Fine-tuning and RLHF turn a base model into a chatbot
The context window
The context window is how much text the model can consider at once
Sampling and temperature
The model picks from a list of likely next tokens
Why models make things up
Hallucination comes from fluent guessing, not lying
Unit 4
What these models genuinely cannot do
Know the built-in limits
Prompting basics: phrasing changes everything
Clear, specific prompts produce better answers
System prompts: the hidden instructions
A system prompt sets rules before you ever type
Tool use and function calling
Models can be wired to call outside tools
Retrieval and RAG
RAG feeds the model relevant documents at answer time
Unit 5
Scaling laws: why bigger got better
More data, parameters, and compute reliably improve models
Bias and safety, honestly
Models reflect their data and can be confidently wrong
Common mistakes people make
Avoid the traps that lead to bad results
A simple routine for better results
A repeatable way to get more from any LLM
The realistic picture
A capable, fast, fallible text tool
Unit 6
Where to go next
Where to go next