AI Engineering

How LLM apps work, explained idea first. You build a small version from scratch before using any library.

Covers calling an LLM from Python, prompting, tool calling, RAG, agents, evaluation and safety. Libraries change fast, so each page says which versions it covers.

How to use this roadmap

Topics are in teaching order. “Builds on” links show what to know first, even from other tracks.
New? Start at the top. Know some already? Jump to a level, or skip any topic you can already explain.
Short on time? Tick “Asked often only” to see what interviews ask most.
Your progress is saved in this browser.

Best after: Python. Do Python foundations and core first. RAG and agents also reuse ideas from System Design.

29 topics in 4 levels, about 12 h of reading. With practice, plan 2 to 3 times that. 1 of 29 topics have lessons so far.

4 practice problems from this track’s lessons, in one list.

Foundations

1. ML refresher: splits, overfitting, bias–varianceComing soon
How models learn from data, and why a model can do well in training but fail in real use.
25 min read
2. Precision, recall, F1 and ROC-AUCComing soon
How to score a model, and why accuracy alone can mislead.
20 min read
3. Neural-network intuitionComing soon
What a neural network does, without the heavy maths.
20 min read
4. What an LLM isAsked oftenComing soon
A model that predicts the next word, and why that gets so far.
15 min read
5. Tokens and tokenizationComing soon
Models read chunks of words called tokens, which set the cost and the limits.
15 min readBuilds on: What an LLM is
6. EmbeddingsAsked oftenComing soon
Turn text into numbers so that similar meanings sit close together.
20 min read
7. Transformers and attention, intuitivelyComing soon
How a model decides which earlier words matter for the next one.
25 min readBuilds on: Embeddings
8. Context windowsComing soon
How much text a model can see at once, and what happens past the limit.
10 min readBuilds on: Tokens and tokenization
9. Sampling: temperature and top-pComing soon
Dials that make answers more predictable or more varied.
10 min read
10. Pretraining vs fine-tuning vs preference tuningComing soon
The stages that turn raw text into a helpful assistant.
15 min read
11. Why models hallucinateComing soon
Why models sometimes state wrong things confidently, and how to reduce it.
15 min read

Core

1. Calling an LLM API from Python: messages, keys, retries and errorsComing soon
Send messages to a model from Python, keep keys safe and handle errors and limits.
25 min readBuilds on: What an LLM is · Exceptions (Python)
2. Prompt engineering: instructions, few-shot, structured output, reasoningAsked oftenComing soon
Write instructions and examples that get reliable answers.
25 min read
3. Context engineeringComing soon
Choose what goes into the model's limited context, and in what order.
20 min readBuilds on: Context windows
4. Tool and function callingAsked oftenComing soon
Let a model call your Python functions to look things up or take actions.
20 min readBuilds on: Calling an LLM API from Python: messages, keys, retries and errors
5. Structured outputs validated with PydanticComing soon
Get JSON back that your code can trust, and check it.
15 min readBuilds on: Calling an LLM API from Python: messages, keys, retries and errors · Type hints (Python)
6. StreamingComing soon
Show the answer word by word as it is written.
10 min readBuilds on: Calling an LLM API from Python: messages, keys, retries and errors
7. Choosing a model; cost and latencyComing soon
Pick a model that is good enough, fast enough and cheap enough.
15 min read
8. RAG basics: why, chunking, embedding, top-kAsked oftenNot started
Look up the right documents first, then let the model answer from them.
- The chatbot doesn’t know your docs
Start
40 min read, about 1 h 30 min with practiceBuilds on: Embeddings · Calling an LLM API from Python: messages, keys, retries and errors
9. Vector databases and ANN indexes: HNSW, IVF, PQAsked oftenComing soon
Find the most similar items among millions, fast.
25 min readBuilds on: RAG basics: why, chunking, embedding, top-k

Advanced

1. Advanced RAG: hybrid search, reranking, query rewriting, metadata filters, GraphRAG, agentic RAGComing soon
Ways to find better documents when basic RAG misses.
35 min readBuilds on: Vector databases and ANN indexes: HNSW, IVF, PQ
2. RAG evaluation: recall@k, MRR, nDCG, faithfulnessAsked oftenComing soon
Measure whether search finds the right documents and answers stick to them.
25 min readBuilds on: RAG basics: why, chunking, embedding, top-k
3. Agents: tools, loops, ReAct, planning, memory, MCP, human-in-the-loopAsked oftenComing soon
Models that plan, call tools in a loop and ask a person when unsure.
35 min readBuilds on: Tool and function calling
4. LLMOps: evals, LLM-as-judge, tracing, guardrails, caching, routingComing soon
Test, trace and guard AI features in production, and keep costs down.
30 min read
5. Safety: prompt injection, jailbreaks, data leakage and PIIComing soon
How AI features get tricked or leak data, and how to defend them.
20 min read
6. Serving open models: inference servers, quantization, KV cache, batchingComing soon
Run open models on your own servers, faster and cheaper.
30 min read
7. Fine-tuning: SFT, LoRA, QLoRA, DPO, distillationComing soon
Teach a model new habits with your own examples, and when not to bother.
30 min readBuilds on: Pretraining vs fine-tuning vs preference tuning

Interview-ready

1. AI system design: docs assistant, support agent, code-review bot, semantic search, LLM gatewayAsked oftenComing soon
Classic AI system design questions, built from v1 to v3.
90 min readBuilds on: The system design interview framework (System Design) · RAG basics: why, chunking, embedding, top-k
2. AI coding questions: cosine top-k, chunker, BM25, RRF, mini RAG, BPE, semantic cache, ReAct loopAsked oftenComing soon
Build the small pieces of AI systems in plain Python.
45 min readBuilds on: Embeddings