AI Engineer
Build products on top of models — RAG, agents, evals, and the boring reliability work.
intermediate · 4–7 months · 5 milestones · 19 topics
Tap a box to open it · right-click to mark done
Tap a box to open it
Full outline
The practical path into AI engineering: prompting that survives contact with users, embeddings and retrieval, tool-calling agents, evaluation harnesses, guardrails, cost and latency control, and shipping all of it to production.
01Model Fundamentals
Enough theory to make good engineering decisions.
- How LLMs actually work
Transformers, tokens and next-token prediction. You do not need to train one, but you must know what it can and cannot do.
- Choosing a model
Capability, latency, cost and context length trade off constantly. The best model is rarely the biggest one.
- Prompt engineering
Clear instructions, examples, and structure. Most 'model failures' are specification failures.
- Structured output
Free text is unusable downstream. Force a schema and validate it.
- How LLMs actually work
02Retrieval (RAG)
Giving the model the facts it was never trained on.
- Embeddings & vector search
Turning text into vectors so you can search by meaning instead of keyword.
- Chunking & ingestion
The unglamorous work that decides whether RAG is good or useless.
- Retrieval quality
Most bad RAG answers are retrieval failures, not generation failures.
- Grounding & citations
Make the answer traceable, so users can verify instead of trust.
- Embeddings & vector search
03Agents & Tools
Models that can do things, not just say things.
- Tool calling
Giving the model typed functions it can invoke — the foundation of every agent.
- Agent loops
Plan, act, observe, repeat — with hard limits so it terminates.
- MCP & integrations
The Model Context Protocol standardises how models reach external tools and data.
- Sandboxing & safety
An agent with real credentials is a real liability. Contain it.
- Tool calling
04Evaluation
Without evals you are not engineering, you are guessing.
- Building an eval set
Fifty real, hand-labelled examples beat any public benchmark for your specific product.
- Scoring methods
Different tasks need different graders.
- Tracing & observability
You need to see every prompt, response, tool call and token in production.
- Building an eval set
05Production
Latency, cost and failure modes users will actually hit.
- Streaming & UX
Perceived speed matters more than total time. Stream everything you can.
- Cost & latency control
AI features get expensive quietly. Instrument early.
- Guardrails
The model will eventually produce something you do not want to ship. Plan for it.
- Fine-tuning & distillationoptional
The last resort, not the first. Prompting and retrieval solve most problems more cheaply.
- Streaming & UX