roadmap ai-engineer --year 2026

AI Engineer Roadmap

A step-by-step path to become an AI engineer in 2026: from Python and model APIs to RAG, agents, MCP, evaluation and production, with what is new this year and what to watch in 2027. Follow the main line top to bottom; click any topic to see why it matters and mark it as done.

~/roadmap/ai-engineer

Legend

  • Learn it
  • Alternative
  • Good to know
  • New in 2026
  • Watch in 2027
  • Legacy
  1. 01 Software foundations 8
  2. 02 Understand the role 5
  3. 03 How LLMs work 8
  4. 04 Use model APIs 9
  5. 05 Prompt engineering 7
  6. 06 Structured output and tool calling 5
  7. 07 Context engineering 6
  8. 08 Choose the right model 7
  9. 09 Run open models yourself 8
  10. 10 Embeddings 6
  11. 11 Vector databases 7
  12. 12 Retrieval-augmented generation (RAG) 10
  13. 13 AI agents 10
  14. 14 Model Context Protocol (MCP) 6
  15. 15 Fine-tuning and adaptation 7
  16. 16 Evaluation 9
  17. 17 Observability 8
  18. 18 Safety, security and ethics 8
  19. 19 Multimodal AI 6
  20. 20 Ship to production 8
  21. 21 AI-assisted development 4
  22. 22 2027: what to watch 6

How to use this roadmap

  • Follow the main line from top to bottom. Each stage builds on the ones above it. The early stages look basic, but most AI projects fail on plain engineering, not on the model.
  • Solid blue topics are the path. Learn those first. Outlined topics are alternatives: pick one per group. Dashed topics are good to know; come back to them when a project needs them.
  • Click any topic to read why it matters and where to learn more, then mark it as done. Your progress stays in this browser only.
  • Use the highlight buttons to see only the main path, what is new in 2026 or what to watch in 2027.

What an AI engineer does in 2026

An AI engineer builds software products on top of pre-trained foundation models. The work is mostly not training models. It is choosing a model, connecting it to the company’s data and tools, constraining what it can say and do, measuring whether it is good enough, and keeping it fast, affordable and safe in production.

Typical deliverables are a support assistant that answers from internal documentation, a pipeline that turns PDFs into structured data, an agent that files tickets or prepares pull requests, or an AI feature inside an existing app. The common thread is software engineering with a non-deterministic component in the middle.

RoleMain focusTypical tools
AI engineerProducts built on existing models: prompts, retrieval, tools, agents, evals, productionModel APIs, vector databases, agent SDKs, MCP, eval and tracing tools
ML engineerTraining, tuning and serving models on the company’s own data, plus MLOpsPyTorch, feature stores, training pipelines, model registries, GPU serving
Data scientistAnalysis, experiments and statistical models that answer business questionsSQL, pandas, notebooks, statistics, BI tools

The boundaries are blurry and many job ads ask for a mix. This roadmap covers the AI engineer core and points out where it touches ML engineering, such as fine-tuning and self-hosted serving.

What changed recently

  • Context engineering became a discipline. With agents running many steps, the hard part is deciding what goes into the context window at each step, not writing one clever prompt.
  • MCP became the standard way to connect tools. In December 2025 Anthropic donated the Model Context Protocol to the Agentic AI Foundation under the Linux Foundation, together with OpenAI’s AGENTS.md and Block’s goose. Most major assistants and IDEs now act as MCP clients.
  • Agent frameworks matured. OpenAI, Anthropic and Google each ship an agent SDK, and Microsoft Agent Framework, the successor to AutoGen and Semantic Kernel, reached version 1.0 in 2026.
  • The Responses API replaced the Assistants API. OpenAI set 26 August 2026 as the Assistants API shutdown date, so new OpenAI projects build on Responses.
  • Coding agents became everyday tools. Agents that read a repository, run tests and open pull requests changed how AI engineers build, and raised the bar for reviewing generated code.
  • Evals and tracing are expected. Teams now ask for an eval suite and production traces before an AI feature ships, and OpenTelemetry is defining standard conventions for GenAI telemetry.
  • Regulation is real. The EU AI Act’s bans and general-purpose AI rules already apply, and a 2026 agreement scheduled most high-risk obligations for December 2027 and August 2028.

A realistic 12-month plan

MonthsFocusStages
1 to 2Python, APIs, Docker and how LLMs work1 to 3
3 to 4Model APIs, prompting, structured output, tool calling and context4 to 7
5 to 6Choosing and running models, embeddings, vector databases and RAG8 to 12
7 to 9Agents, MCP, evaluation and observability13, 14, 16 and 17
10 to 12Security, multimodal, production and fine-tuning where it pays off15 and 18 to 20
OngoingAI-assisted development and the 2027 topics21 and 22

That pace assumes a few hours most days. If you already work as a software engineer, you can move through the first stages in weeks and spend the time on evals, agents and production instead.

Build these projects along the way

  1. A command-line chat app that streams answers, can switch between two providers and prints the token cost of every reply.
  2. An extraction service that turns invoices or receipts (PDF or image) into JSON validated with Pydantic, with a small eval set that measures field accuracy.
  3. A RAG assistant over a real document set, such as a product’s documentation, using PostgreSQL with pgvector, hybrid search, reranking and answers with citations.
  4. An MCP server that exposes a database or API you own, used from an MCP client such as Claude or VS Code.
  5. An agent that completes a multi-step task with tools, asks for approval before risky actions, is traced end to end and has its evals running in CI.
  6. One of the above shipped as a FastAPI service in Docker, with streaming, caching, a fallback model and a monthly budget alert.

Put each project in a public repository with a README that shows your eval results and explains your trade-offs. Numbers like “faithfulness went from 0.71 to 0.89 after adding reranking” say more in an interview than a demo video.

FAQ

Do I need a degree or advanced math to become an AI engineer?

No. Most AI engineering work is software engineering, and employers mostly look for shipped projects and solid coding skills. You need math intuition for vectors, similarity and probability, not the ability to derive training algorithms. A degree helps with some employers and with research-heavy ML roles.

Python or TypeScript?

Python first: most SDKs, eval tools, data libraries and examples start there. Add TypeScript if you build web front ends or Node.js services. The concepts are the same in both, and every major provider has official SDKs for each.

Do I need a GPU?

Not to start. Hosted model APIs cover most of the roadmap for a few dollars of credit. A recent laptop runs small open models through Ollama or LM Studio. Rent a cloud GPU by the hour when you get to fine-tuning or serving with vLLM.

Should I use RAG or fine-tuning?

Start with good prompts, then RAG. Use RAG when the model needs knowledge that changes or must be cited. Fine-tune when you need a consistent behaviour, style or format, or a smaller model to match a bigger one on a narrow task. Many production systems use both.

Which agent framework should I learn?

First build a small agent loop by hand with plain tool calling, so you understand what frameworks do. Then pick the SDK that matches your main model provider or your team’s stack. The underlying skills, such as tool design, context management, evals and guardrails, transfer between frameworks.

How long does it take to become an AI engineer?

For an experienced developer, three to six months of focused practice is enough to be useful on AI projects. From zero, plan for about a year, because the software foundations take time. Real projects with measured results matter more than the calendar.

What is an eval, and why does everyone talk about them?

An eval is a repeatable test of model behaviour: a set of inputs plus a way to score the outputs, by code, by a judge model or by people. Because LLM output varies and every change can cause regressions, evals are how teams decide whether a prompt, model or retrieval change is actually better.

Is AI engineering a good career choice?

Demand for people who can turn models into reliable products has grown quickly, and the skills, from software design to evaluation and security, stay useful even as specific models and tools change. Expect the field to keep moving fast; the habit of testing new tools against your own evals is the skill that lasts.

Inspired by the community AI Engineer roadmap on roadmap.sh, rewritten from scratch and updated for 2026 and 2027. See also the DevOps Roadmap.

Topic guide

01 Software foundations

AI engineering is software engineering first. Models are one component; the rest of the product is ordinary code that has to be reliable.

Python

Learn it

The language most AI SDKs, data tools and examples are written in. Be comfortable with functions, classes, type hints, async/await, virtual environments (uv or venv) and Pydantic for validating data.

TypeScript

Alternative

The other first-class language for AI apps. Every major provider ships an official TypeScript SDK, and it is the natural choice when the AI feature lives in a web front end or a Node.js service.

HTTP APIs and JSON

Learn it

Model providers are reached over HTTPS with JSON bodies. Know headers and auth, status codes, timeouts and Server-Sent Events, which is how most APIs stream tokens back to you.

Git and GitHub

Learn it

Prompts, eval datasets and agent configs change as often as code and belong in version control next to it.

Linux and the terminal

Learn it

GPU servers, containers and most coding agents run in a Linux shell. Navigate the filesystem, manage processes and read logs without a GUI.

Docker

Learn it

Package your app, run a vector database or a local model server with one command, and ship the same image to production.

SQL and data basics

Good to know

Most company knowledge lives in databases. Enough SQL to query it, and enough data handling (CSV, Parquet, pandas) to clean an eval dataset.

Math intuition

Good to know

You do not need research-level math. Vectors, dot products, cosine similarity and basic probability are enough to understand embeddings, similarity search and sampling.

02 Understand the role

Know what the job actually is before you study for it, so you spend your time on the skills employers pay for.

What an AI engineer does

Learn it

An AI engineer builds products on top of pre-trained foundation models: calling model APIs, connecting them to company data and tools, evaluating the results and running them reliably in production. Training large models from scratch is usually someone else's job.

AI engineer vs ML engineer

Learn it

ML engineers typically train, tune and serve models on a company's own data and run the MLOps around them. AI engineers mostly adapt existing models through prompting, retrieval, tools and occasional fine-tuning. The roles overlap and many teams expect a bit of both.

Day-to-day responsibilities

Learn it

Prototype quickly, measure quality with evals, ship behind feature flags, watch cost and latency, and work with product, data and security people on what the model is allowed to do.

Designing AI features

Learn it

Model output is probabilistic and sometimes wrong. Good AI features show sources, let people correct or undo, keep a human in the loop for risky actions and degrade gracefully when the model or provider fails.

Machine learning basics

Good to know

Supervised vs unsupervised learning, training vs test data, overfitting and classic metrics like precision and recall. It helps you talk to data scientists and know when a simple classifier beats an LLM.

03 How LLMs work

A working mental model of large language models explains most of their strange behaviour and most of your bills.

Tokens and tokenization

Learn it

Models read and write tokens, which are word pieces, not words. Context limits, pricing and rate limits are all counted in tokens, and some languages, including Portuguese, use more tokens per sentence than English.

Transformers and attention

Learn it

LLMs are transformer networks trained to predict the next token. Attention lets every token weigh every other token in the context. Understand the idea at a high level; you will not need to implement it.

Pre-training, instruction tuning and RLHF

Learn it

A base model learns language from a huge text corpus. Instruction tuning and preference tuning (RLHF or DPO) then turn it into an assistant that follows requests. This explains why models follow formats, refuse some tasks and sound alike.

Context window

Learn it

The maximum number of tokens a model can take in and produce in one call. Larger windows cost more per call, and quality can drop when the important part is buried in a long context.

Temperature, top-p and top-k

Learn it

Sampling settings control randomness. Use low temperature for extraction and classification, higher for brainstorming. Some reasoning models restrict or ignore these parameters, so check the provider docs.

Hallucinations

Learn it

Models can produce fluent, confident and false statements. You reduce them with grounding (retrieval), citations, constrained output and evals, but you never eliminate them, so design for checking.

Reasoning models

Learn it

Models that spend extra hidden tokens thinking before they answer. They are better at math, code and multi-step planning, and slower and more expensive. Most let you set a reasoning effort or thinking budget.

Knowledge cutoff

Good to know

A model only knows what was in its training data up to a certain date. For anything newer or private, give it the facts through retrieval or tools.

04 Use model APIs

Most AI engineering work starts with an HTTP call to a hosted model. Learn one provider well, then the others are easy.

OpenAI API

Learn it

The Responses API is OpenAI's recommended interface for new projects: text, images, tool calling, built-in tools and conversation state in one endpoint. The older Chat Completions API is still widely supported.

Anthropic Claude API

Alternative

The Messages API for Claude models, with tool use, prompt caching, extended thinking and vision. Also available through Amazon Bedrock and Google Vertex AI.

Google Gemini API

Alternative

Gemini models through Google AI Studio for quick starts or Vertex AI for enterprise use. Strong at long context and at mixing text, images, audio and video in one request.

OpenAI-compatible endpoints

Learn it

Many providers and local servers, such as Ollama, vLLM and OpenRouter, accept the Chat Completions request format. One client library can switch models by changing the base URL and model name.

OpenRouter and LiteLLM

Good to know

Gateways put many providers behind one API. OpenRouter is a hosted service with one key for hundreds of models; LiteLLM is an open-source proxy you can self-host to add budgets, fallbacks and logging.

Streaming responses

Learn it

Send tokens to the user as they are generated so the app feels fast. Handle partial output, cancellation and errors that arrive mid-stream.

Rate limits, retries and cost

Learn it

Providers cap requests and tokens per minute. Retry 429 and 5xx errors with exponential backoff, set max output tokens and track spend per feature. Batch APIs process non-urgent jobs at a discount.

API keys and secrets

Learn it

Keep keys on the server, never in front-end code or Git. Use separate keys per environment, set spending limits and rotate any key that leaks.

OpenAI Assistants API

Good to know Legacy

OpenAI deprecated it in favour of the Responses API, with a shutdown date of 26 August 2026. Assistants map to prompts, threads to conversations and runs to responses. Read it only to migrate old projects.

05 Prompt engineering

Prompts are the first lever you pull and the cheapest to change. Treat them like code: clear, reviewed, versioned and tested.

System prompts and roles

Learn it

The system (or developer) message sets the model's job, tone, rules and output format. User messages carry the request. Keep stable instructions in the system prompt.

Zero-shot and few-shot prompting

Learn it

Start with clear instructions alone. When the format or judgement is subtle, add a few worked examples; good examples often beat long explanations.

Structuring prompts with delimiters

Learn it

Separate instructions from data with XML-style tags or Markdown sections, and say exactly what to do with each part. It makes prompts easier to read and maintain.

Step-by-step reasoning

Learn it

Asking a standard model to work through a problem before answering improves accuracy on multi-step tasks. Reasoning models already do this internally, so give them goals and constraints instead of steps.

Prompt templates and versioning

Learn it

Keep prompts in files or a prompt registry, fill them with variables, and run your evals before you change one in production.

ReAct pattern

Good to know

Interleaving reasoning and actions (tool calls) in a loop, from a 2022 research paper. Most agent frameworks are built on this idea.

DSPy

Good to know

A framework that treats prompts as programs and optimises them automatically against your examples and metric. Useful once you have a solid eval set.

06 Structured output and tool calling

This is what turns a chatbot into software: answers your code can parse and actions your code can run.

Structured outputs

Learn it

Ask for output that matches a JSON Schema. The major APIs can constrain generation to a schema, and Pydantic (Python) or Zod (TypeScript) can define the schema and validate the result.

Function and tool calling

Learn it

You describe tools with a name, a description and a parameter schema. The model replies with which tool to call and the arguments; your code runs it and sends the result back. The model never executes anything itself.

Validation and repair

Learn it

Validate every model output before using it. On failure, retry with the validation error in the prompt, fall back to a simpler path, or ask the user.

Prompt caching

Learn it

Providers can reuse a long, unchanged prompt prefix between calls for lower cost and latency. Put static content (instructions, tool definitions, documents) first and variable content last.

Built-in hosted tools

Good to know

Providers offer ready-made tools such as web search, code execution and file search. They are fast to adopt, but you get less control over cost, data and behaviour.

07 Context engineering

The model only knows what is in its context window. Deciding what goes in, and what stays out, on every call is now a core skill.

What context engineering is

Learn it New in 2026

Designing everything the model sees at each step: instructions, retrieved documents, tool definitions, tool results, memory and conversation history. Prompt engineering is one part of it.

Budgeting the context window

Learn it

More context is not always better: irrelevant or contradictory text lowers quality and raises cost. Include what the current step needs, trim old tool output and keep tool lists short.

Short-term and long-term memory

Learn it

Short-term memory is the conversation history in the window. Long-term memory is facts saved outside the model, in a database or vector store, and retrieved when relevant. Decide what is safe to remember and let users delete it.

Summarising and compacting history

Learn it

Long chats and agent runs outgrow the window. Summarise older turns, keep decisions and open tasks, and drop raw tool output once it has been used.

Isolating context with sub-agents

Good to know

Give a sub-task its own fresh context, such as a research or review sub-agent, and return only a short result. It keeps the main context focused, at the cost of extra calls.

AGENTS.md and project instruction files

Good to know New in 2026

Plain Markdown files in a repository that tell coding agents how to build, test and follow the project's conventions. AGENTS.md is an open format read by many tools.

08 Choose the right model

There is no best model, only the right one for a task, a budget and a data policy. Learn to compare them on your own data.

Proprietary models: GPT, Claude, Gemini

Learn it

The most capable general models, available only through their providers' APIs and clouds. Pay per token, no infrastructure to run, and your data leaves your network, so read each provider's data-use and retention terms.

Open-weight models: Llama, Qwen, Mistral, Gemma, DeepSeek

Learn it

Models whose weights you can download and run anywhere, including OpenAI's gpt-oss. Licences differ a lot, from Apache 2.0 to custom terms with usage limits, so read the licence before you ship.

Open weights vs open source

Good to know

Releasing weights is not the same as releasing training data and code. The Open Source Initiative published an Open Source AI Definition in 2024 that most open-weight models do not fully meet.

Benchmarks and leaderboards

Learn it

Public benchmarks (MMLU-Pro, GPQA, SWE-bench) and arenas such as LMArena give a rough ranking, but they can be saturated or gamed. Your own eval set is the benchmark that matters.

Size, speed and cost trade-offs

Learn it

Small, cheap models handle classification, routing and extraction well; large or reasoning models earn their price on hard problems. Many apps route each request to the cheapest model that passes.

Model cards and licences

Learn it

Read the model card for context length, languages, training cutoff, known limits and acceptable-use policy before you commit to a model.

Small language models

Good to know

Models with a few billion parameters or fewer that run on a laptop or phone. Good for private, offline or very high-volume tasks once fine-tuned or well prompted.

09 Run open models yourself

Running models locally teaches you how inference really works, and it is the answer when data cannot leave your servers.

Hugging Face Hub and Transformers

Learn it

The main hub for open models, datasets and demos (Spaces), plus the Transformers library to load and run them in Python.

Ollama

Learn it

Download and run open models locally with one command. It exposes a local API, including an OpenAI-compatible endpoint, so your app code barely changes.

LM Studio

Alternative

A desktop app to browse, download and chat with local models, with a built-in local API server. Friendly for first experiments.

llama.cpp and GGUF

Alternative

A C/C++ inference engine that runs quantised models on CPUs, Apple Silicon and consumer GPUs. GGUF is its model file format and many local tools build on it.

Quantization

Learn it

Storing weights in 8, 4 or fewer bits instead of 16 cuts memory and speeds up inference with a small quality loss. Common formats include GGUF, AWQ and GPTQ.

vLLM and SGLang for serving

Learn it

High-throughput inference servers for GPUs with continuous batching and efficient KV-cache management, both offering OpenAI-compatible APIs. This is how open models are served to many users.

Sizing GPU memory

Learn it

Weights need roughly parameters times bytes per parameter: an 8B model at 16-bit takes about 16 GB before the KV cache, which grows with context length and concurrent users.

Hosted inference for open models

Good to know

Hugging Face Inference Providers, Together AI, Groq, Fireworks and cloud platforms run open models for you behind an API, a middle ground between closed APIs and your own GPUs.

10 Embeddings

Embeddings turn text, and also images, into vectors so you can search by meaning instead of by keyword.

What embeddings are

Learn it

An embedding model maps a piece of content to a list of numbers. Content with similar meaning ends up close together, which is what semantic search and RAG are built on.

Similarity metrics

Learn it

Cosine similarity, dot product and Euclidean distance. With normalised vectors, cosine similarity and dot product give the same ranking. Use the metric the model was trained for.

Embedding APIs

Learn it

Hosted models from OpenAI (text-embedding-3), Google, Cohere, Voyage AI and others. Compare quality on your data, price, vector size and language support.

Open embedding models

Alternative

Sentence Transformers and open models such as BGE, E5 and Nomic run on your own hardware. The MTEB leaderboard compares them across tasks.

Embedding use cases

Learn it

Semantic search, RAG, clustering, deduplication, classification with few labels, recommendations and anomaly detection.

Multilingual and multimodal embeddings

Good to know

Multilingual models place a Portuguese question near an English answer. Multimodal models (CLIP-style) embed images and text in the same space for image search.

11 Vector databases

Store embeddings and find the nearest ones fast, together with the metadata you need to filter by.

PostgreSQL with pgvector

Learn it

A Postgres extension that adds vector columns and HNSW or IVFFlat indexes. If you already run Postgres, start here: one database, transactions, joins and SQL filters.

Qdrant, Weaviate and Milvus

Alternative

Open-source, purpose-built vector databases you can self-host or use as managed services, with filtering, hybrid search and horizontal scaling.

Pinecone

Alternative

A fully managed, serverless vector database. Nothing to operate; you pay for storage and queries.

Chroma and FAISS

Good to know

Chroma is an easy embedded vector store for prototypes. FAISS is Meta's similarity-search library, not a database, that many tools use under the hood.

Search engines with vector support

Good to know

Elasticsearch, OpenSearch and MongoDB Atlas added vector search. Handy when your documents already live there.

Approximate nearest neighbour indexes

Learn it

HNSW and IVF indexes trade a little recall for a lot of speed. Learn the tuning knobs and measure recall against an exact search on a sample.

Metadata filtering

Learn it

Store source, date, tenant and permissions with each vector and filter on them in the query, so users only retrieve what they are allowed to see.

12 Retrieval-augmented generation (RAG)

Ground the model's answers in your own documents. Most enterprise AI projects are RAG at their core.

The RAG pipeline

Learn it

Ingest, parse, chunk, embed and store documents; then for each question retrieve the best chunks, put them in the prompt and generate an answer that cites its sources.

Loading and parsing documents

Learn it

PDFs, slides, HTML and tables are messy, and bad parsing caps answer quality. Tools such as Docling and Unstructured, or a vision model, extract clean text and layout.

Chunking strategies

Learn it

Split by structure (headings, sections) where you can, otherwise by size with some overlap. Chunks that are too small lose context; too big dilute relevance and waste tokens.

Hybrid search and reranking

Learn it

Combine keyword search (BM25) with vector search to catch exact names and codes, then rerank the top results with a cross-encoder or a rerank API for much better precision.

Query rewriting

Good to know

Use a model to rewrite, expand or split the user's question before searching, which helps with vague or multi-part questions.

Contextual chunk enrichment

Good to know

Before embedding a chunk, prepend a short description of where it sits in the document. It noticeably reduces retrieval failures for chunks that make little sense alone.

Agentic RAG

Learn it New in 2026

Give the model search as a tool and let it decide what to look up, check the results and search again. Better for complex questions, slower and harder to evaluate than a fixed pipeline.

GraphRAG and knowledge graphs

Good to know

Extract entities and relationships into a graph to answer questions about a whole corpus, such as main themes or how things connect, that chunk search handles poorly.

RAG vs fine-tuning vs long context

Learn it

Use RAG for knowledge that changes or must be cited, fine-tuning for behaviour, style and format, and plain long context when the whole corpus fits comfortably in the window.

LlamaIndex, LangChain and Haystack

Alternative

Frameworks with loaders, splitters, retrievers and integrations for most stores. They speed up prototypes; build a simple pipeline by hand first so you know what they do.

13 AI agents

Agents let a model plan and act through tools. Powerful, and the easiest way to build something expensive and unpredictable, so start simple.

What an agent is

Learn it

A model running in a loop: it decides on a tool call, sees the result and continues until the goal is reached or a limit is hit. A workflow follows steps you defined in code; an agent chooses its own steps.

Workflow patterns

Learn it

Prompt chaining, routing, parallel calls, orchestrator-workers and evaluator-optimizer loops. Many problems sold as agents are solved better by one of these fixed patterns.

Write an agent loop by hand

Learn it

Build one with plain tool calling: a loop, a step limit, error handling and logging. Then you will understand what every framework does for you.

OpenAI Agents SDK

Alternative

A lightweight Python and TypeScript framework for agents, handoffs between agents, guardrails and tracing.

Claude Agent SDK

Alternative

Anthropic's SDK built on the same agent harness as Claude Code, with file, shell and web tools, context management and MCP support.

Google Agent Development Kit (ADK)

Alternative

Google's open-source framework for building, evaluating and deploying agents, with a path to Vertex AI.

LangGraph

Alternative

Models agents as graphs of steps with saved state, checkpoints, retries and human-in-the-loop pauses. Good for long or branching processes.

Pydantic AI, CrewAI and Microsoft Agent Framework

Good to know

Other popular options: Pydantic AI for type-safe Python agents, CrewAI for role-based multi-agent teams, and Microsoft Agent Framework, the successor to AutoGen and Semantic Kernel.

Human-in-the-loop and approvals

Learn it

Require a person to confirm irreversible or costly actions such as payments, emails, deletes and deploys. Show what the agent wants to do and why.

Multi-agent systems

Good to know

Several agents with separate roles and contexts. They help when work splits into independent parts, and multiply cost and failure modes when it does not.

14 Model Context Protocol (MCP)

The open standard for plugging tools and data into AI applications. Write an integration once and use it from many clients.

MCP core concepts

Learn it

Introduced by Anthropic in November 2024 and now under open governance at the Linux Foundation. Hosts (AI apps) run clients that connect to servers, which expose tools, resources and prompts.

Transports: stdio and Streamable HTTP

Learn it

Local servers run as a subprocess and talk over stdio. Remote servers use Streamable HTTP, which replaced the older HTTP-plus-SSE transport in 2025.

Build an MCP server

Learn it

Use the official Python or TypeScript SDK to expose a few well-described tools over an API or database you own. Clear tool names and descriptions matter as much as the code.

Connect MCP servers to clients

Learn it

Claude, ChatGPT, VS Code with GitHub Copilot, Cursor and many other apps can act as MCP clients. Test your server with the MCP Inspector before connecting it.

MCP authorization and security

Learn it New in 2026

Remote servers use OAuth-based authorization. Treat third-party servers like any dependency: review them, scope their tokens and watch for tool descriptions that try to inject instructions.

Agent2Agent (A2A) protocol

Good to know Watch in 2027

An open protocol, started by Google and now hosted by the Linux Foundation, for agents built by different vendors to discover each other and delegate tasks. MCP connects agents to tools; A2A connects agents to agents.

15 Fine-tuning and adaptation

Change the model itself when prompting and retrieval are not enough. Less common than people expect, and very effective when it fits.

When to fine-tune

Learn it

Fine-tune for consistent style or format, a narrow task, or to make a small cheap model do a big model's job. It is a poor way to teach changing facts; use RAG for that.

Building training datasets

Learn it

A few hundred to a few thousand high-quality examples usually beat a large noisy set. Keep a held-out test set and check for personal or licensed data.

LoRA and QLoRA

Learn it

Parameter-efficient fine-tuning trains small adapter matrices instead of the whole model. QLoRA does it on a 4-bit quantised base, so a single GPU can tune a mid-sized model.

Hugging Face TRL, PEFT and Unsloth

Alternative

Open-source libraries for supervised fine-tuning, preference tuning and LoRA. Unsloth focuses on making fine-tuning faster and lighter on memory.

Hosted fine-tuning

Alternative

OpenAI, Google Vertex AI, Amazon Bedrock and others fine-tune selected models for you from an uploaded dataset. No GPUs to manage, less control.

Preference tuning (DPO)

Good to know

Train on pairs of a preferred and a rejected answer to steer tone and behaviour. Simpler and more stable than full RLHF.

Distillation

Good to know

Train a smaller model on outputs from a larger one to cut cost and latency. Check the larger model's terms of use first; some providers forbid training competing models on their outputs.

16 Evaluation

Model output is non-deterministic, so evals are your test suite. Teams that ship good AI products are the ones that measure.

Why evals come first

Learn it

Every prompt, model or retrieval change can fix one case and break three. Without evals you are judging by feel, and feel does not scale past a demo.

Build a golden dataset

Learn it

Collect real inputs with expected answers or grading criteria, including hard and adversarial cases. Grow it from production failures.

Code-based checks

Learn it

Exact match, regex, JSON schema validity, correct tool and arguments, citation present, latency and cost. Cheap, fast and reliable; use them wherever a rule can decide.

LLM-as-a-judge

Learn it

A model grades outputs against a written rubric. Check its grades against human labels, and watch for known biases toward longer answers, the first option and its own style.

RAG metrics

Learn it

Score retrieval (did the right chunks come back?) separately from generation (is the answer faithful to them and relevant to the question?).

Eval tools: promptfoo, DeepEval, RAGAS, Inspect

Alternative

Open-source frameworks for running eval suites, comparing prompts and models, and scoring RAG and agents. Pick one and wire it into your workflow.

Evals in CI

Learn it

Run the eval suite on every pull request that touches prompts, models or retrieval, and block the merge on regressions, the same way you gate on unit tests.

Evaluating agents

Good to know New in 2026

Judge the whole trajectory, not just the final answer: task success, wrong or wasted tool calls, steps and cost. Public benchmarks such as SWE-bench and tau-bench show how.

Human review and A/B tests

Good to know

Some quality only people can judge. Use expert review for high-stakes outputs and A/B tests with real users for product decisions.

17 Observability

You cannot fix what you cannot see. Trace every model and tool call in production.

Tracing LLM calls

Learn it

Record prompt, response, model, tokens, latency, cost and tool calls for every request, with nested spans so a whole agent run reads as one trace.

OpenTelemetry for GenAI

Learn it New in 2026

OpenTelemetry's GenAI semantic conventions define standard attributes for model and agent spans, so traces can go to any backend. The conventions are still evolving.

Langfuse

Alternative

Open-source LLM observability with tracing, prompt management and evals. Self-host it or use the cloud version.

LangSmith

Alternative

LangChain's platform for tracing, datasets and evals. Works with or without LangChain.

Arize Phoenix

Alternative

Open-source tracing and evaluation built on OpenTelemetry.

Cost and latency monitoring

Learn it

Track time to first token, total latency, tokens and cost per request, per user and per feature, and alert when they jump.

User feedback loops

Learn it

Capture thumbs up or down, edits and abandoned answers, link them to traces, and turn the worst ones into eval cases.

Privacy in logs

Learn it

Prompts and outputs often contain personal data. Redact what you can, restrict access to traces and set retention limits.

18 Safety, security and ethics

An LLM that can read untrusted text and call tools is a new attack surface. Design as if the model will be tricked.

Prompt injection

Learn it

Instructions hidden in user input or in content the model reads, such as web pages, emails or documents, can hijack it. There is no complete fix, so limit what a hijacked model is able to do.

OWASP Top 10 for LLM Applications

Learn it

The standard checklist of LLM risks, including prompt injection, sensitive information disclosure, excessive agency, system prompt leakage and unbounded consumption.

Least privilege for tools

Learn it

Give agents narrowly scoped credentials, read-only access by default, sandboxes for code execution, allow-lists and approval steps for anything destructive.

Guardrails and moderation

Learn it

Filter inputs and outputs for harmful content, personal data and off-topic requests with moderation APIs, classifier models such as Llama Guard, or frameworks such as NeMo Guardrails.

Data privacy

Learn it

Know what you send to each provider and how long they keep it, use enterprise or zero-retention options where needed, and follow data-protection laws such as GDPR and Brazil's LGPD.

Bias and fairness

Learn it

Test quality across languages, groups and edge cases, document known limitations and keep people involved in decisions that affect them.

AI regulation and frameworks

Learn it

The EU AI Act entered into force in 2024: its bans and general-purpose AI rules already apply, and most high-risk obligations come later. The NIST AI Risk Management Framework is a widely used voluntary guide. Know which apply to your product.

Red teaming

Good to know

Attack your own system before others do. Open-source scanners such as garak and PyRIT automate common jailbreak and injection attempts.

19 Multimodal AI

Modern models see, hear and speak. Many valuable use cases are documents, screenshots and calls, not chat.

Image and document understanding

Learn it

Send images, screenshots and PDF pages straight to a model to read text, tables and charts or describe what is there. Often simpler than a classic OCR pipeline.

Speech-to-text

Learn it

Transcribe audio with hosted APIs or open models such as Whisper. Check accuracy on your accents, languages and audio quality.

Text-to-speech

Alternative

Generate natural voices for assistants, accessibility and content. Disclose synthetic voices and never clone someone's voice without consent.

Realtime voice agents

Good to know

Speech-to-speech APIs such as OpenAI's Realtime API and Gemini's Live API hold low-latency voice conversations with tool calling.

Image generation

Good to know

Create and edit images from text with hosted models or open ones such as Stable Diffusion and FLUX. Learn the licensing and content-provenance (C2PA) basics.

Video understanding and generation

Good to know

Some models can watch and summarise video, and others generate it. Costs are high and the field moves fast; build on stable APIs.

20 Ship to production

A demo works once. Production works every day, at a known cost, when a provider is slow or down.

Backend APIs with FastAPI

Learn it

Async Python endpoints, streaming responses, request validation with Pydantic and background jobs. The most common home for AI back ends.

Streaming user interfaces

Alternative

Show tokens as they arrive, render Markdown and tool progress, and let users stop generation. The Vercel AI SDK handles much of this for TypeScript apps.

Caching

Learn it

Cache identical requests, consider a semantic cache for near-duplicates, and lean on provider prompt caching for long shared prefixes.

Routing and fallbacks

Learn it

Route requests to cheaper models when possible and fail over to another model or provider on errors or timeouts, so one outage does not take your feature down.

Amazon Bedrock, Microsoft Foundry, Google Vertex AI

Alternative

Cloud platforms offering many models with enterprise controls: private networking, IAM, regional data residency and billing on your existing cloud account.

Containers and Kubernetes

Good to know

Package services in containers, and run self-hosted models on GPU nodes in Kubernetes when you outgrow single servers.

Cost control

Learn it

Cap output tokens, trim context, use smaller models, cache, batch non-urgent work and set budgets with alerts per team and feature.

Queues and batch processing

Good to know

Move long or bulk jobs to a queue with workers and retries, and use provider batch APIs for big offline jobs at a lower price.

21 AI-assisted development

AI engineers are also heavy users of AI tools. Use them to move faster without giving up ownership of the code.

Coding agents: Claude Code, Codex, Gemini CLI

Learn it New in 2026

Agents that read your repository, edit files, run commands and tests, and open pull requests from the terminal, the IDE or the cloud.

IDE assistants: GitHub Copilot, Cursor

Alternative

Inline completions, chat and agent modes inside the editor. Most support MCP servers and project instruction files.

Reviewing AI-written code

Learn it

You own every line you merge. Read the diff, run the tests, check for invented APIs, insecure patterns and secrets, and keep changes small enough to review.

Spec-driven development

Good to know New in 2026

Write a short spec and plan first, then let the agent implement and test against it. Toolkits such as GitHub's Spec Kit formalise the workflow.

22 2027: what to watch

Not required yet. Areas that are moving fast and likely to matter more next year.

Agent interoperability standards

Good to know Watch in 2027

MCP and A2A now sit under open foundations, and more vendors build on them. Expect integrations to be written once and used across assistants, IDEs and agent platforms.

Computer-use and browser agents

Good to know Watch in 2027

Agents that operate graphical interfaces by looking at the screen and clicking. Useful for systems without APIs; still slow, costly and risky without tight sandboxes.

Agent identity and delegated access

Good to know Watch in 2027

How an agent proves who it acts for and gets narrowly scoped, short-lived permissions is becoming a core security problem for identity providers and platforms.

On-device models

Good to know Watch in 2027

Smaller models running on phones and laptops for private, offline and low-latency features, with the cloud as a fallback for hard tasks.

Long-running background agents

Good to know Watch in 2027

Agents that work for hours on a task, checkpoint their progress and report back, which raises new needs for evaluation, cost control and oversight.

AI regulation in practice

Good to know Watch in 2027

Under the EU's 2026 Digital Omnibus agreement, high-risk AI Act obligations move to December 2027 for standalone systems and August 2028 for AI inside regulated products. Expect compliance questions in product reviews, and check the final adopted dates.

~/roadmap

↑ ↓ navigate Enter open Esc close