A step-by-step path to become an AI engineer in 2026: from Python and model APIs to RAG, agents, MCP, evaluation and production, with what is new this year and what to watch in 2027. Follow the main line top to bottom; click any topic to see why it matters and mark it as done.
Follow the main line from top to bottom. Each stage builds on the ones above it. The early stages look basic, but most AI projects fail on plain engineering, not on the model.
Solid blue topics are the path. Learn those first. Outlined topics are alternatives: pick one per group. Dashed topics are good to know; come back to them when a project needs them.
Click any topic to read why it matters and where to learn more, then mark it as done. Your progress stays in this browser only.
Use the highlight buttons to see only the main path, what is new in 2026 or what to watch in 2027.
What an AI engineer does in 2026
An AI engineer builds software products on top of pre-trained foundation models. The work is mostly not training models. It is choosing a model, connecting it to the company’s data and tools, constraining what it can say and do, measuring whether it is good enough, and keeping it fast, affordable and safe in production.
Typical deliverables are a support assistant that answers from internal documentation, a pipeline that turns PDFs into structured data, an agent that files tickets or prepares pull requests, or an AI feature inside an existing app. The common thread is software engineering with a non-deterministic component in the middle.
Role
Main focus
Typical tools
AI engineer
Products built on existing models: prompts, retrieval, tools, agents, evals, production
Model APIs, vector databases, agent SDKs, MCP, eval and tracing tools
ML engineer
Training, tuning and serving models on the company’s own data, plus MLOps
PyTorch, feature stores, training pipelines, model registries, GPU serving
Data scientist
Analysis, experiments and statistical models that answer business questions
SQL, pandas, notebooks, statistics, BI tools
The boundaries are blurry and many job ads ask for a mix. This roadmap covers the AI engineer core and points out where it touches ML engineering, such as fine-tuning and self-hosted serving.
What changed recently
Context engineering became a discipline. With agents running many steps, the hard part is deciding what goes into the context window at each step, not writing one clever prompt.
MCP became the standard way to connect tools. In December 2025 Anthropic donated the Model Context Protocol to the Agentic AI Foundation under the Linux Foundation, together with OpenAI’s AGENTS.md and Block’s goose. Most major assistants and IDEs now act as MCP clients.
Agent frameworks matured. OpenAI, Anthropic and Google each ship an agent SDK, and Microsoft Agent Framework, the successor to AutoGen and Semantic Kernel, reached version 1.0 in 2026.
The Responses API replaced the Assistants API. OpenAI set 26 August 2026 as the Assistants API shutdown date, so new OpenAI projects build on Responses.
Coding agents became everyday tools. Agents that read a repository, run tests and open pull requests changed how AI engineers build, and raised the bar for reviewing generated code.
Evals and tracing are expected. Teams now ask for an eval suite and production traces before an AI feature ships, and OpenTelemetry is defining standard conventions for GenAI telemetry.
Regulation is real. The EU AI Act’s bans and general-purpose AI rules already apply, and a 2026 agreement scheduled most high-risk obligations for December 2027 and August 2028.
A realistic 12-month plan
Months
Focus
Stages
1 to 2
Python, APIs, Docker and how LLMs work
1 to 3
3 to 4
Model APIs, prompting, structured output, tool calling and context
4 to 7
5 to 6
Choosing and running models, embeddings, vector databases and RAG
8 to 12
7 to 9
Agents, MCP, evaluation and observability
13, 14, 16 and 17
10 to 12
Security, multimodal, production and fine-tuning where it pays off
15 and 18 to 20
Ongoing
AI-assisted development and the 2027 topics
21 and 22
That pace assumes a few hours most days. If you already work as a software engineer, you can move through the first stages in weeks and spend the time on evals, agents and production instead.
Build these projects along the way
A command-line chat app that streams answers, can switch between two providers and prints the token cost of every reply.
An extraction service that turns invoices or receipts (PDF or image) into JSON validated with Pydantic, with a small eval set that measures field accuracy.
A RAG assistant over a real document set, such as a product’s documentation, using PostgreSQL with pgvector, hybrid search, reranking and answers with citations.
An MCP server that exposes a database or API you own, used from an MCP client such as Claude or VS Code.
An agent that completes a multi-step task with tools, asks for approval before risky actions, is traced end to end and has its evals running in CI.
One of the above shipped as a FastAPI service in Docker, with streaming, caching, a fallback model and a monthly budget alert.
Put each project in a public repository with a README that shows your eval results and explains your trade-offs. Numbers like “faithfulness went from 0.71 to 0.89 after adding reranking” say more in an interview than a demo video.
FAQ
Do I need a degree or advanced math to become an AI engineer?
No. Most AI engineering work is software engineering, and employers mostly look for shipped projects and solid coding skills. You need math intuition for vectors, similarity and probability, not the ability to derive training algorithms. A degree helps with some employers and with research-heavy ML roles.
Python or TypeScript?
Python first: most SDKs, eval tools, data libraries and examples start there. Add TypeScript if you build web front ends or Node.js services. The concepts are the same in both, and every major provider has official SDKs for each.
Do I need a GPU?
Not to start. Hosted model APIs cover most of the roadmap for a few dollars of credit. A recent laptop runs small open models through Ollama or LM Studio. Rent a cloud GPU by the hour when you get to fine-tuning or serving with vLLM.
Should I use RAG or fine-tuning?
Start with good prompts, then RAG. Use RAG when the model needs knowledge that changes or must be cited. Fine-tune when you need a consistent behaviour, style or format, or a smaller model to match a bigger one on a narrow task. Many production systems use both.
Which agent framework should I learn?
First build a small agent loop by hand with plain tool calling, so you understand what frameworks do. Then pick the SDK that matches your main model provider or your team’s stack. The underlying skills, such as tool design, context management, evals and guardrails, transfer between frameworks.
How long does it take to become an AI engineer?
For an experienced developer, three to six months of focused practice is enough to be useful on AI projects. From zero, plan for about a year, because the software foundations take time. Real projects with measured results matter more than the calendar.
What is an eval, and why does everyone talk about them?
An eval is a repeatable test of model behaviour: a set of inputs plus a way to score the outputs, by code, by a judge model or by people. Because LLM output varies and every change can cause regressions, evals are how teams decide whether a prompt, model or retrieval change is actually better.
Is AI engineering a good career choice?
Demand for people who can turn models into reliable products has grown quickly, and the skills, from software design to evaluation and security, stay useful even as specific models and tools change. Expect the field to keep moving fast; the habit of testing new tools against your own evals is the skill that lasts.
AI engineering is software engineering first. Models are one component; the rest of the product is ordinary code that has to be reliable.
Python
Learn it
The language most AI SDKs, data tools and examples are written in. Be comfortable with functions, classes, type hints, async/await, virtual environments (uv or venv) and Pydantic for validating data.
TypeScript
Alternative
The other first-class language for AI apps. Every major provider ships an official TypeScript SDK, and it is the natural choice when the AI feature lives in a web front end or a Node.js service.
HTTP APIs and JSON
Learn it
Model providers are reached over HTTPS with JSON bodies. Know headers and auth, status codes, timeouts and Server-Sent Events, which is how most APIs stream tokens back to you.
Git and GitHub
Learn it
Prompts, eval datasets and agent configs change as often as code and belong in version control next to it.
You do not need research-level math. Vectors, dot products, cosine similarity and basic probability are enough to understand embeddings, similarity search and sampling.
02
Understand the role
Know what the job actually is before you study for it, so you spend your time on the skills employers pay for.
What an AI engineer does
Learn it
An AI engineer builds products on top of pre-trained foundation models: calling model APIs, connecting them to company data and tools, evaluating the results and running them reliably in production. Training large models from scratch is usually someone else's job.
AI engineer vs ML engineer
Learn it
ML engineers typically train, tune and serve models on a company's own data and run the MLOps around them. AI engineers mostly adapt existing models through prompting, retrieval, tools and occasional fine-tuning. The roles overlap and many teams expect a bit of both.
Day-to-day responsibilities
Learn it
Prototype quickly, measure quality with evals, ship behind feature flags, watch cost and latency, and work with product, data and security people on what the model is allowed to do.
Designing AI features
Learn it
Model output is probabilistic and sometimes wrong. Good AI features show sources, let people correct or undo, keep a human in the loop for risky actions and degrade gracefully when the model or provider fails.
Machine learning basics
Good to know
Supervised vs unsupervised learning, training vs test data, overfitting and classic metrics like precision and recall. It helps you talk to data scientists and know when a simple classifier beats an LLM.
03
How LLMs work
A working mental model of large language models explains most of their strange behaviour and most of your bills.
Tokens and tokenization
Learn it
Models read and write tokens, which are word pieces, not words. Context limits, pricing and rate limits are all counted in tokens, and some languages, including Portuguese, use more tokens per sentence than English.
Transformers and attention
Learn it
LLMs are transformer networks trained to predict the next token. Attention lets every token weigh every other token in the context. Understand the idea at a high level; you will not need to implement it.
Pre-training, instruction tuning and RLHF
Learn it
A base model learns language from a huge text corpus. Instruction tuning and preference tuning (RLHF or DPO) then turn it into an assistant that follows requests. This explains why models follow formats, refuse some tasks and sound alike.
Context window
Learn it
The maximum number of tokens a model can take in and produce in one call. Larger windows cost more per call, and quality can drop when the important part is buried in a long context.
Temperature, top-p and top-k
Learn it
Sampling settings control randomness. Use low temperature for extraction and classification, higher for brainstorming. Some reasoning models restrict or ignore these parameters, so check the provider docs.
Hallucinations
Learn it
Models can produce fluent, confident and false statements. You reduce them with grounding (retrieval), citations, constrained output and evals, but you never eliminate them, so design for checking.
Reasoning models
Learn it
Models that spend extra hidden tokens thinking before they answer. They are better at math, code and multi-step planning, and slower and more expensive. Most let you set a reasoning effort or thinking budget.
Knowledge cutoff
Good to know
A model only knows what was in its training data up to a certain date. For anything newer or private, give it the facts through retrieval or tools.
04
Use model APIs
Most AI engineering work starts with an HTTP call to a hosted model. Learn one provider well, then the others are easy.
OpenAI API
Learn it
The Responses API is OpenAI's recommended interface for new projects: text, images, tool calling, built-in tools and conversation state in one endpoint. The older Chat Completions API is still widely supported.
Anthropic Claude API
Alternative
The Messages API for Claude models, with tool use, prompt caching, extended thinking and vision. Also available through Amazon Bedrock and Google Vertex AI.
Google Gemini API
Alternative
Gemini models through Google AI Studio for quick starts or Vertex AI for enterprise use. Strong at long context and at mixing text, images, audio and video in one request.
OpenAI-compatible endpoints
Learn it
Many providers and local servers, such as Ollama, vLLM and OpenRouter, accept the Chat Completions request format. One client library can switch models by changing the base URL and model name.
OpenRouter and LiteLLM
Good to know
Gateways put many providers behind one API. OpenRouter is a hosted service with one key for hundreds of models; LiteLLM is an open-source proxy you can self-host to add budgets, fallbacks and logging.
Streaming responses
Learn it
Send tokens to the user as they are generated so the app feels fast. Handle partial output, cancellation and errors that arrive mid-stream.
Rate limits, retries and cost
Learn it
Providers cap requests and tokens per minute. Retry 429 and 5xx errors with exponential backoff, set max output tokens and track spend per feature. Batch APIs process non-urgent jobs at a discount.
API keys and secrets
Learn it
Keep keys on the server, never in front-end code or Git. Use separate keys per environment, set spending limits and rotate any key that leaks.
OpenAI Assistants API
Good to knowLegacy
OpenAI deprecated it in favour of the Responses API, with a shutdown date of 26 August 2026. Assistants map to prompts, threads to conversations and runs to responses. Read it only to migrate old projects.
05
Prompt engineering
Prompts are the first lever you pull and the cheapest to change. Treat them like code: clear, reviewed, versioned and tested.
System prompts and roles
Learn it
The system (or developer) message sets the model's job, tone, rules and output format. User messages carry the request. Keep stable instructions in the system prompt.
Zero-shot and few-shot prompting
Learn it
Start with clear instructions alone. When the format or judgement is subtle, add a few worked examples; good examples often beat long explanations.
Structuring prompts with delimiters
Learn it
Separate instructions from data with XML-style tags or Markdown sections, and say exactly what to do with each part. It makes prompts easier to read and maintain.
Step-by-step reasoning
Learn it
Asking a standard model to work through a problem before answering improves accuracy on multi-step tasks. Reasoning models already do this internally, so give them goals and constraints instead of steps.
Prompt templates and versioning
Learn it
Keep prompts in files or a prompt registry, fill them with variables, and run your evals before you change one in production.
ReAct pattern
Good to know
Interleaving reasoning and actions (tool calls) in a loop, from a 2022 research paper. Most agent frameworks are built on this idea.
DSPy
Good to know
A framework that treats prompts as programs and optimises them automatically against your examples and metric. Useful once you have a solid eval set.
06
Structured output and tool calling
This is what turns a chatbot into software: answers your code can parse and actions your code can run.
Structured outputs
Learn it
Ask for output that matches a JSON Schema. The major APIs can constrain generation to a schema, and Pydantic (Python) or Zod (TypeScript) can define the schema and validate the result.
Function and tool calling
Learn it
You describe tools with a name, a description and a parameter schema. The model replies with which tool to call and the arguments; your code runs it and sends the result back. The model never executes anything itself.
Validation and repair
Learn it
Validate every model output before using it. On failure, retry with the validation error in the prompt, fall back to a simpler path, or ask the user.
Prompt caching
Learn it
Providers can reuse a long, unchanged prompt prefix between calls for lower cost and latency. Put static content (instructions, tool definitions, documents) first and variable content last.
Built-in hosted tools
Good to know
Providers offer ready-made tools such as web search, code execution and file search. They are fast to adopt, but you get less control over cost, data and behaviour.
07
Context engineering
The model only knows what is in its context window. Deciding what goes in, and what stays out, on every call is now a core skill.
What context engineering is
Learn itNew in 2026
Designing everything the model sees at each step: instructions, retrieved documents, tool definitions, tool results, memory and conversation history. Prompt engineering is one part of it.
Budgeting the context window
Learn it
More context is not always better: irrelevant or contradictory text lowers quality and raises cost. Include what the current step needs, trim old tool output and keep tool lists short.
Short-term and long-term memory
Learn it
Short-term memory is the conversation history in the window. Long-term memory is facts saved outside the model, in a database or vector store, and retrieved when relevant. Decide what is safe to remember and let users delete it.
Summarising and compacting history
Learn it
Long chats and agent runs outgrow the window. Summarise older turns, keep decisions and open tasks, and drop raw tool output once it has been used.
Isolating context with sub-agents
Good to know
Give a sub-task its own fresh context, such as a research or review sub-agent, and return only a short result. It keeps the main context focused, at the cost of extra calls.
AGENTS.md and project instruction files
Good to knowNew in 2026
Plain Markdown files in a repository that tell coding agents how to build, test and follow the project's conventions. AGENTS.md is an open format read by many tools.
There is no best model, only the right one for a task, a budget and a data policy. Learn to compare them on your own data.
Proprietary models: GPT, Claude, Gemini
Learn it
The most capable general models, available only through their providers' APIs and clouds. Pay per token, no infrastructure to run, and your data leaves your network, so read each provider's data-use and retention terms.
Models whose weights you can download and run anywhere, including OpenAI's gpt-oss. Licences differ a lot, from Apache 2.0 to custom terms with usage limits, so read the licence before you ship.
Open weights vs open source
Good to know
Releasing weights is not the same as releasing training data and code. The Open Source Initiative published an Open Source AI Definition in 2024 that most open-weight models do not fully meet.
Benchmarks and leaderboards
Learn it
Public benchmarks (MMLU-Pro, GPQA, SWE-bench) and arenas such as LMArena give a rough ranking, but they can be saturated or gamed. Your own eval set is the benchmark that matters.
Size, speed and cost trade-offs
Learn it
Small, cheap models handle classification, routing and extraction well; large or reasoning models earn their price on hard problems. Many apps route each request to the cheapest model that passes.
Model cards and licences
Learn it
Read the model card for context length, languages, training cutoff, known limits and acceptable-use policy before you commit to a model.
Small language models
Good to know
Models with a few billion parameters or fewer that run on a laptop or phone. Good for private, offline or very high-volume tasks once fine-tuned or well prompted.
09
Run open models yourself
Running models locally teaches you how inference really works, and it is the answer when data cannot leave your servers.
Hugging Face Hub and Transformers
Learn it
The main hub for open models, datasets and demos (Spaces), plus the Transformers library to load and run them in Python.
Download and run open models locally with one command. It exposes a local API, including an OpenAI-compatible endpoint, so your app code barely changes.
A desktop app to browse, download and chat with local models, with a built-in local API server. Friendly for first experiments.
llama.cpp and GGUF
Alternative
A C/C++ inference engine that runs quantised models on CPUs, Apple Silicon and consumer GPUs. GGUF is its model file format and many local tools build on it.
Quantization
Learn it
Storing weights in 8, 4 or fewer bits instead of 16 cuts memory and speeds up inference with a small quality loss. Common formats include GGUF, AWQ and GPTQ.
vLLM and SGLang for serving
Learn it
High-throughput inference servers for GPUs with continuous batching and efficient KV-cache management, both offering OpenAI-compatible APIs. This is how open models are served to many users.
Weights need roughly parameters times bytes per parameter: an 8B model at 16-bit takes about 16 GB before the KV cache, which grows with context length and concurrent users.
Hosted inference for open models
Good to know
Hugging Face Inference Providers, Together AI, Groq, Fireworks and cloud platforms run open models for you behind an API, a middle ground between closed APIs and your own GPUs.
10
Embeddings
Embeddings turn text, and also images, into vectors so you can search by meaning instead of by keyword.
What embeddings are
Learn it
An embedding model maps a piece of content to a list of numbers. Content with similar meaning ends up close together, which is what semantic search and RAG are built on.
Similarity metrics
Learn it
Cosine similarity, dot product and Euclidean distance. With normalised vectors, cosine similarity and dot product give the same ranking. Use the metric the model was trained for.
Embedding APIs
Learn it
Hosted models from OpenAI (text-embedding-3), Google, Cohere, Voyage AI and others. Compare quality on your data, price, vector size and language support.
Open embedding models
Alternative
Sentence Transformers and open models such as BGE, E5 and Nomic run on your own hardware. The MTEB leaderboard compares them across tasks.
Embedding use cases
Learn it
Semantic search, RAG, clustering, deduplication, classification with few labels, recommendations and anomaly detection.
Multilingual and multimodal embeddings
Good to know
Multilingual models place a Portuguese question near an English answer. Multimodal models (CLIP-style) embed images and text in the same space for image search.
11
Vector databases
Store embeddings and find the nearest ones fast, together with the metadata you need to filter by.
PostgreSQL with pgvector
Learn it
A Postgres extension that adds vector columns and HNSW or IVFFlat indexes. If you already run Postgres, start here: one database, transactions, joins and SQL filters.
Open-source, purpose-built vector databases you can self-host or use as managed services, with filtering, hybrid search and horizontal scaling.
Pinecone
Alternative
A fully managed, serverless vector database. Nothing to operate; you pay for storage and queries.
Chroma and FAISS
Good to know
Chroma is an easy embedded vector store for prototypes. FAISS is Meta's similarity-search library, not a database, that many tools use under the hood.
Search engines with vector support
Good to know
Elasticsearch, OpenSearch and MongoDB Atlas added vector search. Handy when your documents already live there.
Approximate nearest neighbour indexes
Learn it
HNSW and IVF indexes trade a little recall for a lot of speed. Learn the tuning knobs and measure recall against an exact search on a sample.
Metadata filtering
Learn it
Store source, date, tenant and permissions with each vector and filter on them in the query, so users only retrieve what they are allowed to see.
12
Retrieval-augmented generation (RAG)
Ground the model's answers in your own documents. Most enterprise AI projects are RAG at their core.
The RAG pipeline
Learn it
Ingest, parse, chunk, embed and store documents; then for each question retrieve the best chunks, put them in the prompt and generate an answer that cites its sources.
Loading and parsing documents
Learn it
PDFs, slides, HTML and tables are messy, and bad parsing caps answer quality. Tools such as Docling and Unstructured, or a vision model, extract clean text and layout.
Chunking strategies
Learn it
Split by structure (headings, sections) where you can, otherwise by size with some overlap. Chunks that are too small lose context; too big dilute relevance and waste tokens.
Hybrid search and reranking
Learn it
Combine keyword search (BM25) with vector search to catch exact names and codes, then rerank the top results with a cross-encoder or a rerank API for much better precision.
Query rewriting
Good to know
Use a model to rewrite, expand or split the user's question before searching, which helps with vague or multi-part questions.
Contextual chunk enrichment
Good to know
Before embedding a chunk, prepend a short description of where it sits in the document. It noticeably reduces retrieval failures for chunks that make little sense alone.
Agentic RAG
Learn itNew in 2026
Give the model search as a tool and let it decide what to look up, check the results and search again. Better for complex questions, slower and harder to evaluate than a fixed pipeline.
GraphRAG and knowledge graphs
Good to know
Extract entities and relationships into a graph to answer questions about a whole corpus, such as main themes or how things connect, that chunk search handles poorly.
RAG vs fine-tuning vs long context
Learn it
Use RAG for knowledge that changes or must be cited, fine-tuning for behaviour, style and format, and plain long context when the whole corpus fits comfortably in the window.
LlamaIndex, LangChain and Haystack
Alternative
Frameworks with loaders, splitters, retrievers and integrations for most stores. They speed up prototypes; build a simple pipeline by hand first so you know what they do.
13
AI agents
Agents let a model plan and act through tools. Powerful, and the easiest way to build something expensive and unpredictable, so start simple.
What an agent is
Learn it
A model running in a loop: it decides on a tool call, sees the result and continues until the goal is reached or a limit is hit. A workflow follows steps you defined in code; an agent chooses its own steps.
Prompt chaining, routing, parallel calls, orchestrator-workers and evaluator-optimizer loops. Many problems sold as agents are solved better by one of these fixed patterns.
Write an agent loop by hand
Learn it
Build one with plain tool calling: a loop, a step limit, error handling and logging. Then you will understand what every framework does for you.
OpenAI Agents SDK
Alternative
A lightweight Python and TypeScript framework for agents, handoffs between agents, guardrails and tracing.
Claude Agent SDK
Alternative
Anthropic's SDK built on the same agent harness as Claude Code, with file, shell and web tools, context management and MCP support.
Google Agent Development Kit (ADK)
Alternative
Google's open-source framework for building, evaluating and deploying agents, with a path to Vertex AI.
LangGraph
Alternative
Models agents as graphs of steps with saved state, checkpoints, retries and human-in-the-loop pauses. Good for long or branching processes.
Pydantic AI, CrewAI and Microsoft Agent Framework
Good to know
Other popular options: Pydantic AI for type-safe Python agents, CrewAI for role-based multi-agent teams, and Microsoft Agent Framework, the successor to AutoGen and Semantic Kernel.
Human-in-the-loop and approvals
Learn it
Require a person to confirm irreversible or costly actions such as payments, emails, deletes and deploys. Show what the agent wants to do and why.
Multi-agent systems
Good to know
Several agents with separate roles and contexts. They help when work splits into independent parts, and multiply cost and failure modes when it does not.
14
Model Context Protocol (MCP)
The open standard for plugging tools and data into AI applications. Write an integration once and use it from many clients.
MCP core concepts
Learn it
Introduced by Anthropic in November 2024 and now under open governance at the Linux Foundation. Hosts (AI apps) run clients that connect to servers, which expose tools, resources and prompts.
Local servers run as a subprocess and talk over stdio. Remote servers use Streamable HTTP, which replaced the older HTTP-plus-SSE transport in 2025.
Build an MCP server
Learn it
Use the official Python or TypeScript SDK to expose a few well-described tools over an API or database you own. Clear tool names and descriptions matter as much as the code.
Connect MCP servers to clients
Learn it
Claude, ChatGPT, VS Code with GitHub Copilot, Cursor and many other apps can act as MCP clients. Test your server with the MCP Inspector before connecting it.
MCP authorization and security
Learn itNew in 2026
Remote servers use OAuth-based authorization. Treat third-party servers like any dependency: review them, scope their tokens and watch for tool descriptions that try to inject instructions.
Agent2Agent (A2A) protocol
Good to knowWatch in 2027
An open protocol, started by Google and now hosted by the Linux Foundation, for agents built by different vendors to discover each other and delegate tasks. MCP connects agents to tools; A2A connects agents to agents.
15
Fine-tuning and adaptation
Change the model itself when prompting and retrieval are not enough. Less common than people expect, and very effective when it fits.
When to fine-tune
Learn it
Fine-tune for consistent style or format, a narrow task, or to make a small cheap model do a big model's job. It is a poor way to teach changing facts; use RAG for that.
Building training datasets
Learn it
A few hundred to a few thousand high-quality examples usually beat a large noisy set. Keep a held-out test set and check for personal or licensed data.
LoRA and QLoRA
Learn it
Parameter-efficient fine-tuning trains small adapter matrices instead of the whole model. QLoRA does it on a 4-bit quantised base, so a single GPU can tune a mid-sized model.
Hugging Face TRL, PEFT and Unsloth
Alternative
Open-source libraries for supervised fine-tuning, preference tuning and LoRA. Unsloth focuses on making fine-tuning faster and lighter on memory.
Hosted fine-tuning
Alternative
OpenAI, Google Vertex AI, Amazon Bedrock and others fine-tune selected models for you from an uploaded dataset. No GPUs to manage, less control.
Preference tuning (DPO)
Good to know
Train on pairs of a preferred and a rejected answer to steer tone and behaviour. Simpler and more stable than full RLHF.
Distillation
Good to know
Train a smaller model on outputs from a larger one to cut cost and latency. Check the larger model's terms of use first; some providers forbid training competing models on their outputs.
16
Evaluation
Model output is non-deterministic, so evals are your test suite. Teams that ship good AI products are the ones that measure.
Why evals come first
Learn it
Every prompt, model or retrieval change can fix one case and break three. Without evals you are judging by feel, and feel does not scale past a demo.
Build a golden dataset
Learn it
Collect real inputs with expected answers or grading criteria, including hard and adversarial cases. Grow it from production failures.
Code-based checks
Learn it
Exact match, regex, JSON schema validity, correct tool and arguments, citation present, latency and cost. Cheap, fast and reliable; use them wherever a rule can decide.
LLM-as-a-judge
Learn it
A model grades outputs against a written rubric. Check its grades against human labels, and watch for known biases toward longer answers, the first option and its own style.
RAG metrics
Learn it
Score retrieval (did the right chunks come back?) separately from generation (is the answer faithful to them and relevant to the question?).
Eval tools: promptfoo, DeepEval, RAGAS, Inspect
Alternative
Open-source frameworks for running eval suites, comparing prompts and models, and scoring RAG and agents. Pick one and wire it into your workflow.
Evals in CI
Learn it
Run the eval suite on every pull request that touches prompts, models or retrieval, and block the merge on regressions, the same way you gate on unit tests.
Judge the whole trajectory, not just the final answer: task success, wrong or wasted tool calls, steps and cost. Public benchmarks such as SWE-bench and tau-bench show how.
Human review and A/B tests
Good to know
Some quality only people can judge. Use expert review for high-stakes outputs and A/B tests with real users for product decisions.
17
Observability
You cannot fix what you cannot see. Trace every model and tool call in production.
Tracing LLM calls
Learn it
Record prompt, response, model, tokens, latency, cost and tool calls for every request, with nested spans so a whole agent run reads as one trace.
OpenTelemetry for GenAI
Learn itNew in 2026
OpenTelemetry's GenAI semantic conventions define standard attributes for model and agent spans, so traces can go to any backend. The conventions are still evolving.
Open-source LLM observability with tracing, prompt management and evals. Self-host it or use the cloud version.
LangSmith
Alternative
LangChain's platform for tracing, datasets and evals. Works with or without LangChain.
Arize Phoenix
Alternative
Open-source tracing and evaluation built on OpenTelemetry.
Cost and latency monitoring
Learn it
Track time to first token, total latency, tokens and cost per request, per user and per feature, and alert when they jump.
User feedback loops
Learn it
Capture thumbs up or down, edits and abandoned answers, link them to traces, and turn the worst ones into eval cases.
Privacy in logs
Learn it
Prompts and outputs often contain personal data. Redact what you can, restrict access to traces and set retention limits.
18
Safety, security and ethics
An LLM that can read untrusted text and call tools is a new attack surface. Design as if the model will be tricked.
Prompt injection
Learn it
Instructions hidden in user input or in content the model reads, such as web pages, emails or documents, can hijack it. There is no complete fix, so limit what a hijacked model is able to do.
OWASP Top 10 for LLM Applications
Learn it
The standard checklist of LLM risks, including prompt injection, sensitive information disclosure, excessive agency, system prompt leakage and unbounded consumption.
Give agents narrowly scoped credentials, read-only access by default, sandboxes for code execution, allow-lists and approval steps for anything destructive.
Guardrails and moderation
Learn it
Filter inputs and outputs for harmful content, personal data and off-topic requests with moderation APIs, classifier models such as Llama Guard, or frameworks such as NeMo Guardrails.
Data privacy
Learn it
Know what you send to each provider and how long they keep it, use enterprise or zero-retention options where needed, and follow data-protection laws such as GDPR and Brazil's LGPD.
Bias and fairness
Learn it
Test quality across languages, groups and edge cases, document known limitations and keep people involved in decisions that affect them.
AI regulation and frameworks
Learn it
The EU AI Act entered into force in 2024: its bans and general-purpose AI rules already apply, and most high-risk obligations come later. The NIST AI Risk Management Framework is a widely used voluntary guide. Know which apply to your product.
Attack your own system before others do. Open-source scanners such as garak and PyRIT automate common jailbreak and injection attempts.
19
Multimodal AI
Modern models see, hear and speak. Many valuable use cases are documents, screenshots and calls, not chat.
Image and document understanding
Learn it
Send images, screenshots and PDF pages straight to a model to read text, tables and charts or describe what is there. Often simpler than a classic OCR pipeline.
Speech-to-text
Learn it
Transcribe audio with hosted APIs or open models such as Whisper. Check accuracy on your accents, languages and audio quality.
Text-to-speech
Alternative
Generate natural voices for assistants, accessibility and content. Disclose synthetic voices and never clone someone's voice without consent.
Realtime voice agents
Good to know
Speech-to-speech APIs such as OpenAI's Realtime API and Gemini's Live API hold low-latency voice conversations with tool calling.
Image generation
Good to know
Create and edit images from text with hosted models or open ones such as Stable Diffusion and FLUX. Learn the licensing and content-provenance (C2PA) basics.
Video understanding and generation
Good to know
Some models can watch and summarise video, and others generate it. Costs are high and the field moves fast; build on stable APIs.
20
Ship to production
A demo works once. Production works every day, at a known cost, when a provider is slow or down.
Backend APIs with FastAPI
Learn it
Async Python endpoints, streaming responses, request validation with Pydantic and background jobs. The most common home for AI back ends.
Streaming user interfaces
Alternative
Show tokens as they arrive, render Markdown and tool progress, and let users stop generation. The Vercel AI SDK handles much of this for TypeScript apps.
Caching
Learn it
Cache identical requests, consider a semantic cache for near-duplicates, and lean on provider prompt caching for long shared prefixes.
Routing and fallbacks
Learn it
Route requests to cheaper models when possible and fail over to another model or provider on errors or timeouts, so one outage does not take your feature down.
Amazon Bedrock, Microsoft Foundry, Google Vertex AI
Alternative
Cloud platforms offering many models with enterprise controls: private networking, IAM, regional data residency and billing on your existing cloud account.
Containers and Kubernetes
Good to know
Package services in containers, and run self-hosted models on GPU nodes in Kubernetes when you outgrow single servers.
Cap output tokens, trim context, use smaller models, cache, batch non-urgent work and set budgets with alerts per team and feature.
Queues and batch processing
Good to know
Move long or bulk jobs to a queue with workers and retries, and use provider batch APIs for big offline jobs at a lower price.
21
AI-assisted development
AI engineers are also heavy users of AI tools. Use them to move faster without giving up ownership of the code.
Coding agents: Claude Code, Codex, Gemini CLI
Learn itNew in 2026
Agents that read your repository, edit files, run commands and tests, and open pull requests from the terminal, the IDE or the cloud.
IDE assistants: GitHub Copilot, Cursor
Alternative
Inline completions, chat and agent modes inside the editor. Most support MCP servers and project instruction files.
Reviewing AI-written code
Learn it
You own every line you merge. Read the diff, run the tests, check for invented APIs, insecure patterns and secrets, and keep changes small enough to review.
Spec-driven development
Good to knowNew in 2026
Write a short spec and plan first, then let the agent implement and test against it. Toolkits such as GitHub's Spec Kit formalise the workflow.
22
2027: what to watch
Not required yet. Areas that are moving fast and likely to matter more next year.
Agent interoperability standards
Good to knowWatch in 2027
MCP and A2A now sit under open foundations, and more vendors build on them. Expect integrations to be written once and used across assistants, IDEs and agent platforms.
Computer-use and browser agents
Good to knowWatch in 2027
Agents that operate graphical interfaces by looking at the screen and clicking. Useful for systems without APIs; still slow, costly and risky without tight sandboxes.
Agent identity and delegated access
Good to knowWatch in 2027
How an agent proves who it acts for and gets narrowly scoped, short-lived permissions is becoming a core security problem for identity providers and platforms.
On-device models
Good to knowWatch in 2027
Smaller models running on phones and laptops for private, offline and low-latency features, with the cloud as a fallback for hard tasks.
Long-running background agents
Good to knowWatch in 2027
Agents that work for hours on a task, checkpoint their progress and report back, which raises new needs for evaluation, cost control and oversight.
AI regulation in practice
Good to knowWatch in 2027
Under the EU's 2026 Digital Omnibus agreement, high-risk AI Act obligations move to December 2027 for standalone systems and August 2028 for AI inside regulated products. Expect compliance questions in product reviews, and check the final adopted dates.