A. Roadmap overview
In 12 weeks (about 160 hours) this roadmap takes you from "I can build a FastAPI backend" to being able to build, evaluate, secure and deploy an AI system for a real use case. That is the profile hiring teams describe for AI application engineers, applied AI engineers and forward deployed (FDE) engineers at SDE-1 level. For each layer you will be able to say what it does, how it fails, how to measure it and how to explain it to a customer.
| Item | Plan |
|---|---|
| Duration | 12 weeks (about 3 months) |
| Daily time | About 2 hours, six learning days plus one review day per week |
| Total | 72 learning days, 12 review days, about 160 hours |
| Structure | 4 levels, 10 phases, 6 projects, 4 level checkpoints |
| Entry point | The outcomes of the Python roadmap: Python, FastAPI, SQL, Git, Docker, pytest |
| Exit point | Interview-ready for SDE-1 AI application engineer, applied AI engineer and associate or junior FDE roles |
After completing it you can:
- Call an LLM API from production code with streaming, retries, timeouts, cost tracking and validated structured output.
- Design tools for a model, run the tool loop safely, and build an MCP server that exposes your own tools.
- Build a RAG system that is measured, not guessed: chunking, hybrid search, reranking, permissions and retrieval metrics.
- Build and run an evaluation suite (golden dataset, LLM-as-judge, regression gate in CI) and use it to decide whether a change helped.
- Defend a system against prompt injection, data leaks and runaway cost, and explain residual risk honestly.
- Trace a bad answer back to its cause: retrieval, prompt, model, tool or data.
- Take a vague customer request, scope it, build the smallest useful version, demo it and plan its rollout.
- Use a coding agent as a fast assistant while still owning every line that ships.
Which roles this targets
| Role | What the job is | What it leans on in this roadmap |
|---|---|---|
| AI application engineer / applied AI engineer | Builds LLM-powered features inside a product: RAG, tools, agents, evals | Phases 2 to 9 |
| Forward deployed engineer (FDE) | Embeds with a customer, understands their data and constraints, builds and deploys AI solutions that work in their environment | Phases 4 to 6 and 10, plus customer-facing practice |
| Agent engineer | Designs reliable multi-step agents with tools and guardrails | Phases 3, 7, 8 and 9 |
A realistic note about entry level. Many FDE postings prefer candidates with prior engineering or consulting experience, but associate and junior FDE roles exist, and AI application engineer roles are open to SDE-1 profiles. The strongest SDE-1 evidence is deployed projects with measured results, not course certificates.
How the "in-demand" list was chosen
The skills here come from 2026 hiring guides, role descriptions and job posting analyses for AI engineers and FDEs. They agree on a stable core: strong Python and backend skills; LLM APIs with structured output and tool use; RAG and vector search; agents and MCP; evals; observability and guardrails; cost and latency control; and customer-facing skills. Many of those sources are hiring or training blogs, so treat their numbers as directional. Week 1 includes a task to collect 20 real job descriptions from companies you want to join and tally their requirements. Adjust your project choices to that tally.
How to use this roadmap
- Evals first. From week 3, every feature you build gets at least five test cases with expected behavior before you tune a prompt. A change without a before and after score is a guess.
- Use messy data. Real customers have scanned PDFs, duplicate rows and inconsistent names. Build on messy public datasets, not clean toy ones.
- Keep a failure journal. After each bad output, write four lines: input, wrong output, cause (retrieval, prompt, model, tool, data), fix. By week 12 you will have 60 or more entries. This is your best interview material.
- Raw API before frameworks. Build the first version of each pattern with the provider's SDK and plain Python. Use a framework only after you can explain what it hides.
- Explain out loud. Finish each day with a two-minute explanation without notes, as if to a non-technical customer.
- Use a coding agent with discipline. Ask it to scaffold, explain and review. Read every diff, run the tests, and never accept code you cannot explain.
- Use official docs as the source of truth. Model names, SDK signatures and pricing change often. Your provider's documentation beats any blog, including this one.
Deliberate design decisions
- Workflows before agents. Most production "agents" are better as fixed workflows with a model step inside. You learn workflows first (week 7) and agents second, and you learn when not to use an agent.
- Retrieval before fine-tuning. Most customer problems are data access problems, not model skill problems. Fine-tuning is awareness only.
- Evals are built in, not bolted on. Retrieval metrics appear in week 5, a formal eval harness in week 9, and an eval gate in CI before the capstone.
- Safety is a thread, not a chapter. Prompt injection appears in weeks 3, 5, 7 and 10 because it shows up in every layer.
- Customer skills are practiced, not read. From week 6 you run discovery conversations, write scoping notes and record demos.
- One provider SDK in depth, a second in awareness. Provider-neutral skills (schemas, evals, retrieval) matter more than API quirks.
Time budget by level
| Level | Weeks | Share of time | Outcome |
|---|---|---|---|
| 1. LLM Foundations for Engineers | 1 to 3 | 25% | LLM APIs, structured output, tools, context |
| 2. Retrieval and Data | 4 to 6 | 25% | RAG, production retrieval, messy data, enterprise integration |
| 3. Agents, Evals and Production | 7 to 10 | 33% | Agents, MCP, evals, observability, safety, cost, deployment |
| 4. Customer-Facing Work and Capstone | 11 to 12 | 17% | Discovery, scoping, demo, capstone, interviews |
Prerequisites
You should be able to do the following before starting. If one is weak, repeat that part of the Python roadmap first.
| Skill | Minimum bar | If weak |
|---|---|---|
| Python | Classes, dataclasses, generators, type hints, virtual environments | Python roadmap, weeks 1 to 4 |
| Async Python | Explain async def and run concurrent calls with asyncio.gather |
Python roadmap, week 4 and week 9 |
| FastAPI | Build an API with validation, errors and tests | Python roadmap, weeks 8 to 9 |
| SQL and a database | Joins, indexes, SQLAlchemy basics | Python roadmap, week 10 |
| Git, Docker, CI | Run an app with Docker Compose; a pipeline that runs tests | Python roadmap, week 11 |
| HTTP and JSON | Timeouts, retries, status codes | Python roadmap, weeks 4 and 9 |
B. Complete roadmap
Each phase has the same shape: goal, a table, production scenarios, interview questions and exit criteria. Every row follows the loop learn, understand, build, break it, measure. The "Role" column shows who values it most: Both, AI dev or FDE.
Level 1: LLM Foundations for Engineers (weeks 1 to 3)
Phase 1: How LLMs behave, for engineers (week 1)
Goal: Understand a model well enough to predict its failures, its cost and its latency. You do not need the math of training.
| Week | Topic | What to learn | Why it matters | Practical task | Role |
|---|---|---|---|---|---|
| 1 | Tokens and context | Tokenization; context window; input vs output tokens; why long inputs cost more and are slower | Cost, latency and truncation bugs all start here | Count tokens for 10 documents; build a cost estimator for 1,000 requests | Both |
| 1 | How generation works | Next-token prediction; sampling and temperature; why outputs differ between runs; stop reasons | Explains non-determinism and "creative" errors | Run one prompt 20 times at three temperatures; compare outputs | Both |
| 1 | Training stages (awareness) | Pretraining, post-training and instruction tuning at a high level; knowledge cutoff; what a model can and cannot know | Explains hallucination and stale knowledge | Ask a model about a recent event and about a private fact; record the failures | Both |
| 1 | Model selection | Small and fast vs large and capable tiers; latency, cost and quality trade-offs; multimodal inputs; open-weight vs hosted | Choosing the cheapest model that passes your evals is a core skill | Run the same 15 prompts on two tiers; record quality, cost and latency | Both |
| 1 | API anatomy | Messages, system prompt, roles, response object, usage fields, rate limits, error types | You will debug these daily | Write a thin script that sends a prompt and prints the usage and stop reason | Both |
| 1 | Reading the market | Collecting and tallying job descriptions | Aims your learning at real demand | Collect 20 job descriptions; tally skills; write a one-page summary | Both |
Production scenarios: a prompt works in testing but is cut off in production because the input exceeded the context window; a monthly bill is ten times the estimate because nobody counted output tokens.
Interview questions:
- Concept: What is a token? What is a context window?
- Why: Why can the same prompt give different answers? When do you want that?
- Internals: Why does an LLM hallucinate, and what can you do about it?
- Scenario: Latency doubled after a release that added more context to the prompt. Why?
- Design: How would you pick a model for a support-ticket classifier?
Exit criteria: you can estimate the cost and latency of a feature before building it, explain hallucination in two sentences, and justify a model choice with data.
Phase 2: LLM API engineering (week 2)
Goal: Wrap a model call in the engineering a production service needs. Project 1 runs through weeks 1 to 2.
| Week | Topic | What to learn | Why it matters | Practical task | Role |
|---|---|---|---|---|---|
| 2 | Prompt design | Clear instructions, role and context, examples (few-shot), delimiters or XML-style sections, asking for reasoning when it helps, saying what to do instead of what not to do | Most quality gains come from clearer prompts | Rewrite one weak prompt five times; score each version on 10 cases | Both |
| 2 | Structured output | JSON schemas; validating with Pydantic; retry with the validation error; handling refusals and truncation | Downstream code needs predictable data | Extract fields from 30 messy emails into a Pydantic model; measure the valid-output rate | Both |
| 2 | Streaming | Server-sent events; streaming through FastAPI; handling disconnects and partial output | Users judge speed by time to first token | Stream an answer from a FastAPI endpoint; handle a client that disconnects | AI dev |
| 2 | Reliability | Timeouts; retries with exponential backoff and jitter; rate limits; idempotency; fallback to a second model | Every API call can fail | Add retry, timeout and fallback to your client; simulate each failure | Both |
| 2 | Concurrency and cost | asyncio with a semaphore; batching; prompt caching; token budgets per user |
Throughput and bill control | Process 500 documents concurrently without breaking rate limits; report cost | Both |
| 2 | Prompt as code | Versioning prompts in files; templates; tests with a fake model client; logging inputs and outputs safely | Prompts are part of the codebase | Move all prompts to versioned files; test the client with a fake provider | Both |
Production scenarios: a retry loop on a rate-limited API causes a retry storm and a large bill; the model returns JSON with a trailing comment and the parser crashes; a slow response blocks every worker.
Interview questions:
- Concept: What is few-shot prompting? What is structured output?
- Why: Why validate model output with a schema?
- Scenario: The model returns invalid JSON 3% of the time. What do you do?
- Debugging: Time to first token is poor. What do you check?
- Coding: Write a function that calls a model with a timeout and exponential backoff.
Exit criteria: you have a tested client that streams, validates, retries and tracks cost, and you can explain what each failure mode looks like in logs.
Phase 3: Tool use and context engineering (week 3)
Goal: Let a model use your systems safely, and manage what goes into its context. Project 1 finishes this week.
| Week | Topic | What to learn | Why it matters | Practical task | Role |
|---|---|---|---|---|---|
| 3 | The tool-use loop | Describing a function with a schema; the model requests a call; your code runs it; you return the result; repeat until done | The basis of every agent | Build an assistant with three tools (search, calculator, a database lookup) | Both |
| 3 | Tool design | Clear names and descriptions; narrow inputs; useful error messages; idempotent tools; read vs write tools | Bad tool design causes most agent failures | Redesign one confusing tool and compare success rates on 15 cases | Both |
| 3 | Safe tool execution | Validate tool inputs; permissions; confirmation for writes; timeouts; never executing model-produced code or SQL blindly | The model is an untrusted caller | Add an approval step to a write tool; try to trick the model into a bad call | Both |
| 3 | Context management | Conversation history; truncation vs summarization; what to keep; ordering of instructions and data; cost growth over a long chat | Context is a limited budget | Build a chat that stays under a token budget over 50 turns | Both |
| 3 | Documents and images | Sending PDFs and images; extracting text from scans; when a vision model beats OCR | Customer data is rarely plain text | Extract fields from 20 scanned invoices; measure accuracy | Both |
| 3 | First eval habit | Five to ten test cases; expected behavior; a simple scoring script | Starts the eval habit before the formal week | A 20-case test file for the assistant with a pass rate | Both |
Production scenarios: the model calls a delete tool with a wrong ID; a tool returns a 500-line error that fills the context; a long conversation grows until every request is slow and expensive.
Interview questions:
- Concept: What is function calling? How does the loop work?
- Why: Why treat the model as an untrusted caller of your tools?
- Internals: What happens to context cost as a conversation gets longer?
- Scenario: The model keeps calling the wrong tool. How do you investigate?
- Design: Design the tools for an assistant that manages calendar events.
Level 1 checkpoint (end of week 3): Project 1 is finished with tests; your assistant has a measured pass rate; you can explain every failure mode in this level; your failure journal has 15 entries. If you cannot reach a stable pass rate on your own 20 cases, repeat weeks 2 and 3 before moving on.
Level 2: Retrieval and Data (weeks 4 to 6)
Phase 4: RAG fundamentals (week 4)
Goal: Build a working retrieval-augmented system and understand each part. Project 2 starts this week.
| Week | Topic | What to learn | Why it matters | Practical task | Role |
|---|---|---|---|---|---|
| 4 | Why RAG | Models do not know your data; context limits; fine-tuning vs retrieval; when RAG is the wrong answer | Pick the right tool before building | List five customer problems and mark which need RAG, which need a database query, which need neither | Both |
| 4 | Embeddings | Vectors; cosine similarity; semantic vs keyword match; embedding model choice; dimension and cost | The core of semantic search | Embed 200 sentences; query them; inspect the nearest neighbors, including bad ones | Both |
| 4 | Chunking | Chunk size and overlap; splitting by structure (headings, paragraphs); keeping context with each chunk | Chunking decides most of the quality | Compare three chunking strategies on 20 questions | Both |
| 4 | Vector stores | pgvector, Chroma, Qdrant or FAISS; indexes; metadata filters; trade-offs | Where the vectors live and how fast you find them | Store chunks in pgvector; query with a metadata filter | Both |
| 4 | Generation with citations | Building the prompt from retrieved text; asking for sources; saying "I don't know" when nothing relevant is found | Trust depends on cited, grounded answers | Answer 20 questions with citations; mark each answer correct, partial or wrong | Both |
| 4 | Retrieval vs generation errors | Check what was retrieved before blaming the model | The most useful debugging habit in RAG | For each wrong answer, label the cause: not retrieved, retrieved but ignored, or model error | Both |
Production scenarios: the system answers confidently from memory instead of the documents; the right passage is split across two chunks and never retrieved; an outdated document gives an old policy.
Interview questions:
- Concept: What is RAG? What is an embedding?
- Why: Why does chunk size affect answer quality?
- Internals: How does vector similarity search find passages?
- Scenario: An answer is wrong. How do you tell whether retrieval or generation failed?
- Design: Design question answering over a company's policy documents.
Exit criteria: you can show for any wrong answer whether it failed at retrieval or generation, and why.
Phase 5: Production RAG (week 5)
Goal: Make retrieval measurably better and safe for multiple users. Project 2 finishes this week.
| Week | Topic | What to learn | Why it matters | Practical task | Role |
|---|---|---|---|---|---|
| 5 | Retrieval metrics | Recall at k; MRR; a labeled set of question and source pairs | You cannot improve what you cannot measure | Label 40 questions with their source passages; compute recall at 5 | Both |
| 5 | Hybrid search | Keyword search (BM25) plus vector search; combining ranked lists | Fixes exact-term misses (IDs, names, codes) | Add hybrid search; show the recall change | Both |
| 5 | Reranking and query rewriting | A reranker over the top results; rewriting follow-up questions into standalone queries; query expansion | Large, cheap quality gains | Add a reranker and a query rewriter; measure each separately | Both |
| 5 | Document parsing | PDFs, HTML, tables, scans; keeping structure; cleaning headers and footers | Messy documents are the main real-world problem | Ingest 30 messy PDFs; list every parsing failure you see | FDE |
| 5 | Freshness and updates | Incremental indexing; deleting and replacing documents; versioning | Stale data causes wrong answers | Re-ingest an updated document without duplicates | Both |
| 5 | Permissions in retrieval | Per-user and per-group document access enforced in the query, not in the prompt | A common enterprise requirement and a common leak | Prove user A cannot retrieve user B's documents; write the test | Both |
| 5 | Indirect prompt injection | Retrieved text is untrusted data; separating data from instructions; limiting what a model can do after reading it | Documents can attack the model | Plant an injection in a document and see what happens; add mitigations | Both |
Production scenarios: a search for an exact error code returns unrelated text; a user sees content from a document they should not access; a PDF's table is flattened into nonsense; a document says "ignore your instructions" and the model complies.
Interview questions:
- Concept: What is hybrid search? What does a reranker do?
- Why: Why enforce permissions in the retrieval query rather than in the prompt?
- Scenario: Recall at 5 is 60%. How do you improve it step by step?
- Debugging: Exact product codes are never found. What is wrong?
- Design: Design an enterprise search assistant with per-user permissions.
Exit criteria: Project 2 has a measured recall score that improved with each change you made, a permissions test and a prompt-injection test.
Phase 6: Data, pipelines and enterprise integration (week 6)
Goal: Work with the kind of data and systems customers actually have. Project 3 is this phase.
| Week | Topic | What to learn | Why it matters | Practical task | Role |
|---|---|---|---|---|---|
| 6 | Messy data | CSV, JSON and PDF quirks; deduplication; normalization; missing values; pandas and SQL for cleaning | Customer data is never clean | Clean a messy public dataset and document every rule | FDE |
| 6 | Extraction and classification | Using a model to classify and extract at scale; batch processing; cost per record; confidence and human review queues | A very common deployment pattern | Triage 500 support tickets into categories with extracted fields | Both |
| 6 | Text-to-SQL with guardrails | Giving a model a schema; read-only database users; query validation; timeouts and row limits; showing the SQL to the user | Natural-language data access is in high demand and high risk | Build a read-only question interface over a sample database; try to make it delete data | Both |
| 6 | Calling customer systems | REST APIs, pagination, rate limits, webhooks, retries; mapping their data model to yours | Integration is half of an FDE's job | Integrate with a mock CRM or ticketing API; handle pagination and 429s | FDE |
| 6 | Authentication in enterprises | OAuth2, service accounts, API keys, SSO and SAML awareness; secrets management | Customers will ask how you authenticate | Add OAuth2 client credentials to the integration; store secrets in environment variables only | FDE |
| 6 | Background jobs and idempotency | Queues or background workers; retries that do not duplicate work; resumable jobs | Large jobs must survive failure | Make the 500-ticket job resumable and safe to re-run | Both |
| 6 | Sensitive data | Personal data in prompts and logs; redaction; data retention; asking what a provider does with your data | Security review is part of every deployment | Add redaction before logging and a data-handling note to the README | Both |
Production scenarios: a re-run of a failed batch job creates duplicate tickets; a text-to-SQL feature returns a customer's data to the wrong user; a customer's API starts returning 429 and the job gives up; logs contain customer email addresses.
Interview questions:
- Concept: What are the risks of letting a model write SQL?
- Why: Why make a batch job idempotent and resumable?
- Scenario: A customer's API rate limits you. What do you change?
- Debugging: Classification accuracy is 85% in testing and 70% in production. Why?
- Design: Design an email triage pipeline for a customer support team.
Level 2 checkpoint (end of week 6): Projects 2 and 3 are finished and measured; you can show recall and classification scores; you can describe how permissions and sensitive data are handled; your failure journal has 35 entries. If you cannot explain why your recall improved after each change, repeat Phase 5.
Level 3: Agents, Evals and Production (weeks 7 to 10)
Phase 7: Workflows, agents and MCP (weeks 7 to 8)
Goal: Build multi-step systems that are reliable, controllable and safe to give real access. Project 4 runs through this phase.
| Week | Topic | What to learn | Why it matters | Practical task | Role |
|---|---|---|---|---|---|
| 7 | Workflow patterns | Prompt chaining; routing; parallel calls; orchestrator and workers; evaluator and optimizer loops | Most reliable systems are workflows, not free-running agents | Build the same task as a chain, then with a router; compare reliability | Both |
| 7 | The agent loop | Model decides the next step; tools; stopping conditions; step limits; state | An agent is a loop with a budget | Build an agent loop from scratch with a maximum number of steps and a token budget | Both |
| 7 | When not to use an agent | Cost, latency and unpredictability; deterministic steps belong in code | Judgment is the senior skill | For three tasks, argue for a workflow, an agent or plain code | Both |
| 7 | Human in the loop | Approval for risky actions; pausing and resuming; audit logs | Customers will not accept silent writes | Add an approval step that pauses the agent and resumes after a decision | Both |
| 8 | Model Context Protocol | What MCP is; servers, clients and tools; building a small MCP server with an official SDK; auth and permission scoping | A common standard for connecting models to tools and data | Expose three tools from Project 3 through an MCP server and connect a client | Both |
| 8 | State, memory and durability | Persisting agent state; resuming after a crash; long-term memory vs retrieval; checkpointing | Agents that lose state are not production systems | Kill the agent mid-task and resume from a saved checkpoint | Both |
| 8 | Frameworks (awareness) | LangGraph, an agent SDK from a model provider, LlamaIndex; what each hides and what it adds | You will meet them on the job | Rebuild your agent in one framework; list what got easier and harder | Both |
| 8 | Agent failure modes | Loops; wrong tool choice; compounding errors; indirect injection through tool results; over-permissioned tools | Agents fail in ways chatbots do not | Run 20 adversarial cases; list failures and fixes | Both |
| 8 | Coding agents (disciplined use) | Delegating scaffolds and refactors; reviewing diffs; tests first; guarding secrets; instructions files | An expected skill; blind acceptance is a red flag | Build one feature with a coding agent; review the diff line by line; list what you changed | Both |
Production scenarios: an agent loops on the same failing tool call and burns tokens; a tool result from a web page contains instructions and the agent follows them; an agent with write access deletes the wrong record; a crash mid-task leaves half-completed work.
Interview questions:
- Concept: What is an agent? What is MCP?
- Why: Why prefer a workflow over an agent when the steps are known?
- Internals: How do you stop an agent from running forever?
- Scenario: Tool output contains instructions for the model. What happens and how do you defend?
- Debugging: The agent works in demos and fails in production. How do you find out why?
- Design: Design an operations assistant that can read tickets and, with approval, update them.
Exit criteria: Project 4 runs with step and token limits, requires approval for writes, survives a crash, exposes tools through MCP, and has an adversarial test set.
Phase 8: Evals and quality (week 9)
Goal: Know whether a change made the system better, with numbers. Project 5 starts this week.
| Week | Topic | What to learn | Why it matters | Practical task | Role |
|---|---|---|---|---|---|
| 9 | Golden datasets | Collecting real or realistic cases; labeling; covering edge cases and adversarial inputs; versioning the set | Evals are only as good as their data | Build a 100-case golden set for Project 2 and 4 with labels | Both |
| 9 | Error analysis | Reading failures; grouping them by cause; counting each group; fixing the largest first | The step most people skip | Categorize 50 failures; fix the biggest category; re-score | Both |
| 9 | Metrics | Exact match, schema validity, groundedness, task success, tool-call accuracy, latency and cost per task | Different tasks need different metrics | Define three metrics for each project and compute them | Both |
| 9 | LLM-as-judge | Writing a grading rubric; calibrating the judge against human labels; known biases; using a stronger or separate model | Scales evaluation but can be wrong | Label 40 answers by hand; compare with a judge; report agreement | Both |
| 9 | Regression gates | Running evals in CI; thresholds; blocking a merge on a score drop; handling non-determinism with repeats | Stops silent quality loss | A CI job that fails if the pass rate drops more than a set margin | Both |
| 9 | Eval tooling (awareness) | Promptfoo, Ragas, DeepEval, Phoenix, Langfuse or LangSmith | Teams use these; understand what they automate | Run one tool on your set and compare with your own script | Both |
| 9 | Online evaluation | Sampling production traffic; user feedback signals; drift detection | The real data is in production | Add thumbs up and down logging and a weekly review query | AI dev |
Production scenarios: a prompt tweak fixes three complaints and silently breaks twenty other cases; the judge model scores its own style higher than correct answers; a metric improves while users get less happy.
Interview questions:
- Concept: What is an eval? What is LLM-as-judge?
- Why: Why calibrate a judge against human labels?
- Scenario: Quality dropped after a model version change. How do you find out and respond?
- Debugging: Scores are noisy between runs. What do you do?
- Design: Design an evaluation pipeline for a customer support agent.
Exit criteria: you can show a versioned golden set, a score for each project, and a CI check that blocks a regression.
Phase 9: Production, safety, cost and deployment (week 10)
Goal: Operate an AI system: see what it does, keep it safe and control cost. Project 5 finishes this week.
| Week | Topic | What to learn | Why it matters | Practical task | Role |
|---|---|---|---|---|---|
| 10 | Observability | Tracing each model call and tool call; token, cost and latency per request; OpenTelemetry; privacy-aware logging | You cannot debug what you cannot see | Add traces to Projects 2 and 4; find the slowest step of one request | Both |
| 10 | Prompt injection defense | Direct and indirect injection; separating instructions from data; least-privilege tools; output checks; the OWASP list for LLM applications | The top security risk of LLM systems | Run an injection test set against your agent; record what gets through | Both |
| 10 | Guardrails | Input and output filters; topic limits; PII detection; refusing and escalating; why guardrails are layered and imperfect | Security teams ask for these | Add layered checks; measure false blocks and misses | Both |
| 10 | Cost and latency control | Prompt caching; routing easy requests to a smaller model; semantic caching; limiting output length; per-user budgets | The bill decides whether a feature survives | Cut the cost of Project 2 by 40% with no score drop; prove it with evals | Both |
| 10 | Deployment | Docker; environment config; secrets; scaling a stateless service; queues for slow jobs; rate limiting; managed vs self-hosted models | Customers run in different environments | Deploy Project 4 with Docker Compose and a health check | FDE |
| 10 | Customer environments | Private networks, no outbound internet, data residency, approval for new vendors; what changes in each case | A defining FDE constraint | Write a deployment note for a customer who forbids sending data outside their network | FDE |
| 10 | Fine-tuning and local models (awareness) | When fine-tuning helps (style, format, small-model distillation); LoRA at a high level; running a small open-weight model locally | Know the option and when to refuse it | Run a small local model; compare it with a hosted one on your golden set | Both |
Production scenarios: a spike in traffic triples the bill; an agent leaks a system prompt to a user; the provider has a 20-minute outage and the product shows errors; a customer's security team asks where prompts are stored and nobody knows.
Interview questions:
- Concept: What is prompt injection? What is indirect injection?
- Why: Why are guardrails layered rather than a single filter?
- Scenario: The provider is down. What should the user see?
- Debugging: Cost per request doubled after a release. How do you find out why?
- Design: Design logging for an AI feature that handles personal data.
Level 3 checkpoint (end of week 10): Projects 4 and 5 are finished; you have a CI eval gate, traces, an injection test set and a cost report; you can answer a customer's security questions honestly; your failure journal has 55 entries. If your eval gate is flaky or your traces cannot explain a slow request, repeat Phase 8 or 9.
Level 4: Customer-Facing Work and Capstone (weeks 11 to 12)
Phase 10: Forward deployed skills, capstone and interview sprint (weeks 11 to 12)
Goal: Turn an ambiguous customer problem into a deployed, measured system, and explain it clearly. Project 6 is this phase.
| Week | Topic | What to learn | Why it matters | Practical task | Role |
|---|---|---|---|---|---|
| 11 | Customer discovery | Asking about the workflow, users, data, systems, volume and what "good" looks like; finding the real problem under the request | "We need a chatbot" is rarely the real need | Run a mock discovery call with a friend; write down what you learned | FDE |
| 11 | Scoping and success metrics | A one-page scope: problem, users, data, success metric, constraints, risks, out-of-scope list; the smallest useful first version | Prevents building the wrong thing | Write a one-pager for your capstone customer | FDE |
| 11 | Rapid prototyping | A working demo in a day; using coding agents and templates; deciding what to fake and what to build | Speed wins trust early | Build a clickable prototype of the capstone in one day | Both |
| 11 | Demos and communication | Telling a story; showing failures honestly; explaining in business language; handling hard questions live | Customers buy confidence | Record a five-minute demo; then present it to a friend and take questions | FDE |
| 11 | Estimation and risk | Rough effort estimates; what could go wrong; telling the customer bad news early | Trust is built on honest timelines | Write a risk list and a rollout plan with phases | FDE |
| 12 D1 to D3 | Capstone hardening | Evals, traces, guardrails, failure drills, cost report, runbook | Production readiness | Break the capstone on purpose; fix and re-measure | Both |
| 12 D4 | Write-up | A 3-page design and results document with diagrams, metrics and limits | The artifact a hiring manager reads | Publish it with the repository | Both |
| 12 D5 to D6 | Mock interviews | One AI system design round, one coding round with an LLM API, one debugging round, one customer scenario round | Practice under pressure | Record and review each round | Both |
Level 4 checkpoint (end of week 12): the capstone is deployed with a measured score, an eval gate, traces, a runbook and a demo recording; you can walk through it in 15 minutes; your failure journal has 60 or more entries; you have done four mock rounds.
C. Topic dependency graph
Python + FastAPI + SQL + Docker (prerequisites)
│
▼
Tokens and context → LLM API → Structured output → Streaming
│ │
▼ ▼
Tool use → Context management Prompts as code → Eval habit
│ │
▼ │
Embeddings → Chunking → Vector search → RAG │
│ │
▼ │
Hybrid search → Reranking → Permissions → Retrieval metrics ◄─┘
│
▼
Messy data → Extraction at scale → Text-to-SQL → Enterprise integration
│
▼
Workflows → Agent loop → Human in the loop → MCP → Durable state
│
▼
Golden sets → Error analysis → LLM-as-judge → CI eval gate
│
▼
Tracing → Injection defense → Guardrails → Cost control → Deployment
│
▼
Discovery → Scoping → Demo → Capstone → Interviews
The dependencies that matter most, and the week each is taught:
- Tokens and context (week 1), then cost control and context management (weeks 3 and 10).
- Structured output and validation (week 2), then tool inputs (week 3), then extraction at scale (week 6).
- The tool-use loop (week 3), then the agent loop (week 7), then MCP (week 8).
- Embeddings and chunking (week 4), then retrieval metrics (week 5), then the eval harness (week 9).
- Retrieval permissions (week 5), then sensitive data handling (week 6), then guardrails and deployment constraints (week 10).
- Prompt injection basics (week 3), then indirect injection through documents (week 5), through tool results (week 8), and a full defense test (week 10).
- The eval habit (week 3), then golden sets and judges (week 9), then a CI gate before the capstone (week 10).
- Integration and idempotency (week 6), then durable agent state (week 8), then deployment (week 10).
D. In-demand skills map
Use this table to check that your portfolio shows evidence for each commonly requested skill, not just a mention on a resume.
| Skill employers ask for | Where it is taught | Evidence to show |
|---|---|---|
| Python and async backend | Prerequisites, week 2 | Tested FastAPI services with streaming |
| LLM API mastery with structured output | Weeks 1 to 2 | Project 1 with a measured valid-output rate |
| Prompt design and versioning | Week 2 | Prompt files with before and after scores |
| Tool use and function calling | Week 3, week 7 | A tool-using assistant with failure cases |
| RAG and vector search | Weeks 4 to 5 | Project 2 with recall numbers and a written failure analysis |
| Hybrid search and reranking | Week 5 | A chart showing recall after each change |
| Enterprise integration and data pipelines | Week 6 | Project 3 against a mock CRM, with idempotent batch processing |
| Agents and workflows | Weeks 7 to 8 | Project 4 with step limits, approvals and crash recovery |
| MCP server and client | Week 8 | A published MCP server exposing real tools |
| Evals and error analysis | Week 9 | A golden dataset, judge calibration and a CI gate |
| Observability and tracing | Week 10 | Traces that explain one slow and one wrong request |
| Prompt injection defense and guardrails | Weeks 3, 5, 8 and 10 | An adversarial test set and its results |
| Cost and latency optimization | Weeks 2 and 10 | A cost report showing savings with no score drop |
| Deployment in constrained environments | Week 10 | A deployment note for a customer with strict network rules |
| Customer discovery, scoping and demos | Weeks 6 and 11 | A one-page scope and a recorded demo |
| Disciplined use of coding agents | Week 8 and throughout | A reviewed diff and notes on what you changed |
E. Skill levels: where you stand
Two marks tell you how far you have come: week 6 (can build a measured AI feature) and week 12 (can ship and defend an AI system for a customer).
| Area | By week 6 | By week 12 |
|---|---|---|
| LLM APIs | Streams, validates, retries and tracks cost | Chooses models by evals, controls cost with caching and routing |
| Prompting | Writes and versions prompts; scores versions | Treats prompts as tested code with regression checks |
| Tools | Designs safe tools and runs the loop | Builds MCP servers and permission-scoped tools |
| Retrieval | Chunking, vectors, citations, recall metric | Hybrid search, reranking, permissions, freshness, injection defense |
| Agents | Builds workflows; understands the loop | Builds agents with limits, approvals, durability and an adversarial test set |
| Evals | Hand-built test cases | Golden sets, calibrated judges, CI gates, error analysis |
| Data and integration | Cleans messy data; batch extraction | Idempotent pipelines, OAuth2 integrations, text-to-SQL with guardrails |
| Safety | Knows direct and indirect injection | Layered defenses, residual-risk statement, data handling note |
| Operations | Basic logging | Traces, cost and latency dashboards, runbook |
| Customer skills | Explains work to a non-technical friend | Runs discovery, writes a scope, demos honestly, plans a rollout |
| Debugging | Separates retrieval errors from generation errors | Traces any wrong answer to retrieval, prompt, model, tool or data |
How to tell which side you are on
You are in SDE-1 AI-developer territory when you can do all of these without help:
- Given "the answer is wrong", you name the first three places to look and how to tell them apart.
- Given a feature idea, you state how you would measure whether it works before building it.
- Given an agent design, you list what it is allowed to do, what needs approval, and how it stops.
- Given a customer request, you ask five questions that change what you build.
- Given a bill or a latency number, you estimate where it comes from and the cheapest fix.
If two or more are still hard, repeat the related phase before applying.
F. Projects
None of these is a tutorial copy. Each project adds one new class of real problem, and each must ship with tests, an eval score, a README, an architecture sketch and failure journal entries. Keep every project in its own Git repository with small, reviewable commits.
| Project | Weeks | New class of problem it introduces |
|---|---|---|
| 1. Reliable LLM client and extraction toolkit | 1 to 3 | Retries, streaming, validation, tools, cost |
| 2. Document Q&A with measured retrieval | 4 to 5 | Retrieval quality, permissions, injection |
| 3. Support ticket triage pipeline | 6 | Messy data, batch processing, integration, idempotency |
| 4. Operations agent with an MCP server | 7 to 8 | Agent limits, approvals, durability, tool safety |
| 5. Eval harness and observability | 9 to 10 | Golden sets, judges, CI gates, tracing, cost control |
| 6. Customer deployment capstone | 11 to 12 | Discovery to deployed, measured solution |
Project 1: Reliable LLM client and extraction toolkit (weeks 1 to 3)
| Aspect | Specification |
|---|---|
| Requirements | A library and command-line tool that sends prompts to a model with retries, timeouts, streaming, fallback and cost tracking; extracts structured data from messy text; supports a small tool loop |
| Interface | llmtool extract --schema invoice.json --input emails/, llmtool chat --tools ... |
| Architecture | A client class behind an interface; a fake client for tests; Pydantic schemas; versioned prompt files |
| Key decisions | What counts as a retryable error; when to fall back to a second model; where prompts live; how cost is computed and stored |
| Failure scenarios | Rate limit; timeout mid-stream; invalid JSON; a tool called with bad arguments; a truncated response |
| Evaluation | 30-email extraction set with field-level accuracy and valid-output rate; 20-case tool test |
| Observability | One log line per call with tokens, latency, cost and outcome (no raw personal data) |
| Explain in an interview | How your retry policy avoids a retry storm; how you validate output; how you test code that calls a model |
Project 2: Document Q&A with measured retrieval (weeks 4 to 5)
| Aspect | Specification |
|---|---|
| Requirements | Upload documents; ask questions; get cited answers; "I don't know" when sources do not support an answer; per-user document access |
| APIs | POST /documents, POST /ask (streamed), GET /documents, DELETE /documents/{id} |
| Data | PostgreSQL with pgvector (or another vector store); document, chunk and permission tables |
| Architecture | FastAPI; ingestion pipeline (parse, chunk, embed, store); hybrid retrieval; reranker; prompt builder; model client from Project 1 |
| Key decisions | Chunk size and overlap; how many passages to pass; how permissions are enforced in the query; how refusals are triggered |
| Failure scenarios | Answer from memory instead of sources; exact codes not found; stale document; injected instructions in a document; user sees another user's text |
| Evaluation | 40 labeled questions with source passages; recall at 5 and answer groundedness reported after each change |
| Observability | Retrieval and generation timed separately; retrieved passage IDs logged per answer |
| Explain in an interview | How you tell a retrieval failure from a generation failure; what each change did to recall; how permissions and injection are handled |
Project 3: Support ticket triage pipeline (week 6)
| Aspect | Specification |
|---|---|
| Requirements | Ingest 500 or more messy tickets; classify category and urgency; extract customer, product and error code; route to a queue; send low-confidence items to a human review list; write results into a mock ticketing API |
| Interface | A batch job plus a GET /review-queue endpoint |
| Data | CSV or JSON input; PostgreSQL for results and job state; a mock ticketing REST API you build with pagination and rate limits |
| Architecture | Background worker; idempotency key per ticket; checkpointed job state; redaction before logging |
| Key decisions | Confidence threshold for human review; how to detect duplicates; what to do when the ticketing API returns 429; where personal data is removed |
| Failure scenarios | Job crashes halfway; re-run creates duplicates; the API rate limits; a ticket in another language; sensitive data in logs |
| Evaluation | 150 hand-labeled tickets; accuracy per field and per category; cost per ticket |
| Observability | Progress, failure and review-queue metrics; cost totals per run |
| Explain in an interview | How you made the job safe to re-run; how you chose the review threshold; how accuracy in testing differs from production |
Project 4: Operations agent with an MCP server (weeks 7 to 8)
| Aspect | Specification |
|---|---|
| Requirements | An assistant that answers questions about tickets and customers, drafts replies, and with human approval updates ticket status; exposes the same tools through an MCP server |
| Tools | Search tickets (read), get customer (read), run read-only SQL (read), draft reply (read), update status (write, needs approval) |
| Architecture | Agent loop with step and token limits; approval pause and resume; persisted state; MCP server using an official SDK; audit log of every tool call |
| Key decisions | Workflow vs agent for each task; which tools need approval; how state is stored; how the agent stops |
| Failure scenarios | Loops on one failing tool; wrong ID on a write; tool results containing instructions; a crash mid-task; an over-broad SQL query |
| Evaluation | 40 normal and 20 adversarial cases; task success, tool-call accuracy, steps per task and cost per task |
| Observability | One trace per task showing each model and tool call; audit log with the approving user |
| Explain in an interview | Why parts are workflows rather than agent steps; how approvals work; how you defended against injected tool results |
Project 5: Eval harness and observability (weeks 9 to 10)
| Aspect | Specification |
|---|---|
| Requirements | A reusable harness that runs golden sets against Projects 2 and 4; reports metrics; supports an LLM judge calibrated against human labels; blocks a merge on a regression; shows traces and cost |
| Interface | evalrun --suite rag --compare main in CI and locally |
| Data | Versioned golden sets and results stored as files or in PostgreSQL |
| Architecture | Suite definitions; metric functions; judge with a rubric; repeat runs to handle noise; trace export with OpenTelemetry |
| Key decisions | Which metrics gate a merge; judge model and rubric; number of repeats; thresholds and what a "meaningful drop" is |
| Failure scenarios | Judge disagrees with humans; scores swing between runs; a prompt change improves one metric and breaks another |
| Evaluation | Judge agreement with 40 human labels; a recorded regression that the gate caught |
| Observability | A dashboard of pass rate, latency and cost per commit |
| Explain in an interview | How you know your judge can be trusted; how you handle noisy scores; a regression your gate caught |
Project 6: Customer deployment capstone (weeks 11 to 12)
This project simulates an FDE engagement. Pick a realistic fictional customer, such as an insurance claims team, a compliance review group or a logistics support desk, and work through the whole path.
| Aspect | Specification |
|---|---|
| Requirements | A discovery document; a one-page scope with a success metric; a working solution combining retrieval, tools or an agent, and at least one customer-system integration; a deployed instance; an eval report; a demo recording; a runbook |
| Deliverables | Discovery notes and questions asked; scope one-pager; repository with README and architecture diagram; golden set and eval results; failure drill report; cost estimate for 10,000 requests a month; five-minute demo recording; risks and rollout plan |
| Architecture | Your choice, but it must include permissions, redaction, tracing, a CI eval gate and Docker deployment |
| Key decisions | The smallest useful first version; what stays human; how success is measured; how the system behaves under a constraint you invent (for example no outbound internet) |
| Failure scenarios | Model times out; provider outage; malformed output; prompt injection through customer data; customer API change; cost spike |
| Evaluation | A score against the success metric from your scope, reported before and after at least three improvements |
| Observability | Traces, cost and latency per request, and an alert-style log for quality drops |
| Explain in an interview | A 15-minute walkthrough: the customer problem, what you asked, what you built first, how you measured, what failed, what you would do next |
G. Daily learning framework
Every learning day follows one loop: Learn, Build, Break it, Measure, Review. The 72 learning days are numbered with the 12 review days in between: week 1 is days 1 to 7, week 2 is days 8 to 14, and so on, up to day 84. Day 7, day 14 and every seventh day after that is a review day.
The daily loop (2 hours)
| Step | Purpose | Time at 2 hours | Time at 1.5 hours |
|---|---|---|---|
| Learn | Read the provider or library documentation for one topic; no long video sessions | 20 min | 15 min |
| Build | Make the smallest working version in your project repository | 55 min | 40 min |
| Break it | Force a failure or attack: bad input, timeout, injection, empty retrieval | 20 min | 15 min |
| Measure | Add or run five test cases; record a score, cost or latency | 15 min | 10 min |
| Review | Explain out loud for two minutes; write a failure journal entry | 10 min | 10 min |
Weekly rhythm
| Day | Focus |
|---|---|
| D1 and D2 | New concept: learn, understand, small experiments |
| D3 and D4 | Build the concept into the current project |
| D5 | Break it and measure: failure injection, attacks, scores, cost |
| D6 | Integrate and explain: finish the task, answer the week's questions aloud |
| D7 (review) | Re-read the failure journal, rerun your evals, clean up, plan next week |
Task format ProgVale can generate
Each day is one record, so the roadmap can be generated into a calendar, a tracker or a daily email.
day: 59
week: 9
phase: 8
level: 3
topic: Calibrating an LLM judge
prerequisites: [golden-datasets, rubric-writing]
learn: Provider documentation on evaluation, plus one article on judge bias
understand: Draw answer -> judge prompt with rubric -> score -> compare with human label
build: A judge that scores 40 answers on a three-point rubric
break: Swap the order of two answers; use a long answer vs a short answer; observe score changes
measure: Agreement rate with your own human labels; list where they differ
think: What if the judge prefers its own style? How many repeats do you need?
interview: Why calibrate an LLM judge against human labels?
review: [failure journal entry, 2 flashcards]
minutes: 120
project_link: project-5
Three sample days
Day 24 (week 4, day 3): Chunking
- Learn: how chunk size and overlap affect retrieval, from your vector store's documentation.
- Understand: a chunk is the unit that gets retrieved and shown to the model, so a bad split can hide the answer.
- Build: three chunkers (fixed size, by paragraph, by heading) over the same 10 documents.
- Break it: ask questions whose answers cross a chunk boundary; see which chunker loses them.
- Measure: recall at 5 for each chunker on 20 questions; record the winner.
- Interview: "Why does chunk size change answer quality?"
- Review: failure journal entry with the lost question and the chunker that fixed it.
Day 45 (week 7, day 3): The agent loop with approval
- Learn: the structure of an agent loop with step and token limits.
- Understand: the model chooses the next action, your code decides whether to run it, and the loop stops on a final answer or a limit.
- Build: an agent with a read tool and a write tool; the write tool pauses until a human approves.
- Break it: ask the agent to do something that requires an unsafe write; make a tool fail repeatedly and confirm the loop stops at the limit.
- Measure: steps per task and cost per task on 15 cases.
- Interview: "How do you stop an agent from running forever?"
- Review: failure journal entry and two flashcards.
Day 66 (week 10, day 3): Prompt injection defense
- Learn: the difference between direct and indirect injection, and the OWASP guidance for LLM applications.
- Understand: any text the model reads can contain instructions; the defense is limiting what the model can do, not trusting a clever prompt.
- Build: a test set of 20 injection attempts in documents, tool results and user messages; run it against Project 4.
- Break it: add a document that asks the agent to send data to an outside address.
- Measure: how many attempts succeed before and after you add least-privilege tools and output checks.
- Interview: "How do you defend an agent against instructions hidden in a web page?"
- Review: failure journal entry with the attempt that got through and the fix.
Converting the roadmap into days
- Take each row of the tables in section B; its "Practical task" becomes the Build step and its "Why it matters" becomes the Understand step.
- Add a Break it step from the phase's production scenarios.
- Add a Measure step: five test cases, a score and a cost number.
- Attach the phase's interview questions to the matching days.
- Link each day to the project it extends.
- Add the review day with the failure journal re-read and the week's flashcards.
H. Interview preparation
Interview preparation is built into every phase through the question lists in section B. This section shows how those questions map to what interviewers expect and how to answer them.
Eight question types, and how to answer each
| Type | Interviewer's goal | How to answer | Example |
|---|---|---|---|
| Concept | Do you know the term? | One-sentence definition, then one example | What is RAG? |
| Why | Do you know the motivation? | State the problem it solves and what you would do without it | Why validate model output? |
| Internals | Did you look under the hood? | Walk the runtime sequence step by step | How does the tool-use loop work? |
| Scenario | Can you predict behavior? | State assumptions, trace the outcome, name the edge case | A document contains hidden instructions. What happens? |
| Debugging | Can you investigate calmly? | Start from the symptom, name the first three checks, then narrow | Answers got worse after a release |
| AI system design | Can you make trade-offs? | Clarify requirements, estimate scale and cost, sketch, then evals, failures and safety | Design a support assistant for 10,000 users |
| Coding | Can you write correct code? | Clarify, write simply, handle the failure cases, test | A model call with retry, timeout and validation |
| Customer scenario | Can you work with ambiguity and people? | Clarify the goal, find the real problem, propose the smallest useful version, state how to measure it and what could go wrong | A customer says "we need a chatbot" |
A reusable answer shape: what it is, why it exists, how it works, how it fails, how you measure it, and the trade-off.
A reusable shape for customer scenarios: goal and users, data and systems, success metric, smallest useful version, evaluation, rollout, monitoring, risks.
Concept to interview map
| Concept (phase) | A basic question | A stronger-candidate question |
|---|---|---|
| Tokens and context (1) | What is a context window? | Latency doubled after adding context. Why and what do you do? |
| Structured output (2) | How do you get JSON from a model? | It is invalid 3% of the time. How do you reach a rate you can live with? |
| Tool use (3) | What is function calling? | The model keeps picking the wrong tool. How do you investigate? |
| RAG (4) | What is RAG? | Wrong answer: how do you tell retrieval from generation failure? |
| Production RAG (5) | What does a reranker do? | Recall at 5 is 60%. What do you change, in what order, and how do you check? |
| Data and integration (6) | What are the risks of text-to-SQL? | A customer API rate limits you in the middle of a batch job. What changes? |
| Agents and MCP (7 and 8) | What is an agent? What is MCP? | When would you not use an agent? How do you bound one? |
| Evals (9) | What is an eval? | Quality dropped after a model update. How do you prove it and respond? |
| Safety (10) | What is prompt injection? | Design defenses for an agent that reads web pages and can send email |
| Cost and operations (10) | How do you reduce cost? | Cost per request doubled. Walk through your investigation |
| Customer work (11) | What does a forward deployed engineer do? | A customer asks for an agent. How do you decide whether they need one? |
Interview preparation schedule
| When | Activity |
|---|---|
| Weeks 1 to 6 | Answer each phase's questions out loud on the review day; keep the failure journal |
| Week 6 | One mock round: explain Projects 2 and 3 as if to an interviewer |
| Weeks 7 to 10 | One "symptom to cause" debugging drill per week using your own traces |
| Week 10 | One mock AI system design round |
| Weeks 11 to 12 | Two customer scenario role plays, one coding round with an LLM API, one debugging round; record and review each |
| After week 12 | Apply; keep one mock per week while interviewing |
A note on general coding rounds: most SDE-1 loops, including those for AI roles, still include data structures and algorithms or general coding rounds. This roadmap does not teach them. Keep practicing 20 minutes a day alongside it (see the Python roadmap's DSA section).
I. Limits, left-out topics and next steps
Deliberately left out
- Training models from scratch and research-level ML: the job here is building on foundation models. Deep learning, transformer internals beyond the awareness level, and distributed training are a separate path.
- Fine-tuning in depth: awareness only. Retrieval, better prompts and evals solve most customer problems first.
- GPU and inference engineering: serving models at scale (CUDA, custom kernels, inference servers) is its own specialty. You run a small local model once in week 10 to see the trade-offs.
- Large framework deep dives: frameworks are used once in week 8 to understand what they hide. Frameworks change faster than fundamentals.
- Multi-agent systems in depth: you learn that they exist and when a single agent or workflow is better. Most production value comes from simpler designs.
- Voice, video and robotics: the same principles apply, but each needs its own latency and data skills.
- Heavy front-end work: a simple interface for demos is enough.
Where to go next
- Add caching, queues and rate limiting at scale (the Java and Python roadmaps cover the backend side).
- Study system design for AI products: multi-tenant RAG, cost-aware routing, evaluation at scale.
- Go deeper in one specialty: evals and reliability, retrieval, agents and tool ecosystems, or deployment in regulated industries.
- Do a real engagement: volunteer or freelance for a small business, because real customers teach more than simulated ones.
A version and accuracy caveat
This is a fast-moving field. Model names, SDK signatures, pricing, framework APIs, MCP details and eval tools change often. The concepts here (retrieval, evals, tool design, guardrails, cost control, discovery) are stable, but check the official documentation of the provider and library versions you install, and re-verify anything specific in weeks 8, 9 and 10. Demand claims in this roadmap come from hiring guides and posting analyses of mixed quality, so confirm them against the job descriptions you collect in week 1.
The honest summary
The roadmap covers the engineering path hiring teams describe: calling a model reliably, giving it tools and data, measuring quality, keeping it safe and affordable, and explaining it to a customer. It is demanding but finishable at two hours a day. If time runs short, protect these four phases above all others: production RAG (5), agents and tool safety (7), evals (9) and the capstone (10). They are where most candidates have a demo but no evidence.
Your notes
Notes & links
Saved automatically.