Building AI Agents: From Models to Production

The decision before the build

The reframe and the decision tree. What separates an agent from an assistant, and how to tell — before writing code — whether a task wants a single call, a fixed workflow, or a real agent.

• 1.1Assistant vs agent — the real line
• 1.2The five layers, end to end
• 1.3When not to build an agent
• 1.4What 'production' actually demands

ToolkitYour agent spec and a written 'done' definition.

The brain

Reasoning about model capability, context, and limits — and choosing a model deliberately instead of by default.

• 2.1LLM and reasoning models, in practice
• 2.2Context windows and token budgets
• 2.3Choosing a model — capability, latency, cost
• 2.4Prompting as the primary control surface

ToolkitA baseline planning call, with its tokens and cost recorded.

The manager

Building the agent loop by hand before reaching for a framework — and adding memory and guardrails without bloating context.

• 3.1What a harness is — and how it maps to 'orchestration'
• 3.2The agent loop: model, tool, result, repeat
• 3.3Planning and task decomposition
• 3.4Memory and guardrails

ToolkitAn agent loop with a clean stop condition and scratchpad memory.

The hands

Designing a tool surface, not a pile of functions — and deciding when to promote an action to a typed, gated tool.

• 4.1Designing a tool surface
• 4.2Tool definitions and prescriptive descriptions
• 4.3Server-side and client-side tools
• 4.4MCP — connecting third-party capabilities

ToolkitYour agent answering a question it couldn't answer from the model alone.

The expertise

Telling skills and tools apart — cleanly, up front — and packaging reusable expertise the agent loads on demand.

• 5.1Skill vs tool — the distinction, drawn clearly
• 5.2Skill structure and progressive disclosure
• 5.3Domain skills, and when a prompt should become one
• 5.4Pre-built skills vs custom

ToolkitA report-writing skill producing a consistent, house-style document.

Knowing it works

Answering the question that decides whether an agent survives contact with users — how do I know it works, and didn't regress?

• 6.1Why evals are the production skill
• 6.2Golden sets, rubric grading, and LLM-as-judge
• 6.3Tracing a run to find where it broke
• 6.4Turning failures into permanent tests

ToolkitA rubric and eval set that grades the agent and catches a regression.

The environment

Running an agent somewhere real, safely, with the bill under control — treating cost, state, and security as first-class.

• 7.1Execution environments — managed and self-hosted
• 7.2State across turns and sessions
• 7.3Security and permissions
• 7.4Observability and cost, made explicit

ToolkitA deployed agent with logging, cost tracking, and a permission gate.

More than one

Knowing when more than one agent helps — and when it just adds cost — and coordinating specialists.

• 8.1When multiple agents help — and when they don't
• 8.2Coordinator and specialists
• 8.3Parallel fan-out, handoffs, aggregation
• 8.4The cost and latency math

ToolkitA multi-agent version that beats the single agent on the eval suite.

Fit for an organization

Taking an agent from 'works on my machine' to fit for an organization — retrieval, governance, and cost at scale.

• 9.1RAG and knowledge systems
• 9.2Governance, compliance, and audit
• 9.3Cost optimization at scale
• 9.4Deployment patterns and case studies

ToolkitA retrieval source and an audit log wired into the agent.

Prove it

Final assembly across all five layers, a hardening pass, and a demo — the proof you can build and ship a production agent.

• 10.1Final assembly across all five layers
• 10.2The hardening pass
• 10.3Demo and review

ToolkitA deployed research-analyst agent, an eval report, and a cost readout.