- A hands-on Anthropic workshop on decomposing an over-complex agent ("Stock Pilot," an inventory agent with a 400-line system prompt, 12 tools, and 3 subagents) whose evals had degraded.
- The fix uses the right agentic primitives at the right time: move business logic from the system prompt into skills (progressive disclosure), replace most custom tools with Claude Code primitives (bash/read/write, code execution), and keep only the subagents you truly need.
- Migrating to Claude managed agents and "hill climbing" on evals took the agent from a ~62–83% baseline to ~92%, with dramatically lower token usage, cost, and latency.
Tool, skill, or subagent? Decomposing an agent that outgrew its prompt
- The common failure pattern: an agent that worked well gets capabilities bolted on over time until the system prompt is hundreds of lines, there are dozens of tools and subagents, and evals start regressing.
- Evals are central: Stock Pilot has 12 eval tasks across 5 grader types — regression (R) single-turn tasks and failure-mode (F) multi-turn tasks — using both deterministic graders (turn count, latency, tokens) and non-deterministic LLM-as-a-judge graders (tone, style, output quality).
- Specific eval failures had specific root causes: an inefficient winding path (F1), a communication breakdown between an orchestrator and a subagent (F2), and contradicting policies in different parts of the long system prompt causing a hallucinated multiplier (R8) — a context problem, not a model problem.
- Skills enable progressive disclosure: package information Claude needs only some of the time and let it pull that into context when needed, instead of stuffing everything into the system prompt; this cut the prompt from ~400 lines to ~15 and reduced context pollution.
- Start with human-like primitives: give Claude the same tools a person has at work — a file system, web search, code execution, a to-do list — then remove what you don't need and add custom tools only as required.
- Code execution is far more efficient than loading raw data: giving Claude a bash tool to write and run Python over CSVs/Excel dropped token usage from over 200,000 to a fraction, also lowering cost and execution time.
- Tool strategy ordering: start with Claude Code primitives, then create local custom tools, and only reach for MCP when multiple agents/clients need the same standardized, governed tools — and consider code execution (CLIs/APIs) as an alternative to MCP to avoid context pollution.
- Two strong reasons to use subagents: to throw a lot of Claude at a parallelizable problem (deep research, codebase exploration), or to give a fresh, context-isolated mind to a problem (e.g. code review, or a forecasting subagent kept separate from the main conversation); Claude managed agents provide a native callable-agents capability with built-in observability instead of wrapping subagents as tools.
Claude managed agents (CMA) — Anthropic's managed platform that offloads the agentic harness (hosting, scaling, security, memory) so you focus on agent design.
Agentic primitive — A building block of agent design: a tool, a skill, or a subagent.
Skill — Packaged, composable information Claude can pull into context only when it realizes a task needs it (progressive disclosure).
Progressive disclosure — Surfacing information to the model only when needed, instead of preloading it all into the system prompt.
Subagent / callable agent — A separate Claude instance for parallelism or fresh, context-isolated work; CMA exposes a native callable-agents capability.
Eval — A graded task used to measure agent performance over time.
Hill climbing — Iteratively establishing a baseline, tweaking architecture, and rerunning evals to improve scores.
Human-like primitive — A foundational capability humans have at work — file system, web search, code execution, to-do list — that agents start with.
LLM-as-a-judge — A non-deterministic grader using a model to evaluate qualities like tone, style, and output quality.
Orchestrator — The top-level agent that coordinates tools and subagents.
All right, fantastic. Everyone, I hope you've had a fantastic day at Code with Claude London. My name is Will. I'm on our engineering team at Anthropic. I sit on a team called Applied AI, which means I essentially split my time between internal engineering work and time spent building agents with customers.
Imagine that you built and shipped an agent to solve a problem — something a lot of folks in this room have done. And imagine this agent worked fantastic. But it works so well that a few weeks after shipping, you were asked to add some additional capability. A few weeks after that, you received more business requirements and added more capability. This pattern continued until your system prompt had grown to be several hundred lines long. You have dozens of tools and subagents. And because of the complexity, you've started to see regressions in the areas your agent was previously accelerating in. If this is you, you're not alone — we see this commonly with customers, and with ourselves included.
In this workshop, we're going to simulate an agent that's grown to a complexity where we see degradation in performance. We'll walk through some of the decisions we make as engineers and architects to improve the design and restore performance. Specifically, we'll make decisions around tools, skills, and subagents. As we modernize the stack of our agent, we want to use the right agentic primitives at the right time. When do you use a tool, when do you use a skill, and when do you use a subagent?
For this session, we're focusing on an agent called Stock Pilot — an inventory management agent designed by and for a mid-size retailer. The agent can flag low levels of stock, forecast demand, pick suppliers, file POs, and write weekly reports. None of these capabilities are particularly complex on their own, but the issue is we've bolted capabilities onto our agent over time without modernizing our architecture. Today the agent is facilitated by a single orchestrator. It has a system prompt that's grown to about 400 lines long, 12 different tools, and three of those tools happen to be wrappers around subagents with completely isolated context windows. In the repo there's an agent under a folder called "before" that walks through this exactly. The result is that our evals have started to dip.
Let's talk about evals. For this agent we have 12 different eval tasks across five different types of graders. On the left side of the screen you see IDs starting with R — that stands for regression. These are more realistic single-turn tasks. You also see F IDs — that stands for failure mode — more complex multi-turn tasks. We have deterministic graders (turn count, latency, number of tokens) that we track over time, and we use LLM-as-a-judge to evaluate non-deterministic characteristics like personality, tone, style, and output quality.
When you run the evals, the agent is struggling. F1 simulates a daily low-stock sweep; it fails because the agent does the right thing but takes a very winding, inefficient path. F2 evaluates the ordering process under a particular promotion; it fails because we're using a subagent for this task and there's a communication breakdown between the subagent and the orchestrator — a really common point of failure with complicated systems. R8 checks forecasting during a particular promotion month; it fails because two different policies live in very different parts of the system prompt and contradict each other. In the repo, the evals pass at about 83% up front, which is okay, but in manufacturing a 17% failure is really expensive. Double-clicking on R8: the agent pulled the right forecasting baseline (12 units a day) and the right promotion multiplier (3.1x), but in the calculation it hallucinated and used 1.35 instead. This isn't a model problem — it's an issue with the information we're surrounding the model with. Our system prompt has grown long and confusing with conflicts.
Our objective is to run our suite of evals, triage the issues, and update the design of our agent. Then we'll do something we call internally "hill climbing" — we run our evals to get a baseline (about 83%), optimize the architecture, and continue running evals so we climb. We're also going to start with an agent that's self-created on our messages API and migrate it to Claude managed agents. Claude managed agents allow us to offload the messiness that comes with maintaining an agentic harness and scaling agents safely and securely to thousands of users. If I want to build and run my agent locally, I can do that easily. But the moment I need to host it remotely and allow thousands of users to engage with it at the same time, there's an infrastructure problem, a scaling problem, memory, security — so much to account for. To offload that, so I can just worry about the architecture of my agent and decisions around tools, skills, and subagents, I offload everything else to Claude managed agents. This is where we separate the agent from the session details and from the sandboxed environment where tool calls actually happen.
Let's get hands-on. When you open the workshop link, you'll clone the repo. We have a UV project set up, so run UV sync to get all your packages and dependencies for the Anthropic SDK and to deploy your agent to Claude managed agents. You'll need an API key — go to your Claude console account, create one, copy the .env example, and paste your API key in. All 12 evals are set up. To get a baseline, run uv run evals --agent before. We're going to use Claude Code to run the evals, triage the results, and climb accordingly. There's a "before" folder (the messages-API version of the agent) and a "starter" folder (the same agent deployed on Claude managed agents). To deploy on CMA, run uv run deploy --starter.
I'm using Claude Code with Opus 4.7, effort level set to extra high — I usually set effort to extra high with Opus 4.7 and forget about it, because it gets great performance. The first thing I did was run uv run evals --agent before. The results we got were actually 62%, worse than the 83% I told you before — we passed seven out of 12. Claude provided a diagnosis. Using Claude to find themes, the first is that our model is taking on a lot of work that it should have tools to do — reasoning across information it doesn't have the tools to complete. There are issues with the enforcement of output structure. And there are policy issues, because the long system prompt has confusions. Claude found root causes, and we'll go one by one to address them.
The first thing I asked Claude: look at my agent.py file, where the main CMA agent loop is. "Do you have any thoughts on the system prompt? Maybe I can use skills instead of a long-running system prompt for progressive disclosure." Skills are packaged and composable information that Claude can pull into context whenever it realizes it needs that information to complete a task. Skills are useful with Claude Code — providing information on your testing process, or packaging up your brand and UI components. They're also useful within the agents you build for customers. In our case, we have a lot of policies and procedures for inventory management. As I accumulated requirements over time, instead of building skills, I kept appending to my system prompt, so it got longer and longer. We don't recommend that. Leave the system prompt only for information Claude needs regardless of the task; skills are fantastic for packaging information Claude needs some of the time, not all of the time. If I ask Claude to build a forecast, it doesn't need forecasting information unless I specifically ask for it — so for that task I want Claude to pull forecasting information into its context window. Skills are also great for being efficient with context, because stuffing everything into the system prompt pollutes the context window with information Claude doesn't need. So we activated a number of skills and swapped our system prompt from about 400 lines to about 50.
Next, tools. We have 12 different tools — a tool for everything: retrieving data, analyzing data, everything. I asked Claude to analyze the tools and help me optimize. A tip we carry at Anthropic: whenever we build agents, we lean into the same primitives we as humans have access to. Imagine yourself at work — you have a computer, you can navigate a file system, type in a browser, search the web, and (if you're an engineer) write and execute code. With Claude Code, we've effectively given Claude access to all the same primitives. This is powerful because we can drop in better versions of Claude as we release new models, and Claude just uses those primitives better. We lean into human-like primitives first — code execution, file-system navigation, a to-do list, web search — and remove them as needed. An example: for document analysis with a lot of CSVs or Excel sheets, code execution is one of the best ways to do data analysis. Giving Claude a bash tool to write and run a quick Python script and reason across the results is much more effective than uploading the entire CSV into Claude's context window. So we removed most of the tools and replaced them with these primitives. When you build using Claude managed agents, these tools are included by default — you don't have to write a tool to give Claude code execution or file-system access; you rely on the built-in tools we built for Claude Code and make available through CMA.
There's always a need for some custom tools — you'll only get so far with the same tools we give Claude Code. So we start with primitives, remove what we don't need, and add custom tools only as needed. When it comes to MCP: you can first lean on Claude Code primitives, then create custom standalone tools only your agent uses, and then connect to MCP. We see a lot of folks run to MCP first and end up in an ecosystem of chaotic, overlapping MCP servers. We only reach for MCP when we have a common collection of tools that multiple clients benefit from — multiple agents or Claude Code clients that need the same standardized, governed tools. Increasingly common is leaning on Claude's ability to use code execution to invoke CLIs and APIs as a means of executing tools instead of MCP. One drawback of MCP is that it can pollute context and take up a lot of space, so in some cases you can rely on code execution to create more flexibility without using MCP.
Looking at the before and after, the first thing that jumps off the screen is token usage. Before, I was using over 200,000 tokens for a particular task; after leaning on file-system primitives, that went down dramatically — a direct result of giving my agent code execution. Instead of reading a full CSV into context, I give my agent the ability to write and run Python. Cost went down because we're not using as many tokens, and execution time went down too. This was a good case where using primitives over more stagnant tools was clearly the right decision; it won't happen all the time, but here it did.
Now subagents. We had 12 tools, three of which wrapped subagents. The two main instances where subagents are effective: first, when you want to throw a lot of Claude at a problem — deep research, web search, codebase exploration — where many minds running at the same problem makes sense and you can parallelize. Second, when you need a fresh mind to look at a problem. As a developer, I don't want to be the same person writing and reviewing my code. In Claude Code it makes sense to have one instance of Claude writing code and another reviewing it without context about the first instance. We kept one subagent in our inventory agent — for forecasting. I want my forecasting separate from my main instance of Claude; I don't want anything in my initial context window to distort the forecasting process. I have a skill that walks through the step-by-step sequence for forecasts, but I don't want the same Claude my customer is talking with to also write the forecast.
We're not going to expose our subagent as a tool. Using Claude managed agents, there's a native capability for subagents that allows logging and observability to be effective. One problem with subagents is that communication between orchestrators and subagents is hard to get accurate and seamless — a lot can get lost in translation. Logging is also difficult because you have to collect transcripts from multiple agents. Within Claude managed agents, we've added this native "callable agents" capability — like managed subagents — so within your session information you have observability and metrics about what your subagents are doing, as accurate as your orchestrator. You can also just define your subagent as a tool, which we did previously, but we moved away from that to use the CMA native capability. There are a lot of cases where you can scrap the subagent entirely and give more flexibility to your main agent, because frontier models have gotten intelligent enough to manage more information — so you don't need as many subagents.
Let's look at the architecture we ended with. We started with an orchestrator, a ~400-line system prompt, 12 tools (three of them subagents). We ended with an orchestrator deployed on Claude managed agents, because I didn't want to worry about infrastructure, scaling, or security — I just wanted to worry about my agent. We simplified our tools to three: bash, read, and write. When our agent starts executing, we sync some data into the CMA environment so it can reason across that data. We simplified our system prompt to 15 lines and replaced all of our business logic with skills, so Claude pulls that information into its brain only when it realizes it needs it. As a result, we showed how to start hill climbing on evals — my eval score is now about 92%, up from the baseline. I've simplified my design, leaned into the primitives that make Claude great, and seen positive results: we're faster, using fewer tokens because of code execution. Turn count stays about the same, but because token usage and cost go down, I'm okay with Claude taking more turns. For high-intelligence agents where forecasting is at play, I'm willing to take a little higher latency in exchange for better performance and lower cost.
Some takeaways. When we build agents, we start with a single agent loop equipped with simple primitives that give Claude human-like capabilities — the file system, web search, code execution, sometimes a to-do list — and build accordingly. Next, we use progressive disclosure through skills: instead of stuffing the system prompt, we make information accessible to Claude whenever it realizes it needs it, so we run more efficiently and don't pollute the context window. Lastly, write evals. Hill climbing is a concept we lean on heavily at Anthropic: establish a baseline, tweak your architecture, rerun evals, and get better over time. Make sure your evals are updated as your product capability expands so you're always measuring the things you care about. With that, we're going to wrap. I really appreciate your time today, and I hope you have a great rest of your day at Code with Claude in London.
TL;DR
- A hands-on Anthropic workshop on decomposing an over-complex agent ("Stock Pilot," an inventory agent with a 400-line system prompt, 12 tools, and 3 subagents) whose evals had degraded.
- The fix uses the right agentic primitives at the right time: move business logic from the system prompt into skills (progressive disclosure), replace most custom tools with Claude Code primitives (bash/read/write, code execution), and keep only the subagents you truly need.
- Migrating to Claude managed agents and "hill climbing" on evals took the agent from a ~62–83% baseline to ~92%, with dramatically lower token usage, cost, and latency.
Takeaways
- The common failure pattern: an agent that worked well gets capabilities bolted on over time until the system prompt is hundreds of lines, there are dozens of tools and subagents, and evals start regressing.
- Evals are central: Stock Pilot has 12 eval tasks across 5 grader types — regression (R) single-turn tasks and failure-mode (F) multi-turn tasks — using both deterministic graders (turn count, latency, tokens) and non-deterministic LLM-as-a-judge graders (tone, style, output quality).
- Specific eval failures had specific root causes: an inefficient winding path (F1), a communication breakdown between an orchestrator and a subagent (F2), and contradicting policies in different parts of the long system prompt causing a hallucinated multiplier (R8) — a context problem, not a model problem.
- Skills enable progressive disclosure: package information Claude needs only some of the time and let it pull that into context when needed, instead of stuffing everything into the system prompt; this cut the prompt from ~400 lines to ~15 and reduced context pollution.
- Start with human-like primitives: give Claude the same tools a person has at work — a file system, web search, code execution, a to-do list — then remove what you don't need and add custom tools only as required.
- Code execution is far more efficient than loading raw data: giving Claude a bash tool to write and run Python over CSVs/Excel dropped token usage from over 200,000 to a fraction, also lowering cost and execution time.
- Tool strategy ordering: start with Claude Code primitives, then create local custom tools, and only reach for MCP when multiple agents/clients need the same standardized, governed tools — and consider code execution (CLIs/APIs) as an alternative to MCP to avoid context pollution.
- Two strong reasons to use subagents: to throw a lot of Claude at a parallelizable problem (deep research, codebase exploration), or to give a fresh, context-isolated mind to a problem (e.g. code review, or a forecasting subagent kept separate from the main conversation); Claude managed agents provide a native callable-agents capability with built-in observability instead of wrapping subagents as tools.
Vocabulary
Claude managed agents (CMA) — Anthropic's managed platform that offloads the agentic harness (hosting, scaling, security, memory) so you focus on agent design.
Agentic primitive — A building block of agent design: a tool, a skill, or a subagent.
Skill — Packaged, composable information Claude can pull into context only when it realizes a task needs it (progressive disclosure).
Progressive disclosure — Surfacing information to the model only when needed, instead of preloading it all into the system prompt.
Subagent / callable agent — A separate Claude instance for parallelism or fresh, context-isolated work; CMA exposes a native callable-agents capability.
Eval — A graded task used to measure agent performance over time.
Hill climbing — Iteratively establishing a baseline, tweaking architecture, and rerunning evals to improve scores.
Human-like primitive — A foundational capability humans have at work — file system, web search, code execution, to-do list — that agents start with.
LLM-as-a-judge — A non-deterministic grader using a model to evaluate qualities like tone, style, and output quality.
Orchestrator — The top-level agent that coordinates tools and subagents.
Transcript
All right, fantastic. Everyone, I hope you've had a fantastic day at Code with Claude London. My name is Will. I'm on our engineering team at Anthropic. I sit on a team called Applied AI, which means I essentially split my time between internal engineering work and time spent building agents with customers.
Imagine that you built and shipped an agent to solve a problem — something a lot of folks in this room have done. And imagine this agent worked fantastic. But it works so well that a few weeks after shipping, you were asked to add some additional capability. A few weeks after that, you received more business requirements and added more capability. This pattern continued until your system prompt had grown to be several hundred lines long. You have dozens of tools and subagents. And because of the complexity, you've started to see regressions in the areas your agent was previously accelerating in. If this is you, you're not alone — we see this commonly with customers, and with ourselves included.
In this workshop, we're going to simulate an agent that's grown to a complexity where we see degradation in performance. We'll walk through some of the decisions we make as engineers and architects to improve the design and restore performance. Specifically, we'll make decisions around tools, skills, and subagents. As we modernize the stack of our agent, we want to use the right agentic primitives at the right time. When do you use a tool, when do you use a skill, and when do you use a subagent?
For this session, we're focusing on an agent called Stock Pilot — an inventory management agent designed by and for a mid-size retailer. The agent can flag low levels of stock, forecast demand, pick suppliers, file POs, and write weekly reports. None of these capabilities are particularly complex on their own, but the issue is we've bolted capabilities onto our agent over time without modernizing our architecture. Today the agent is facilitated by a single orchestrator. It has a system prompt that's grown to about 400 lines long, 12 different tools, and three of those tools happen to be wrappers around subagents with completely isolated context windows. In the repo there's an agent under a folder called "before" that walks through this exactly. The result is that our evals have started to dip.
Let's talk about evals. For this agent we have 12 different eval tasks across five different types of graders. On the left side of the screen you see IDs starting with R — that stands for regression. These are more realistic single-turn tasks. You also see F IDs — that stands for failure mode — more complex multi-turn tasks. We have deterministic graders (turn count, latency, number of tokens) that we track over time, and we use LLM-as-a-judge to evaluate non-deterministic characteristics like personality, tone, style, and output quality.
When you run the evals, the agent is struggling. F1 simulates a daily low-stock sweep; it fails because the agent does the right thing but takes a very winding, inefficient path. F2 evaluates the ordering process under a particular promotion; it fails because we're using a subagent for this task and there's a communication breakdown between the subagent and the orchestrator — a really common point of failure with complicated systems. R8 checks forecasting during a particular promotion month; it fails because two different policies live in very different parts of the system prompt and contradict each other. In the repo, the evals pass at about 83% up front, which is okay, but in manufacturing a 17% failure is really expensive. Double-clicking on R8: the agent pulled the right forecasting baseline (12 units a day) and the right promotion multiplier (3.1x), but in the calculation it hallucinated and used 1.35 instead. This isn't a model problem — it's an issue with the information we're surrounding the model with. Our system prompt has grown long and confusing with conflicts.
Our objective is to run our suite of evals, triage the issues, and update the design of our agent. Then we'll do something we call internally "hill climbing" — we run our evals to get a baseline (about 83%), optimize the architecture, and continue running evals so we climb. We're also going to start with an agent that's self-created on our messages API and migrate it to Claude managed agents. Claude managed agents allow us to offload the messiness that comes with maintaining an agentic harness and scaling agents safely and securely to thousands of users. If I want to build and run my agent locally, I can do that easily. But the moment I need to host it remotely and allow thousands of users to engage with it at the same time, there's an infrastructure problem, a scaling problem, memory, security — so much to account for. To offload that, so I can just worry about the architecture of my agent and decisions around tools, skills, and subagents, I offload everything else to Claude managed agents. This is where we separate the agent from the session details and from the sandboxed environment where tool calls actually happen.
Let's get hands-on. When you open the workshop link, you'll clone the repo. We have a UV project set up, so run UV sync to get all your packages and dependencies for the Anthropic SDK and to deploy your agent to Claude managed agents. You'll need an API key — go to your Claude console account, create one, copy the .env example, and paste your API key in. All 12 evals are set up. To get a baseline, run uv run evals --agent before. We're going to use Claude Code to run the evals, triage the results, and climb accordingly. There's a "before" folder (the messages-API version of the agent) and a "starter" folder (the same agent deployed on Claude managed agents). To deploy on CMA, run uv run deploy --starter.
I'm using Claude Code with Opus 4.7, effort level set to extra high — I usually set effort to extra high with Opus 4.7 and forget about it, because it gets great performance. The first thing I did was run uv run evals --agent before. The results we got were actually 62%, worse than the 83% I told you before — we passed seven out of 12. Claude provided a diagnosis. Using Claude to find themes, the first is that our model is taking on a lot of work that it should have tools to do — reasoning across information it doesn't have the tools to complete. There are issues with the enforcement of output structure. And there are policy issues, because the long system prompt has confusions. Claude found root causes, and we'll go one by one to address them.
The first thing I asked Claude: look at my agent.py file, where the main CMA agent loop is. "Do you have any thoughts on the system prompt? Maybe I can use skills instead of a long-running system prompt for progressive disclosure." Skills are packaged and composable information that Claude can pull into context whenever it realizes it needs that information to complete a task. Skills are useful with Claude Code — providing information on your testing process, or packaging up your brand and UI components. They're also useful within the agents you build for customers. In our case, we have a lot of policies and procedures for inventory management. As I accumulated requirements over time, instead of building skills, I kept appending to my system prompt, so it got longer and longer. We don't recommend that. Leave the system prompt only for information Claude needs regardless of the task; skills are fantastic for packaging information Claude needs some of the time, not all of the time. If I ask Claude to build a forecast, it doesn't need forecasting information unless I specifically ask for it — so for that task I want Claude to pull forecasting information into its context window. Skills are also great for being efficient with context, because stuffing everything into the system prompt pollutes the context window with information Claude doesn't need. So we activated a number of skills and swapped our system prompt from about 400 lines to about 50.
Next, tools. We have 12 different tools — a tool for everything: retrieving data, analyzing data, everything. I asked Claude to analyze the tools and help me optimize. A tip we carry at Anthropic: whenever we build agents, we lean into the same primitives we as humans have access to. Imagine yourself at work — you have a computer, you can navigate a file system, type in a browser, search the web, and (if you're an engineer) write and execute code. With Claude Code, we've effectively given Claude access to all the same primitives. This is powerful because we can drop in better versions of Claude as we release new models, and Claude just uses those primitives better. We lean into human-like primitives first — code execution, file-system navigation, a to-do list, web search — and remove them as needed. An example: for document analysis with a lot of CSVs or Excel sheets, code execution is one of the best ways to do data analysis. Giving Claude a bash tool to write and run a quick Python script and reason across the results is much more effective than uploading the entire CSV into Claude's context window. So we removed most of the tools and replaced them with these primitives. When you build using Claude managed agents, these tools are included by default — you don't have to write a tool to give Claude code execution or file-system access; you rely on the built-in tools we built for Claude Code and make available through CMA.
There's always a need for some custom tools — you'll only get so far with the same tools we give Claude Code. So we start with primitives, remove what we don't need, and add custom tools only as needed. When it comes to MCP: you can first lean on Claude Code primitives, then create custom standalone tools only your agent uses, and then connect to MCP. We see a lot of folks run to MCP first and end up in an ecosystem of chaotic, overlapping MCP servers. We only reach for MCP when we have a common collection of tools that multiple clients benefit from — multiple agents or Claude Code clients that need the same standardized, governed tools. Increasingly common is leaning on Claude's ability to use code execution to invoke CLIs and APIs as a means of executing tools instead of MCP. One drawback of MCP is that it can pollute context and take up a lot of space, so in some cases you can rely on code execution to create more flexibility without using MCP.
Looking at the before and after, the first thing that jumps off the screen is token usage. Before, I was using over 200,000 tokens for a particular task; after leaning on file-system primitives, that went down dramatically — a direct result of giving my agent code execution. Instead of reading a full CSV into context, I give my agent the ability to write and run Python. Cost went down because we're not using as many tokens, and execution time went down too. This was a good case where using primitives over more stagnant tools was clearly the right decision; it won't happen all the time, but here it did.
Now subagents. We had 12 tools, three of which wrapped subagents. The two main instances where subagents are effective: first, when you want to throw a lot of Claude at a problem — deep research, web search, codebase exploration — where many minds running at the same problem makes sense and you can parallelize. Second, when you need a fresh mind to look at a problem. As a developer, I don't want to be the same person writing and reviewing my code. In Claude Code it makes sense to have one instance of Claude writing code and another reviewing it without context about the first instance. We kept one subagent in our inventory agent — for forecasting. I want my forecasting separate from my main instance of Claude; I don't want anything in my initial context window to distort the forecasting process. I have a skill that walks through the step-by-step sequence for forecasts, but I don't want the same Claude my customer is talking with to also write the forecast.
We're not going to expose our subagent as a tool. Using Claude managed agents, there's a native capability for subagents that allows logging and observability to be effective. One problem with subagents is that communication between orchestrators and subagents is hard to get accurate and seamless — a lot can get lost in translation. Logging is also difficult because you have to collect transcripts from multiple agents. Within Claude managed agents, we've added this native "callable agents" capability — like managed subagents — so within your session information you have observability and metrics about what your subagents are doing, as accurate as your orchestrator. You can also just define your subagent as a tool, which we did previously, but we moved away from that to use the CMA native capability. There are a lot of cases where you can scrap the subagent entirely and give more flexibility to your main agent, because frontier models have gotten intelligent enough to manage more information — so you don't need as many subagents.
Let's look at the architecture we ended with. We started with an orchestrator, a ~400-line system prompt, 12 tools (three of them subagents). We ended with an orchestrator deployed on Claude managed agents, because I didn't want to worry about infrastructure, scaling, or security — I just wanted to worry about my agent. We simplified our tools to three: bash, read, and write. When our agent starts executing, we sync some data into the CMA environment so it can reason across that data. We simplified our system prompt to 15 lines and replaced all of our business logic with skills, so Claude pulls that information into its brain only when it realizes it needs it. As a result, we showed how to start hill climbing on evals — my eval score is now about 92%, up from the baseline. I've simplified my design, leaned into the primitives that make Claude great, and seen positive results: we're faster, using fewer tokens because of code execution. Turn count stays about the same, but because token usage and cost go down, I'm okay with Claude taking more turns. For high-intelligence agents where forecasting is at play, I'm willing to take a little higher latency in exchange for better performance and lower cost.
Some takeaways. When we build agents, we start with a single agent loop equipped with simple primitives that give Claude human-like capabilities — the file system, web search, code execution, sometimes a to-do list — and build accordingly. Next, we use progressive disclosure through skills: instead of stuffing the system prompt, we make information accessible to Claude whenever it realizes it needs it, so we run more efficiently and don't pollute the context window. Lastly, write evals. Hill climbing is a concept we lean on heavily at Anthropic: establish a baseline, tweak your architecture, rerun evals, and get better over time. Make sure your evals are updated as your product capability expands so you're always measuring the things you care about. With that, we're going to wrap. I really appreciate your time today, and I hope you have a great rest of your day at Code with Claude in London.