Skip to main content

The thinking lever

TL;DR

  • Matt Bleifer, a product manager on Anthropic's research team, explains how Claude leverages test-time compute to solve hard problems, and the levers — effort and budgets — that let you influence how Claude spends tokens.
  • Tokens fall into three buckets (thinking, tool calling, and text), and a traffic-simulation demo shows how raising the effort dial (low → high → max) trades 10x time and tokens for progressively better results.
  • Claude has evolved from think-then-tool-then-text, to interleaved thinking, to adaptive thinking (think whenever appropriate); effort is recommended over thinking toggles, evals are the best guide, and "extra high" is the default suggestion for coding.

Takeaways

  • Test-time compute (test time compute) scales intelligence by letting models spend more time on a problem, complementary to training-time compute; both improve performance with diminishing returns.
  • Claude can spend three types of tokens: thinking tokens (internal monologue and chain of thought), tool calling tokens (interfacing with the world), and text tokens (communicating with the user).
  • Because tokens cost money and waiting time, users can constrain Claude with an effort dial (trade off time, cost, and quality) and with task budgets (an upper bound in tokens, time, or cost before checking in).
  • The thinking architecture evolved from think → tool → text, to interleaved thinking (reason between tool calls), to adaptive thinking, where Claude can think, call tools, and emit text in any order — or skip thinking for simple queries.
  • Adaptive thinking is not a model router or an automated thinking toggle; it's the difference between forcing at least one thinking token and letting Claude think whenever and however needed, and it has been the benchmark setting since Opus 4.6.
  • Effort is a better abstraction than thinking toggles: a toggle turns a core capability on or off, whereas effort expresses how hard to work and moves thinking, tool use, and text together.
  • Practical effort guidance: max for the hardest tasks (watch for diminishing returns), extra high as the default for coding/agentic use (the Claude Code and Claude.ai default for Opus 4.7), high to balance intelligence and tokens, medium for cost-sensitive, low for short-scope and latency-sensitive work.
  • Low effort can be surprisingly clever — Claude Plays Pokémon on low effort behaved like a speedrun (skipping trainer battles, using healing items and repels) — so always read transcripts, and choose small models for fast time-to-first-token versus bigger models at low effort for fast time-to-last-token.

Vocabulary

Test-time compute — Compute spent at inference time letting a model work longer on a problem to get better results. Reasoning models — Models that scale test-time compute by spending more time reasoning before answering. Thinking tokens — Tokens representing Claude's internal monologue and step-by-step chain of thought. Tool calling tokens — Tokens Claude spends to interface with the world, such as searching or reading and writing files. Effort — A dial telling Claude how to trade off time, cost, and quality when responding. Task budgets — An upper bound (in tokens, time, or cost) on how much Claude spends before checking in. Interleaved thinking — A mode where Claude reasons in between tool calls rather than only up front. Adaptive thinking — The setting where Claude is free to think, call tools, and emit text in any order as needed. Time to first token — How quickly a model starts producing output, where smaller models tend to excel.

Transcript

All right. Hello everyone and welcome. My name is Matt Bleifer. I am a product manager on Anthropic's research team. And today I will be sharing a little bit about how Claude leverages compute at inference time, otherwise known as test-time compute, in order to break down and solve some of your hardest software engineering challenges. Along the way, I'll share a little bit about what levers you have at your disposal in order to influence how Claude spends tokens, and I will also share some best practices to help you get the most out of it.

One of the key developments in large language models over the last couple of years has been the scaling of test-time compute, creating something that we've all come to know as reasoning models. Similar to how we can scale compute at training time by training bigger models over longer time horizons using more data, we can also scale compute at test time by allowing those models to spend more time working on a problem. If you look at this graph on the left, you can see that when we move from Haiku to Sonnet to Opus, as the model gets more intelligent, it's able to get a better score on our agentic coding evaluation. And similarly in the graph on the right, as that same model Opus actually just spends more time working on a problem, it's able to correspondingly get better and better scores. This is what we mean by scaling test-time compute. This isn't true just of software engineering. It's really true of a whole variety of knowledge work domains, whether it's agentic search, computer use, or PhD-level academic reasoning. If we can allow models to spend more time working on a problem, they can achieve better and better results.

Looking at charts and graphs is great for understanding the data, but nothing beats seeing a tangible example. So what I did is I ran Opus 4.7 on a few different effort levels, scaling the amount of time it works on a given prompt, where I asked it to create a realistic simulation of cars going down a one-way street at a traffic light. The first result is Opus 4.7 running on low effort. It took about 50 seconds and worked for about 4,600 output tokens. It accomplished something fairly reasonable — we have cars going down a one-way street, stopping at the traffic light — but overall it's a pretty basic simulation. The traffic flow is fairly basic, the graphics are limited, and for some reason Claude thought it would be a great idea to put the traffic light right in the middle of the road, which maybe wasn't the best design decision, but we'll still call it functionally passing. Next I cranked that effort dial up a bit. When I moved effort up to high, it took about twice the time and double the output tokens, but it achieved a better result. It has cars of different types, it smartly moved the traffic light over to the side of the road, and Opus told me it even implemented what it called an intelligent driver model where every car would respond to the dynamics of the car around it. The last thing I did was crank that dial all the way up to max. In this setting, Opus 4.7 took 10x the amount of time and 10x the amount of tokens it did on low effort, but it achieved the best results yet: the best graphics, my favorite traffic light, and really realistic driving patterns. This is an example of how, even on the same model, when you allow Claude to spend more time on a problem, it can get better results. As we continue to scale test-time compute further, Claude isn't just going to work for seconds or minutes or hours — it's going to work for days, weeks, months, even years, spending tokens to solve some of humanity's toughest challenges.

When I talk about test-time compute, I mean any form of Claude spending tokens at inference time to solve your problem. We can break these token types down into three distinct buckets. The first bucket is thinking tokens. This is the classic form of tokens that underlies what we know as reasoning models. Thinking tokens represent Claude's internal monologue — its space to reason step-by-step, consider different options, do chain-of-thought reasoning, create a scratchpad. The second form is tool calling tokens. Tool calling is Claude's way of interfacing with the rest of the world, whether using tool calls to execute a search or reading and writing files to build out software engineering projects. The last type is text. This is Claude's way of interacting with you — giving updates, a summary at the end, or simply responding to a question. So we have three different types of tokens: thinking, tool calling, and text. All three are fundamental to the way Claude works. But all these tokens have direct costs to users in the form of both practical token costs and waiting time. So we think it's important to give users the ability to influence or constrain how Claude spends tokens.

Users can express their preferences and constraints in a couple of ways. The first is the effort dial. Effort is a way to tell Claude how you want it to trade off time, cost, and quality when responding to your task. Should Claude spend more time to get a better answer, or less time to get a faster answer? Another form of constraint is budgets. Recently we launched a feature we call task budgets, which allows you to tell Claude an upper bound of the amount of tokens it will spend when working on a task. You might say, "I want you to build out this feature, but I don't want you to spend more than 100,000 tokens before you stop and check in with me." Budgets could come in the form of tokens, time, or cost. This will get increasingly important as Claude works for days, weeks, or months. Given all these preferences and constraints, it's up to Claude to figure out the best way to spend those tokens to maximize outcome.

When reasoning models were first introduced, they followed a specific pattern: first they would think, then move on to tool calling, and then lastly to text. We improved on this when we introduced interleaved thinking, which allowed Claude to use thinking and reasoning in between tool calls. In this mode, Claude could call a tool, get a result, think about that result, then determine what to call next, all the way until it decides to give a final answer. Recently we launched adaptive thinking, the next evolution on top of interleaved thinking. In this new paradigm, Claude is free to think whenever appropriate. There's no constraint on when Claude needs to think, how much, or in what order it spends these tokens. It can leverage thinking, tool use, and text in whatever order is needed. Claude could start with a text response to acknowledge the request, stop to call a tool, think about that tool, respond to give an update, continue calling tools, and so on. Claude could also choose not to think at all for simple queries. In practice, Claude will typically think more often and longer in response to higher effort levels, but everything is prompt dependent. If I asked someone "what is 2 + 2?" and asked them to spend a little or a lot of time, they'd spend roughly the same. But the story would change a lot for a sophisticated research task.

Adaptive thinking is not a model router, and it's not an automated thinking toggle. It's not taking your query, classifying it by difficulty, and figuring out whether to use a thinking or non-thinking version of the model. Rather, it's the difference between telling Claude "you must spend at least one thinking token at the start of this response" and telling Claude "you can spend thinking tokens whenever and however needed." It's about Claude having the option to think at every step. We run all of our benchmarks on adaptive thinking since Opus 4.6, and it's our intelligence-maximizing setting that shows performance parity or better with interleaved thinking while delivering a better user experience.

I want to dig more into effort and contrast it to how we've used thinking in the past. Historically, users have used thinking toggles a lot like an effort dial. If you wanted Claude to spend more time, you might turn on thinking inside Claude.ai or Claude Code. That's a reasonable instinct. However, thinking toggles are a poor proxy for an effort dial. Rather than expressing how hard you want Claude to work, you're turning on and off a core capability of the model. You're constraining how it's allowed to work, not how hard you want it to work. An effort dial is a much better expression of the idea "spend more tokens to get a better answer." It moves thinking, tool use, and output text all together instead of toggling one of them. As an analogy with tool use, we don't tell Claude to always search or never search — we tell Claude to figure out when it should search based on the problem at hand. Similarly, when we work with teammates, we don't ask them to turn their inner monologue on and off; we ask them how hard to try.

Let me give you some practical guidance on setting effort levels. First, whenever possible, it's best to run evals and chart performance, comparing on your x-axis something like total tokens, time, or cost, and on your y-axis performance. This lets you create an effort curve and get a better idea of the trade-offs. Higher effort improves performance on most intelligence-bound tasks, but it may show diminishing returns. You might say "I'll spend whatever tokens I need for the best intelligence," or "the improvement between extra high and max isn't worth the difference, so extra high is better for my use case." Low effort can accomplish a task much quicker and save tokens, but it limits how thorough Claude is. As a tip, when using low effort, Claude is trying to save tokens, so sometimes you may catch it taking unexpected shortcuts. In addition to evals, it's a best practice to spend time reading your transcripts. On the flip side, low efforts have surprised us. One of my favorite evaluations is Claude Plays Pokémon, where Claude works through the original Pokémon Red game. When we ran it on low effort, it treated the game much like a speedrun: it skipped trainer battles to save time, used healing items it stocked up on instead of going back to Pokémon Centers, and spammed an item called a repel to limit disruptive encounters, making it through caves more quickly. We might correlate low effort with lower intelligence, but this is actually a super clever strategy. It takes intelligence to minimize token spend and get through levels as fast as possible.

Evals are always ideal, but here are some quick rules of thumb for selecting an effort setting. Max effort can deliver gains on your hardest tasks but can show diminishing returns; test it for your most intelligence-demanding use cases, but don't assume it's the ceiling on performance or the best bang for your buck. Extra high effort is a new setting we introduced with Claude Opus 4.7, and we found it the best for most coding and agentic use cases. It's currently our default in Claude Code and Claude.ai for Opus 4.7, maximizing intelligence without going overboard. High effort is great if you're balancing token usage and intelligence, and it's probably the value I'd recommend for any intelligence-sensitive use case. Medium is good for cost-sensitive use cases. Low is good for short-scope tasks and latency-sensitive workloads, but it's always good to put it in practice and see what happens.

I mentioned that test-time compute is a second way of scaling intelligence compared to training-time compute. So when should you use a smaller model versus a lower effort level on a bigger model? Low effort on a bigger model is good for an intelligence-demanding use case where you're optimizing for speed. Going back to our traffic light simulation, Opus 4.7 on low effort spent about the same output tokens and only took a little longer than Haiku 4.5 on max effort, but achieved a much better result. On the flip side, smaller models are good if you're optimizing cost and your use case is not too intelligence-demanding — simpler LLM tasks like classification, information extraction, and basic summarization, especially in bulk. Another case for small models is if your application demands a really low time to first token. The way I think about it: use small models for a fast time to first token, use bigger models at lower effort for a fast time to last token. Wherever possible, evaluate both.

Before closing, I want to summarize three key actionable items. One, enable thinking whenever possible to give Claude that space to reason. If you want to modulate how much time Claude spends thinking, use effort levels or budgets. Second, if you have evals, use them. Chart your curves, test different effort settings, budgets, and models, and don't forget to read those transcripts. And lastly, if you just need to make a choice and you're working on anything coding and software engineering related, my advice would be go with extra high. It gives a great bang for your buck while delivering great intelligence. Our North Star for Claude overall is that it allocates compute incredibly well when asked for it, and that you can set a quality bar and a budget and Claude will figure out the rest. Adaptive thinking, effort levels, and budgets are all a step in this direction, but they're just the beginning and there's a lot more to come. Thanks so much for taking the time. If you want to chat more about this, I'll be around the conference, and I'm always happy to nerd out about these things. Thank you.

Feedback / ReportSpotted an issue or have an improvement idea?