A road speed limit works because every car has a speedometer and every sign has a number on it. In We Must Pace the Frontier, an essay dated September 2026, Anthropic co-founder Dario Amodei proposes a speed limit on recursive self-improvement, the loop in which AI systems do more and more of the work of building the next AI systems. The essay supplies neither the number nor the speedometer. We read it beside two other Anthropic documents to find out what the term means, and what anyone would have to measure before a limit on it could be enforced.
What is recursive self-improvement in AI?
The essay’s definition is loose. Amodei writes that since roughly this summer AI has been advancing much faster, driven primarily by “AI’s growing ability to build the next generation of AI”, and that this dynamic is called recursive self-improvement. Under that wording the term describes a trend.
The Anthropic Institute sets the bar higher in When AI builds itself, a report the essay points to. The report reserves the term for an AI system that can design and develop its own successor fully autonomously, and it says of that point: “We are not there yet”. So one company uses the same three words in two senses. In the essay, recursive self-improvement is a process that has begun. In the report, it is an end state that has not arrived. A limit on a process and a limit on an end state are different rules.
A third document comes closest to a test. Anthropic’s Responsible Scaling Policy, version 3.4, effective July 8, 2026, is the company’s own rulebook for when a model’s abilities call for stronger safeguards. It sets a threshold for automated research and development, and its change log says that threshold is intended to capture the onset of dramatic recursive self-improvement.
Is recursive self-improvement already happening?
The essay says it is starting to happen across the industry, including at Anthropic. It gives no figure of its own for the claim. The figures are in the Institute report, and they are Anthropic’s account of its own workplace.
According to the report, more than 80% of the code merged into Anthropic’s codebase as of May 2026 was authored by Claude. In the second quarter of 2026 the typical engineer merged 8 times as much code per day as in 2024, a figure the report says almost certainly overstates the real productivity gain. In a March 2026 poll of 130 employees on Anthropic’s research teams, the median respondent estimated about 4 times as much output with Mythos Preview, an Anthropic model, as with no AI models at all. From outside the company, the report cites the length of task a model can finish on its own, which it says has been doubling roughly every four months, up from every seven.
The same report names what is missing. It says large performance gaps persist when Claude has to exercise judgement in choosing goals. That is evidence of people directing much faster work. It is not evidence of a loop that has closed.
None of these figures is a rate of AI progress. They count code merged and task length, and they record what staff estimate. The nearest public thing to an output measure is a benchmark, and benchmarks move in jumps. Our record of how ARC-AGI-3 scores moved in 2026 starts with frontier AI at 0.51% at the launch on 25 March and reaches 62.7% under the shared test setup by 3 September. That is one test, and the Institute report describes other tests going from low scores to saturation within two years.
What is the speed limit on recursive self-improvement?
The essay lists four levels of possible agreement between the United States and China, in order of increasing difficulty. Level 1 prohibits narrow and obviously dangerous uses of AI, such as producing biological weapons. Level 2 has both sides test their models before release for acute risks in cybersecurity, biology and alignment. Level 3 is a speed limit on the rate of recursive self-improvement. Level 4 is a full pacing or pause, in which governments agree to substantially limit the overall rate of AI development.
So the speed limit sits third, above a testing regime and below a pause. Amodei’s case for it is that slowing the rate from extremely fast to only somewhat fast gives up relatively little strategic advantage and could greatly improve safety. He compares it to the SALT treaties, where capping the number of missiles limited the potential for destruction while preserving each country’s deterrent. He rates Level 3 as difficult but just on the edge of being possible, and Level 4 as unlikely to happen any time soon.
The comparison has a weak joint, and Anthropic’s own report says where. Missiles can be counted. The Institute report says training runs are far easier to conceal than missile silos and that their inputs are general-purpose.
Level 3 belongs to the last step of a three-step plan. The first step is embedded evaluators, outside teams with employee-like access to a company. The second is coordination among companies in democratic countries, and the third is global. Part of the second step is to protect the lead of the democracies, which includes a crackdown on unauthorized distillation, where a lagging company trains on a leading model’s outputs. Our breakdown of the distillation campaign Anthropic alleged against Alibaba covers what that fight looks like in numbers.
How would a recursive self-improvement speed limit be measured?
The essay names two ways to pace, both in its section on democracies, and does not say which one a Level 3 agreement would use. The one Amodei prefers is pacing by what a system can do and how safe it is observed to be: a model with a given capability must arrive with certified alignment properties. The other is to limit the ingredients, which he lists as training compute, the nature of training runs, or internal use of AI to improve AI. He worries that some of these are easier to game than external behaviour, and leaves the question to discussion with embedded evaluators.
Training compute is the countable one. The essay does not say what limiting the nature of training runs would involve. Internal use of AI would need something like the 80% code share, and the Institute report’s own footnote says its attribution pipeline has gaps.
A limit on the rate itself asks for more, and the Responsible Scaling Policy shows how much. Its threshold is met in one of two ways. Either Anthropic’s models could fully substitute for its entire set of research scientists and research engineers at competitive cost, defined as within a factor of 5. Or Anthropic observes or expects double the rate of progress in aggregate AI capabilities, set against both the rate it would expect and the fastest extended rate it has seen without significant AI contributions, with the doubling plausibly attributable to automation. A footnote defines double as one year’s progress matching two years at baseline, and extended as at least three model generations. Its worked example is a baseline 9x effective scaleup becoming something like 81x.
Three things would therefore have to be measured: aggregate capability, a baseline for what progress would have been without AI help, and how much of the difference automation caused. The policy names no benchmark or metric for the first. The essay never uses the words baseline or benchmark. The policy’s change log says the threshold “has proven difficult to operationalize”, and that is one company measuring itself. Between rivals the reading also depends on who runs the test. On ARC-AGI-3 we found one model scoring 62.7% and 99.9% on the same day. The lower figure came from the shared test setup and the higher from the vendor’s own.
The first real reading may come from inside Anthropic. The essay says the company intends to invite an embedded external review team in the near future, with the right to publish key findings without editorial control by Anthropic. If that team’s first report puts a figure on the rate of progress inside one company and says how the figure was reached, the speed limit will have a speedometer. Until then Level 3 is a sign with no number on it.
