Anthropic Just Published How Much of Its Own R&D Claude Runs. Steal the Ladder for Your Own Agent Workflow

On September 17, the Anthropic Institute published a prototype R&D Automation Index: a measurement of how much of Anthropic's own AI research and engineering work Claude performs. The headline numbers are Anthropic's own, and the company lists caveats. But the method is more useful to builders than the headline. It gives you a vocabulary for something every vibe coder faces daily: how much should I let the agent do, and how closely should I watch it?

What You'll Learn

What the AL0 to AL5 automation scale means, what the reported numbers do and do not show, and a five-step way to apply the same ladder to your own AI-assisted projects: classify tasks, match oversight to level, monitor the action stream, track drift, and keep a human-verified release. The Vibe Coding Ebook covers the security and tooling side in its monthly intel brief.

What Anthropic reported

The index catalogs R&D tasks, rates how automated each one is, and aggregates the ratings. Anthropic identified roughly 15,000 granular tasks from July 2026 sampling and organized them into a hierarchy of 542 nodes, 378 of them leaf categories. Each task is rated on a six-point scale:

• AL0 to AL2: no AI involvement up to minimal involvement.
• AL3, "collaborates": AI can do large chunks of work under close human direction.
• AL4, "leads": AI can complete most of the task end to end from a high-level prompt while a human supervises.
• AL5, "autonomous": not yet achieved at Anthropic, per the report.

The reported figures for August 2026: about 26% of Anthropic's AI R&D work at AL4, versus under 1% in February. Over 90% of work at AL3 or above. About 30,000 agents working at once. Anthropic says its online monitors cover 100% of activity and block roughly 0.002% of agent actions, about 1 in 47,000.

What to be careful about

These are self-reported numbers, and Anthropic itself notes that ratings depend on a judge model and need human validation, that the task list is frozen (so it measures automation of existing work, not brand-new work), and that some compute-share figures come from a one-week snapshot. Read it as one lab's instrument, not an industry benchmark.

Step 1: Put every task on the ladder

List the recurring tasks in your project: writing components, migrations, test generation, dependency updates, deploys, content edits. Give each an honest level using Anthropic's definitions. If you write a detailed spec and review every line the agent produces, that is AL3. If you hand over a one-line goal and check the result, that is AL4. Most people overestimate how many of their tasks are AL4 because the agent's output looks finished.

Step 2: Match oversight to the level

AL3 work needs diff-by-diff review, because you are the quality gate. AL4 work cannot rely on reading every line, so the check has to move to outcomes: tests that fail when behavior is wrong, type checks, a preview you actually click through, and a sampled diff review. If a task is AL4 and has no automated check, it is unsupervised in practice. Either add the check or drop the task back to AL3. We covered this separation of finding and checking in verify the work, not the demo.

Step 3: Monitor the action stream, not just the output

The most transferable number in the report is the monitor: full coverage, with blocks so rare they run at about one in 47,000 actions. Copy the shape of that design at your scale. Log every command and file write the agent makes. Use an allowlist for what it may run, and hard-block destructive operations (force pushes, recursive deletes, production database writes, secret reads). Then track your block rate. Frequent blocks mean the agent is mis-scoped for the task. A block rate of exactly zero means you should test that the monitor works by deliberately attempting a forbidden action in a sandbox.

Step 4: Watch for drift

Once a week, recount how many of your tasks sit at each level. Anthropic's own AL4 share moved from under 1% to 26% in about six months, and that speed is the point: delegation creeps upward because it is convenient. The risk is that your ability to check does not creep upward with it. If AL4 tasks grow faster than your tests, review time or monitoring, you are accumulating unverified work.

Step 5: Keep a human-verified release

Even Anthropic reports no AL5, fully autonomous work. Copy that boundary. Whatever else you automate, keep one step where a person confirms what ships to production: the merge, the deploy, the payment or data migration. Agents can prepare and test the release. A human owns the button.

A worked example

Suppose you are building a small SaaS with an AI coding agent. Dependency bumps with a passing test suite: AL4, checked by CI. New billing logic: AL3, you review each diff and run the checkout flow by hand. Production deploys: agent prepares, you approve. Every agent command is logged, destructive ones are blocked, and every Friday you spend ten minutes recounting tasks by level and asking whether any AL4 task lacks a real check.

Conclusion

The value of Anthropic's index for builders is not the 26%. It is the habit of measuring delegation instead of feeling it: name the level, match the oversight, monitor the actions, watch the drift, keep a human at release. Apply that to your own projects this week and you will know, with numbers, how much you are trusting your agents and whether your checks have kept up. For more weekly analysis like this, join the Vibe Coding Academy newsletter.