Andrej Karpathy on Humans as the Bottleneck

When Andrej Karpathy tells you something has changed, it is worth paying attention.

Karpathy is not a startup founder talking his book. He is one of the few people in AI who has had a formative role at both ends of the field's defining institutions: a founding research scientist at OpenAI from 2015 to 2017, then Director of AI at Tesla, where he built the computer vision system behind Autopilot, then a second stint at OpenAI in 2023 to 2024. He created Stanford's CS231n, one of the most influential deep learning courses ever made, growing from 150 students to 750 in its first two years alone. He coined the term "vibe coding" in February 2025. Now an independent researcher and founder of Eureka Labs, named one of TIME's 100 Most Influential People in AI in 2024, he is perhaps the most credible outside observer the field has.

On the No Priors podcast last week, Karpathy described what he called "AI psychosis" — a state he has been in since December 2024, when something flipped.

"I kind of went from 80-20 to like 20-80 of writing code by myself versus delegating to agents," he told host Sarah Guo. "And I don't even think it's 20-80 by now. I don't think I've typed a line of code probably since December."

The shift is not, he argues, primarily about capability. It is about recognising what the actual constraint is. For years, the binding factor in software development was human typing speed, context, attention. Agents dissolved that. Which means the constraint moved.

"You are the bottleneck in the system that is max capability," he said. "I feel nervous when I have subscription left over. That just means I haven't maximised my token throughput."

The GPU anxiety metaphor he reaches for is precise. As a PhD student, he felt nervous when GPUs sat idle. That same anxiety has transferred. The scarce resource is no longer compute flops — it is tokens, and the agent-hours they represent. The human who sits between tasks, deciding what to work on next, is the new inefficiency.

The Loop Closes

Karpathy's project AutoResearch is his attempt to eliminate that inefficiency from the domain he knows best: AI research itself.

The concept is simple and its implications are not. Rather than running ML experiments manually — checking results, adjusting hyperparameters, deciding what to try next — AutoResearch creates a fully autonomous loop. Set an objective, set a metric, define the constraints, and let it run without a researcher in the loop at all.

When he ran it overnight against nanochat, his newly released minimal LLM training codebase, it came back with improvements that two decades of hands-on ML intuition had missed. Weight decay on value embeddings. Insufficiently tuned Adam betas. Small parameters that interact in ways that compound.

"I shouldn't be running these hyperparameter optimisations," he said. "I shouldn't be looking at the results. There are objective criteria in this case."

What Karpathy built as a personal experiment, the frontier labs have been scaling industrially. At the World Economic Forum in Davos in January, Anthropic CEO Dario Amodei described the mechanism in plain terms: "We would make models that were good at coding and good at AI research, and we would use that to produce the next generation of models and speed it up to create a loop." He told the same audience that AI is already doing most of the coding at Anthropic itself, and predicted it would be handling most or all of what software engineers do end-to-end within six to twelve months. The machinery to close that loop fully is being built right now — and the capex chart above is what that conviction looks like in dollars.

What It Implies

Karpathy frames AutoResearch as a "little playpen" for an idea he believes defines the entire frontier. Every research organisation, he argues, can be described as a set of markdown files: roles, processes, decision rules. Those files can be optimised, just as hyperparameters can. Different configurations produce different outcomes. Once a process is code, it can be improved by the same machinery that improves everything else.

"You can definitely imagine that you have multiple research orgs, and they all have code, and once you have code, you can imagine tuning the code," he said.

The implication is structural. Researchers are not eliminated from the picture — but their role migrates. The human contribution shifts from execution (running experiments, writing code, reviewing outputs) toward architecture: designing the environment, setting objectives, deciding what the agents should optimise for. The question "what should we build?" remains human. Nearly everything downstream of that question does not.

That transition is already underway at the desk of one of AI's most credible practitioners. It is almost certainly underway, at a very different scale, inside every major lab on the planet.

—

Sources: No Priors transcript · YouTube episode · AutoResearch on GitHub · Fortune — Amodei at Davos · WEF — Amodei on engineers not writing code · IEEE ComSoc — Capex data