Context

a16z infrastructure partner Lisha Li, a former PhD mathematician, sits down with OpenAI mathematicians Mehtaab Sawhney and Mark Sellke to unpack what actually changed when OpenAI's models (branded internally as Astra in this conversation) began producing real progress on open mathematical problems, including sphere packing, error-correcting codes, and a decades-old open question about "sofic groups." Sawhney and Sellke, both practicing research mathematicians who joined OpenAI after being convinced by GPT-5's capabilities, walk through specific results in technical detail and then step back to discuss what it means for a field when the traditional bottleneck (finding the proof) stops being the hard part. For a PM building or evaluating AI products, this is a rare inside look at how model reasoning, task framing, and human oversight actually interact on genuinely open-ended, hard problems, not benchmark tasks with known answers.

The Big Idea

AI models are not just searching faster than humans across known mathematical literature: their reasoning traces increasingly resemble the actual thought process of an expert mathematician, choosing promising approaches, abandoning dead ends without getting emotionally attached to them, and combining ideas across fields in ways that were previously bottlenecked by human working memory and risk tolerance.

Sawhney and Sellke argue this shift changes what the scarce resource in mathematics research actually is: not the ability to produce a correct proof, but the ability to communicate, absorb, and build on the flood of new results these models can now generate.

Key Insights

Models do not get "polluted" by failed approaches

Sellke observes a concrete human limitation the models sidestep: when a mathematician spends a long time on an idea that ends up not working, that failure gets psychologically tangled up with the idea itself, making it hard to mentally "start fresh" and try a genuinely different direction. A model can simply be told to try again from a clean context, without carrying forward the discouragement or mental anchoring a human would. This is not really about the model always avoiding mistakes; it still backtracks and gets things wrong, but it can reset far more cleanly than a person can.

Models take real mathematical risks humans skip for time reasons

Sawhney's key example: a unit distance problem used an approach originally suggested decades ago by mathematician Paul Erdos, but no human had pushed it through to completion because doing so required an enormous number of extremely finicky details with no guarantee of success. A human mathematician has to weigh that risk against a limited career and limited time; a model, told to pursue the idea, simply works through it. Sawhney calls this "a renaissance of reachable results," meaning problems that were technically approachable all along, but never worth a human's bet, given the odds.

The reasoning traces look like an expert's actual notes, not brute force

Both researchers stress that before publishing these results, they worried outside observers would assume the model was simply guessing at massive scale. What convinced them otherwise was reading the model's own summarized reasoning traces, which Sellke compares to reading a close collaborator's raw notes or emails: somewhat disorganized, but recognizably the same kind of thinking a working mathematician does; picking a plausible approach based on partial knowledge, testing it, and updating quickly when it fails, rather than exhaustively trying every possibility.

A single vague follow-up prompt more than doubled a result's reach

In one case, researchers asked the model to improve a bound on error-correcting codes, and it delivered an improvement using representation theory. They then asked a simple, low-effort follow-up: "can you push this further?" The model responded with a substantially more sophisticated version of the same technique, one that ended up matching a separate, previously unexplained conjecture about sphere packing. Sellke's interpretation: it was not a capability gap, the model was simply being narrowly task-oriented and had not been asked to go further. This is a concrete illustration of how much value can sit behind minimal-effort prompting when a model is otherwise capable but stops at exactly the scope it was given.

Judgment is measured by how much it speeds up problem-solving

Asked directly what "mathematical taste" even means for a model, Sawhney gives a deliberately utilitarian answer: taste is whatever lets you make better judgments that let you solve problems faster. By that definition, a model solving harder and harder problems is, by construction, demonstrating better judgment, since doing so requires correctly pruning a vast space of possible approaches down to a small, tractable set worth actually pursuing.

Mental Models & Frameworks

Separating "taste" from "execution" as two roles

Sellke proposes a working framework for how AI research systems might be structured going forward: one model (or process) responsible for high-level judgment about which direction is promising, and a separate model responsible for the long, grinding work of actually executing that direction to completion, effectively a supervisor and an underling. He frames this less as a hard architectural claim and more as a practical current-state observation: keeping these two functions from "polluting" each other's context currently seems to work well, similar to how a human mathematician benefits from having a colleague step back and ask "what are you doing?" after getting too deep into one approach.

The harness-versus-model debate applies beyond math

Sawhney draws an explicit parallel to applied AI and startups more broadly: there's a constant tension between building scaffolding (a harness) around a model to guide its behavior, and the fact that a very minimal harness sometimes exposes a model's raw capability best, only for the next model generation to make that harness obsolete. He notes some labs are now training models jointly with their harness, partly to keep the system proprietary, and partly because it gives more control over the reasoning traces the model produces. For a PM building AI products, this is a direct analogy: over-engineering an interaction layer around a model can constrain it in ways a newer, more capable model release will make you regret.

Trade-offs & Nuance

Short elegant proofs versus fears of unreadable AI output

Before seeing the actual results, both researchers expected AI-generated mathematics might come out as massive, thousand-page outputs that no human could realistically verify or understand, similar to some existing human-generated proofs that already run hundreds of pages. In practice, the results they discuss (including the sofic-groups counterexample) came out shorter and more elegant than many existing human proofs in the same area. This matters for anyone building tools around AI-generated technical work: the more realistic risk observed so far was not "output too complex to check," it was underestimating how compact and legible frontier-model output could be.

Faster proof production does not remove the human bottleneck, it relocates it

Sawhney argues that when producing a correct proof was extremely hard, that difficulty ensured whoever solved a problem also deeply understood it and could explain it to others, understanding came bundled with the achievement. Now that the model has taken over much of the proof-production difficulty, the bottleneck moves to a different scarce skill: understanding, communicating, and integrating a flood of new results into the field's shared body of knowledge. This has a direct organizational analogy: automating the hardest visible step of a workflow does not eliminate bottlenecks, it exposes whichever step was previously riding along for free.

Practical Application

Push past a model's default stopping point with minimal extra prompting

If a model delivers a genuinely strong result but appears to stop short of its apparent capability, try a low-effort, open-ended follow-up prompt (something as simple as "can you push this further?") before assuming a capability limit. Sawhney and Sellke's own experience with the error-correcting-codes result shows a single vague follow-up unlocked a meaningfully more advanced result, because the model had simply completed the literal scope of the original task rather than hitting a real ceiling.

Separate judgment-heavy prompts from execution-heavy prompts

When using an AI system for a genuinely hard, multi-step problem, consider structuring the interaction so that one pass is dedicated to deciding direction (what approach is worth pursuing, and why) and a separate pass or session is dedicated to grinding out that direction to completion. Sellke's observation is that mixing these roles in one continuous context risks the kind of "pollution" that also limits human problem-solving, where an earlier dead end quietly biases the reasoning that follows it.

Read the model's actual reasoning trace, not just its final answer

Before trusting or building on a surprising AI-generated result, read the summarized reasoning trace itself rather than only the output. Sawhney and Sellke both describe this as the thing that converted their own skepticism: seeing that a model chose a specific, well-motivated path (backtracking when it failed) rather than presenting an opaque final answer built through undocumented brute-force search.

Questions to Consider

• Where in your own workflow are you stopping a capable AI tool at the literal scope of your first prompt, the way OpenAI's Astra model initially stopped its code-bound improvement until researchers simply asked "can you push this further"?
• If an AI tool in your product removes the hardest visible step of a task, what previously-invisible bottleneck (communication, verification, integration into existing work) is likely to surface next, the way understanding and absorbing new results became the bottleneck once producing a mathematical proof got easier?
• Are you evaluating an AI system's output only on the final answer, or are you also checking its reasoning trace to see whether it reflects genuine judgment versus exhaustive brute-force search?
• If you had to split a hard task between a model responsible for high-level direction and a model or process responsible for grinding execution, where would that split naturally fall in your own workflow?

Bottom Line

OpenAI's mathematicians found that once frontier models could take genuine mathematical risks and reset cleanly from failed approaches, the traditional bottleneck in the field, producing a correct proof, stopped being the hard part. The harder resource now is understanding, communicating, and integrating results fast enough to keep up, a pattern likely to repeat in any knowledge-work field where AI first automates the most visible, hardest-looking step of the job.

Case Studies Mentioned

Solving a 50-year-old sphere-packing bound

For decades, the best known bound on how densely spheres can pack into high-dimensional space came from a 1970s paper by two mathematicians and was based on numerics with no explanation for why the number was correct. OpenAI's model not only matched a previously unexplained conjectured value, it proved the bound could not be improved further within its mathematical framework, providing the first real explanation for a number mathematicians had used without understanding for fifty years. Sellke notes he had personally worked on this exact problem for six months as a graduate student and made zero progress, underscoring how difficult the underlying reasoning was.

Disproving a conjecture with a fifteen-page proof

A related, stronger conjecture in probability (the Aldous-Lyons conjecture) had recently been disproved using a 250-page proof built on another 200 pages of prior work, drawing on quantum complexity theory that few mathematicians could fully follow. The model's proof of a narrower but related result, that a non-sofic group exists, achieved something comparable using only about fifteen pages and stayed entirely within standard group theory, with no exotic cross-field machinery required.

People to Follow

Mehtaab Sawhney

A research mathematician specializing in combinatorics who joined OpenAI after being given early access to GPT-5 and quickly becoming convinced of its research potential. He works alongside Mark Sellke at OpenAI on applying frontier models to open mathematical problems.

Mark Sellke

A research mathematician who joined OpenAI shortly after seeing the model achieve a gold-medal-equivalent result at the International Mathematical Olympiad. He and Sawhney had previously collaborated academically before both moving into applied AI mathematics research.

Notable Quotes

"It's very much like reading a colleague's notes. It's a little more disorganized in some way, but especially if you work close enough with a collaborator, sometimes you'll just see them spill out their thoughts in an email to you." (Mark Sellke)

"A nice thing about math is that the ceiling for difficulty of a math problem is pretty high. Even if AI continues getting exponentially better at math, it will plausibly never solve something like P versus NP." (Mehtaab Sawhney)