GPT-5 and the future of mathematical discovery

How GPT-5 helped mathematician Ernest Ryu solve a 40-year-old open problem | OpenAI

November 24, 2025

How GPT‑5 helped mathematician Ernest Ryu solve a 40-year-old open problem

How a mathematician used GPT‑5 to explore ideas faster and find a path to solving a long-standing optimization problem.

Share

The problem and why it mattered

Every significant math problem has a story—someone who posed a question, someone who tried to solve it, someone who could not, and eventually, maybe, someone who could. The story behind answering one frustratingly simple optimization theory question⁠ is no different, except the researcher worked with a tool capable of quickly surfacing ideas and techniques from across a wide range of mathematical papers.

With 15 years in applied mathematics and optimization theory, Professor Ernest Ryu of the University of California, Los Angeles (UCLA), was curious about the large language model (LLM) everyone was talking about. In 2023, he decided to test ChatGPT‑3.5’s ability to solve simple math and logic problems, like scheduling meetings with multiple people across time zones. He noticed it would understand implicit constraints (like no one wanting to have a meeting between 12 a.m.–6 a.m.) but its ability to produce accurate results widely varied. It had many strengths, but in his opinion, it still had a long way to go.

When OpenAI unveiled GPT‑5 two years later, Ryu began hearing about its rapidly advancing capabilities in mathematics. He decided to try again now that the model had matured to see if it could handle a more complex problem. He did not expect that this would meaningfully contribute to solving a longstanding question in his field.

Ryu decided to tackle an “open” problem, meaning it was unsolved and recognized in the community as something of interest. His mathematical intuition told him it may admit a simple solution; a human just wasn’t able to find it yet.

The question: when an algorithm uses a phenomenon called the Nesterov Accelerated Gradient, or NAG, it becomes dramatically faster—but does the extra momentum added from NAG not affect the algorithm’s stability?

Unlike a car engine, which might fail if pushed too hard for too long, NAG seemed to deliver a “speed boost” without introducing instability. For decades, researchers observed this behavior, but they could not fully explain the underlying reason why the method remained both fast and stable.

NAG was first introduced in 1983 by mathematician Yurii Nesterov and is an optimization method that uses a form of prediction—often described as “looking ahead”—to speed up how quickly an algorithm converges. Unlike traditional algorithms, which take incremental steps and calculate a gradient—or the direction and steepness of a function's slope at a given point—NAG calculates the gradient at a look-ahead point, enabling algorithms to make a more informed final update. By anticipating where a parameter will be, as opposed to its current state at the time of calculation, NAG helps guide the algorithm’s steps more effectively, converges faster, and manages its inherent oscillations as it approaches the function’s minimum, resulting in quicker progress toward a solution.

To put it more simply, optimization theorists wondered why an increase in an algorithm’s momentum did not significantly affect its stability. In training machine learning models or solving engineering problems, efficiency is critical to avoid wasting computational resources and producing slower results.

“By expanding the theoretical tools available in optimization theory, we collectively design algorithms optimized for efficiency, stability and safety,” Ryu said. He and several others had tried solving this mystery years before but failed to produce a mathematical proof that uncovered the solution.

Exploring the problem with GPT‑5

As Ryu continued prompting GPT‑5, often into the late hours of the night after his kids went to sleep, he was impressed with its creativity and unconventional approaches in attempting to solve this fundamental problem. However, as he double-checked the model’s work, he would notice mistakes in its reasoning. The problem remained unsolved.

“It had an interesting approach that I had not thought about,” he said—just like the human minds he valued so much in mathematical brainstorming. “So, that’s why I decided, okay, let me push this further.”

Throughout this process, GPT‑5 was not inventing new mathematical tools and principles. It was very good at wielding existing tools and finding equations, solutions and ideas from papers slightly outside the given area of expertise, which Ryu might not have come across otherwise. “It astonished me with the weird things it would try. Its ability to pull from this massive scale of reading and learning is what makes it really powerful.”

Ryu kept prompting, all the while treating GPT‑5 as a collaborator he would bounce ideas off of. He noticed it would continue to produce creative ideas that pushed him in new and unexpected directions. When he posed a question, it would offer a direction—right or wrong—and Ryu would assess it quickly. If the idea felt like a dead end, he would pivot immediately; if it showed promise, he would pursue it to see where it led him. The sheer speed of this process condensed what would normally take days into hours.

“GPT‑5 was a very unusual collaborator,” he said, “in that it would propose something completely out of the blue.”

> “The way math research works is you have ideas, and whenever you or your colleagues come up with a rough idea, you have a sense of whether or not it is going to work. This is where the partnership between AI and humans can work especially well.”

GPT‑5 enabled Ryu to consider new potential paths for solving the proof that he would not have otherwise thought about, whether because he did not see a connection or because they were from an adjacent field of mathematics he was not as familiar with. As Ryu continued to explore ideas with GPT‑5, one thing became more and more apparent: these systems can be powerful exploratory tools when paired with subject-matter expertise and careful verification.

“The way math research works is you have ideas, and whenever you or your colleagues come up with a rough idea, you have a sense of whether or not it is going to work, like a mathematical intuition,” Ryu said. “This is where the partnership between artificial intelligence and humans can work especially…