August 29, 2023
ChatGPT has been struggling with math problems according to a study by Stanford University and UC Berkeley.
The study found that ChatGPT can perform simple math operations with small numbers, but it struggles with large numbers and complex problems. The study also found that ChatGPT’s performance has been getting worse over time.
The introduction of GPT-4 showed improvements in handling mathematical queries, yet persistent errors stemmed primarily from miscalculations and arithmetic blunders. Acknowledging this issue, Google, a prominent player in the tech industry, stepped in to address the limitations of LLMs. Their research, titled “Teaching Language Models to Reason Algorithmically,” embraces an in-context learning approach to enhance reasoning capabilities.
This algorithm helps models like ChatGPT understand and perform logical processes better. Google thinks this can improve math skills in AI chatbots. The new method teaches models step by step instead of all at once. Researchers, including Stanford’s James Zou, tested ChatGPT on 1,000 numbers. GPT-4 identified 84% as prime in March, but dropped to 51% by June, showing declining performance.
Wolfram Research, a pioneer in fusing technology with math education, is also working on a way to improve ChatGPT’s math skills with a plugin that solves math problems and creates equations. The plugin offers step-by-step solutions, visuals like graphs, and converts text into math equations using ChatGPT’s language skills and Wolfram’s programming expertise.
Conrad Wolfram, co-founder, shared that after the implementation, ChatGPT scored 43% independently on a math exam, but when paired with Wolfram’s resources, it scored 96%.
The sources for this piece include an article in AnalyticsIndiaMag.
