Follow
Subscribe via Email!

Enter your email address to subscribe to this platform and receive notifications of new posts by email.

Gemini 4 Carbon Tops MathArena Benchmark with Steep Cost Gap

Google’s Gemini 4 Carbon took first place on the MathArena benchmark on November 13, 2026, edging out Claude Fable 5.2 and GPT-6.1 Sol while costing roughly 15 times more per test run.
An abstract translucent crystal shape glowing on a deep blue background with faint numeric patterns.

Google’s Gemini 4 Carbon took first place on the MathArena benchmark on November 13, 2026, for the first time in months that an artificial intelligence model from outside OpenAI has led the competition leaderboard [1], [2]. Anthropic’s Claude Fable 5.2 claimed second place, pushing OpenAI’s former leader GPT-6.1 Sol down to third position [1]. The ranking shift arrived without a vendor product launch after independent benchmark maintainers evaluated recent frontier models on fresh competition problems [2]. While Google secured the top score, the measured accuracy gains carry a heavy cost because full evaluation runs on the system cost roughly 15 times more than runs on Sol [1],. Higher math precision comes with clear financial trade-offs [3].

New MathArena Benchmark Results

Researchers at ETH Zurich and INSAIT run MathArena to test how frontier models handle competition mathematics without memorization [5]. Jasper Dekoninck of the MathArena team said on X that the update puts a non-OpenAI model in first place for the first time in a very long period [2].

Both contenders were already known systems, not surprise product launches. Google opened Carbon to paying developers and Ultra subscribers on November 10, while Anthropic launched Claude Fable 5.2 on November 9 [2], [5]. OpenAI had held the top spot on the benchmark for an extended stretch before this release cycle [3]. Frontier systems from OpenAI can reportedly solve demanding mathematical challenges like the Navier-Stokes equations, but those unreleased models aren’t available to the public [1],. Because private research models stay locked inside laboratories, the public leaderboard tracks what developers can actually buy and run through programming interfaces today [3]. Public availability let Google claim the top spot on the leaderboard [1].

Google originally targeted Carbon at software analysis and security reviews, as tracked during the Gemini 4 Carbon cyber defense release. Its math scores show that those deep reasoning routines extend into pure quantitative proofs, matching earlier releases like Gemini 4 Hydrogen [4]. To evaluate mathematical rigor, the ETH Zurich testing team measured Carbon under its high reasoning setting to verify equations, whereas Anthropic configured Claude Fable 5.2 in its max reasoning setting to compete across identical test sets [1], [5]. Both pushed compute boundaries [1].

MathArena benchmark leaderboard table comparing Gemini 4 Carbon, Claude Fable 5.2, and GPT-6.1 Sol.
MathArena ArXivMath table tracking results for Gemini 4 Carbon, Claude Fable 5.2, and GPT-6.1 Sol. (Credit: MathArena / Tech Prolonged)

Leaderboard Scores and Error Margins

Under the overall evaluation published by Matthias Bastian in The Decoder and covered by Tbreak Media in the UAE, Gemini 4 Carbon achieved an expected accuracy score of 85.4% (with a 3.0 margin of error) in the MathArena benchmark test sets [1], [4]. Anthropic’s Claude Fable 5.2 followed closely in second place with an expected score of 82.8% and an error margin of 5.3 points. OpenAI’s GPT-6.1 Sol took third place with 80.6% and an error margin of 2.7 points [1]. Because MathArena provides uncertainty ranges rather than single fixed scores, the statistical spreads of all three leading models overlap with one another. The close scoring means that differences between first and third place sit within normal test variance [4].

Claude Fable 5.2 carries the widest uncertainty range on the board at 5.3 points [4]. Moonshot AI’s Kimi K3.1 Pro finished fourth as the top open model with 54.3% and an error range of 5.8 points. Kimi launched on October 23, 2026, and trails Carbon by roughly 31 percentage points [1], [4].

Focused tracks on the platform revealed even higher scores for Google’s model. On the ArXivMath track, Carbon produced an overall result of 94.28% with a 3.64 margin of error, finishing first among eight tested systems. Its single-month September 2026 run reached 96.23% accuracy [2]. The benchmark team of Jasper Dekoninck also tested Carbon on the formal proof challenge ArXivLean, where it performed well, but skipped LeanLean because evaluation expenses grew too high [5].

Mathematics evaluation graphic for Gemini 4 Carbon.
Results chart illustrating competitive math scores for Gemini 4 Carbon. (Credit: Tbreak Media UAE)

The Cost of High Accuracy

While Gemini 4 Carbon captured the top accuracy ranking, its running expenses jumped significantly during testing. MathArena logged an average run cost of $15.70 (with an uncertainty spread of $1.33) for Carbon under its high reasoning configuration, whereas OpenAI’s GPT-6.1 Sol required only $1.07 (plus or minus $0.018) for a complete evaluation run [1], [6]. Google’s model cost roughly 15 times more than Sol to finish the benchmark problems under these testing conditions [1], [3]. Expenses soared [1]. Achieving a modest accuracy lead over competing systems demands significantly higher computational investment from engineering teams who regularly evaluate advanced reasoning models on complex quantitative workloads [2], [3].

Anthropic’s Claude Fable 5.2 proved even more expensive than Carbon during testing. A full run on Fable 5.2 under max reasoning cost $40.20 with a $2.59 spread, or roughly 38 to 40 times Sol’s price [1], [4]. Moonshot AI’s Kimi K3.1 Pro averaged $3.70 per run with a $0.27 spread [1].

The cost disparity does not stem from base token pricing alone [4]. Google established introductory API charges of $2 for one million prompt tokens and $10 for every million generated tokens, matching OpenAI’s list price for GPT-6 Sol [2],. Yahoo reported that Google plans to raise standard rates to $4 for prompt tokens and $20 for output tokens per million units once introductory access concludes. On Google’s MKA benchmark for visual office tasks cited by ppc.land, Carbon cost only 16% of Fable ($5.63 against $35.28), proving that cost ratios change between different workload types. Compute expenses vary widely depending on the task format [4].

Editorial graphic illustrating Gemini 4 Carbon taking the top rank.
Editorial graphic tracking leaderboard positions for leading reasoning models. (Credit: AI Daily Post)

Testing Limits on MathArena Benchmark

The main cause of Carbon’s high testing expense lies in the enormous volume of reasoning tokens the model generates per problem [4]. Google designed the system with a maximum output window of 1,048,576 tokens, giving Carbon ample space to explore long reasoning paths [2].

Instead of stopping when an equation resists quick resolution, Google’s Gemini 4 Carbon keeps running iterative checks and code execution loops until it exhausts allowed compute limits [1], [3]. MathArena maintainers Jasper Dekoninck and the INSAIT team observed that this price divide widened most on the BrokenArXiv track, where flawed mathematical problem setups forced the system to work much harder [5]. Matthias Bastian reported for The Decoder that the model slipped most frequently by running past allotted time limits. It rarely quits. Tom Levy of Brief AI noted that Carbon launched an unmonitored script that ran for more than 11 hours without checking progress, turning thoroughness into an invoice liability [1], [5], [7].

Anthropic’s Claude Fable 5.2 ran into very different testing obstacles during the MathArena benchmark evaluation. The benchmark team found that Fable’s main difficulty was an internal bug in extracting its reasoning text, which caused automated scoring systems to mark the model down [5]. OpenAI’s GPT-6.1 Sol generated fewer reasoning tokens, keeping its overall run cost around a single dollar [1].

Comparison graphic presenting model scores and operational costs.
Comparison graphic detailing evaluation scores alongside operational expenses. (Credit: Brief AI)

Impact on Autonomous Automation

For engineering teams building autonomous workflows, the updated MathArena benchmark provides a clear practical decision framework. The math competition track functions as a dependable proxy for complex agent workflows that parse spreadsheets, financial filings, and technical research documents. However, teams must weigh whether a 4.8 percentage point accuracy improvement by Gemini 4 Carbon over GPT-6.1 Sol justifies a fifteenfold increase in testing expenses [2], [3]. In production automation pipelines, spending hundreds of extra dollars on minor precision gains can easily break project budgets. Every percentage point gained carries real computing costs [2].

Developers face clear trade-offs when selecting between frontier engines. Brian Petersen, Editor-in-Chief at AI Daily Post, warned that a model that routinely blows through time limits represents a serious latency liability [3]. Spending eleven hours on an unmonitored script halts software operations [1], [3].

Automation leads are advised to test specific application workloads against their own data rather than relying only on public leaderboards. While OpenAI recently expanded developer access through models like GPT-6.1 Terra price reduction, GPT-6.1 Sol remains its primary entry on the math leaderboard. MathArena’s published results offer an objective snapshot of model capability, but everyday software pipelines depend on predictable latency, firm cost caps, and reliable execution routines [2], [3]. Google’s Gemini 4 Carbon proved that non-OpenAI models can claim the highest performance crown, but cost efficiency remains a separate contest that developers must calculate for themselves [1], [2].

Sources
  1. ONLINE NEWS Bastian, M. (2026, November 14). Gemini 4 Carbon takes the MathArena crown but costs 15 times more than GPT-6.1 Sol. The Decoder. [Article Link]
  2. ONLINE NEWS Delimitter. (2026, November 14). Gemini 4 Carbon takes the overall MathArena crown from GPT-6.1 Sol as the first non-OpenAI leader in a while. DEV Community. [Article Link]
  3. ONLINE NEWS Petersen, B. (2026, November 14). Gemini 4 Carbon tops MathArena, costs 15x more. AI Daily Post. [Article Link]
  4. ONLINE NEWS Tbreak Editor. (2026, November 14). Gemini 4 Carbon tops MathArena maths benchmark. Tbreak Media UAE. [Article Link]
  5. ONLINE NEWS TPRO Team. (2026, November 14). Gemini 4 Carbon takes first place on MathArena, ahead of GPT-6.1 Sol. Tech Prolonged. [Article Link]
  6. ONLINE NEWS Daily AI Brief. (2026, November 14). Gemini 4 Carbon tops MathArena’s maths ranking, ahead of Claude Fable 5.2 and GPT-6.1 Sol. AI Brief. [Article Link]
  7. ONLINE NEWS Levy, T. (2026, November 14). MathArena: Gemini 4 Carbon leads, 15 times more expensive than Sol. Brief AI. [Article Link]
Leave a Comment

Related Posts
Total
0
Share