A dataset of 12,500 challenging competition mathematics problems requiring multi-step reasoning.

MATH

Anthropic's most intelligent model to date and the first hybrid reasoning model on the market. Claude 3.7 Sonnet can produce near-instant responses or extended, step-by-step thinking that is made visible to the user. API users have fine-grained control over how long the model can think for (up to 128K tokens). Priced at $3 per million input tokens and $15 per million output tokens at launch, including thinking tokens.

Claude 3.7 Sonnet

The o1 model series is trained with large-scale reinforcement learning to reason using chain-of-thought. OpenAI o1 is the next model in this series (previously OpenAI o1-preview), with advanced reasoning capabilities that provide new avenues for improving safety and robustness. The model can reason about safety policies in context when responding to potentially unsafe prompts, through deliberative alignment.

Google's best model yet for coding performance and complex prompts, with better understanding and reasoning of world knowledge than any previous release. Features a massive 2 million token context window and the ability to call tools like Google Search and code execution.

Gemini 2.0 Pro

Next iteration of Gemini, released in three versions (Flash, Flash-Lite, Pro). Represents Google's state-of-the-art multimodal model, likely incorporating Mixture-of-Experts at unprecedented scale and targeting dominance in both text and image understanding.

Gemini 2.0 Flash

A powerful Mixture-of-Experts (MoE) language model with 671B total parameters and 37B activated for each token. Features Multi-head Latent Attention (MLA) and DeepSeekMoE architectures with innovative auxiliary-loss-free load balancing and multi-token prediction training. Pre-trained on 14.8T high-quality tokens with only 2.788M H800 GPU hours.

DeepSeek-V3

Best for cost-efficient performance. Better quality than 1.5 Flash, at the same speed and cost. Features a 1 million token context window and multimodal input.

Gemini 2.0 Flash-Lite

GPT-4o ('o' for 'omni') is a multimodal model that accepts any combination of text, audio, image, and video inputs and generates text, audio, and image outputs. It matches GPT-4 Turbo performance on English text and code, with significant improvements for non-English languages. It responds to audio inputs in as little as 232 milliseconds (avg. 320ms), similar to human conversation response time. GPT-4o is 50% cheaper in the API than previous models and features superior vision and audio understanding capabilities.

GPT-4o

Claude 3.5 Sonnet raises the industry bar for intelligence, outperforming competitor models and Claude 3 Opus on a wide range of evaluations, with the speed and cost of a mid-tier model. It operates at twice the speed of Claude 3 Opus.

Claude 3.5 Sonnet

Anthropics fastest model at time of launch, delivering intelligence at blazing speeds. Claude 3.5 Haiku is optimized for quick responses while maintaining high quality.

Claude 3.5 Haiku

Anthropic's most intelligent model at release, with best-in-market performance on highly complex tasks. It can navigate open-ended prompts with remarkable fluency and human-like understanding. Pricing at launch: $15 per million input tokens, $75 per million output tokens.

Rank	Model	Provider	Score	Parameters	Released	Type
1	Claude 3.7 Sonnet	Anthropic	96.2		2025-02-24	Multimodal
2	o1	OpenAI	94.8		2024-09-12	Multimodal
3	Gemini 2.0 Pro	Google	91.8		2025-02-05	Multimodal
4	Gemini 2.0 Flash	Google	90.9		2025-02-25	Multimodal
5	DeepSeek-V3	DeepSeek	90.2	671B total, 37B activated	2024-12-26	Text
6	Gemini 2.0 Flash-Lite	Google	86.8		2025-02-25	Multimodal
7	GPT-4o	OpenAI	76.6		2024-05-13	Multimodal
8	Claude 3.5 Sonnet	Anthropic	71.1		2024-06-20	Multimodal
9	Claude 3.5 Haiku	Anthropic	69.4		2024-10-22	Multimodal
10	Claude 3 Opus	Anthropic	60.1		2024-03-04	Multimodal

MATH

MATH Leaderboard

About MATH

Description

Methodology

Publication

Related Benchmarks

AIME-2024

AIME-2025

AIME