A challenging benchmark of novel problems designed to test the limits of AI capabilities.

Humanitys-Last-Exam

xAI's latest generation model with enhanced mathematical reasoning capabilities, showing significant improvements in competition-level mathematics benchmarks. Features 2x faster end-to-end latency, supports 5 different voices, and achieves 10x daily user seconds compared to previous models.

Grok 4

GPT-OSS-120B is a state-of-the-art open-weight language model that delivers strong real-world performance at low cost. This 120 billion parameter mixture-of-experts model achieves near-parity with OpenAI o4-mini on core reasoning benchmarks while running efficiently on a single 80 GB GPU. It was trained using reinforcement learning and techniques informed by OpenAI's most advanced internal models, including o3 and other frontier systems.

GPT-OSS-120B

Gemini 2.5 Pro is capable of reasoning through its thoughts before responding, resulting in enhanced performance and improved accuracy. Features Deep Think, an enhanced reasoning mode, and native audio outputs that capture subtle nuances of speech.

Gemini 2.5 Pro

A smaller model optimized for fast, cost-efficient reasoning. Achieves remarkable performance for its size and cost, particularly in math, coding, and visual tasks. It is the best-performing benchmarked model on AIME 2024 and 2025, with significantly higher usage limits than o3.

o4-mini

GPT-OSS-20B is a medium-sized open-weight language model that delivers similar results to OpenAI o3‑mini on common benchmarks and can run on edge devices with just 16 GB of memory. This makes it ideal for on-device use cases, local inference, or rapid iteration without costly infrastructure. Despite its smaller size, it demonstrates strong performance on reasoning tasks, tool use, and competition mathematics.

GPT-OSS-20B

Improved across key benchmarks for reasoning, multimodality, code and long context while getting even more efficient. Best for fast performance on complex tasks.

Gemini 2.5 Flash

Google's most cost-efficient and fastest 2.5 model yet. Higher quality than 2.0 Flash-Lite on coding, math, science, reasoning and multimodal benchmarks. Excels at high-volume, latency-sensitive tasks like translation and classification.

Rank	Model	Provider	Score	Parameters	Released	Type
1	Grok 4	xAI	38.6	Unknown	2025-07-09	Multimodal
2	GPT-OSS-120B	OpenAI	19	117B total (5.1B active per token)	2025-08-05	Text
3	Gemini 2.5 Pro	Google	17.8		2025-05-06	Multimodal
4	o4-mini	OpenAI	17.7		2025-04-16	Multimodal
5	GPT-OSS-20B	OpenAI	17.3	21B total (3.6B active per token)	2025-08-05	Text
6	Gemini 2.5 Flash	Google	11		2025-05-20	Multimodal
7	Gemini 2.5 Flash-Lite	Google	6.9		2025-06-17	Multimodal

Humanitys-Last-Exam

Humanitys-Last-Exam Leaderboard

About Humanitys-Last-Exam

Methodology

Publication

Related Benchmarks

BIG-bench

Scale-MultiChallenge