MMLU
knowledge
Massive Multitask Language Understanding (MMLU) tests knowledge across 57 subjects including mathematics, history, law, and more.
Published: 2020
Scale: 0-100
Top Score: 92.3
MMLU Leaderboard
| Rank | Model | Provider | Score | Parameters | Released | Type |
|---|---|---|---|---|---|---|
| 1 | o1 | OpenAI | 92.3 | 2024-09-12 | Multimodal | |
| 2 | DeepSeek-R1 | DeepSeek | 90.8 | 671B (37B activated) | 2025-01-20 | Text |
| 3 | o1-preview | OpenAI | 90.8 | 2024-09-12 | Text | |
| 4 | Claude 3.5 Sonnet | Anthropic | 88.7 | 2024-06-20 | Multimodal | |
| 5 | GPT-4o | OpenAI | 88.7 | 2024-05-13 | Multimodal | |
| 6 | DeepSeek-V3 | DeepSeek | 88.5 | 671B total, 37B activated | 2024-12-26 | Text |
| 7 | Gemini 2.5 Flash | 88.4 | 2025-05-20 | Multimodal | ||
| 8 | Claude Opus 4 | Anthropic | 87.4 | 2025-05-22 | Multimodal | |
| 9 | Claude 3 Opus | Anthropic | 86.8 | 2024-03-04 | Multimodal | |
| 10 | Claude 3.7 Sonnet | Anthropic | 86.1 | 2025-02-24 | Multimodal |
About MMLU
Description
Massive Multitask Language Understanding (MMLU) tests knowledge across 57 subjects including mathematics, history, law, and more.
Methodology
MMLU evaluates models on a scale of 0 to 100. Higher scores indicate better performance. For detailed information about the methodology, please refer to the original paper.
Publication
This benchmark was published in 2020.Read the full paper