OpenAI-o4-mini has been tested across a wide range of benchmarks, demonstrating state-of-the-art performance in multiple domains. Here are the results: (more details here)
AIME 2024 — Competition Math
| Model | Accuracy (%) |
|---|---|
| o1 | 74.3 |
| o3-mini | 87.3 |
| o3 (no tools) | 91.6 |
| o4-mini (no tools) | 93.4 |
AIME 2025 — Competition Math
| Model | Accuracy (%) |
|---|---|
| o1 | 79.2 |
| o3-mini | 86.5 |
| o3 (no tools) | 88.9 |
| o4-mini (no tools) | 92.7 |
Codeforces — Competition Code
| Model | ELO |
|---|---|
| o1 | 1891 |
| o3-mini | 2073 |
| o3 (with terminal) | 2706 |
| o4-mini (with terminal) | 2719 |
GPQA Diamond — PhD-Level Science Questions
| Model | Accuracy (%) |
|---|---|
| o1 | 78.0 |
| o3-mini | 77.0 |
| o3 (no tools) | 83.3 |
| o4-mini (no tools) | 81.4 |
Humanity’s Last Exam — Expert-Level Questions Across Subjects
| Model | Accuracy (%) |
|---|---|
| o4-pre | 8.12 |
| o3 (no tools) | 13.40 |
| o3 (tools) | 20.32 |
| o4-mini (no tools) | 24.90 |
| o3 (w/ Python + browsing) | 14.28 |
| o4-mini (w/ Python + browsing) | 17.70 |
| Deep research | 26.60 |
MMMU — College-Level Visual Problem-Solving
| Model | Accuracy (%) |
|---|---|
| o1 | 77.6 |
| o3 | 82.9 |
| o4-mini | 81.6 |
MathVista — Visual Math Reasoning
| Model | Accuracy (%) |
|---|---|
| o1 | 71.8 |
| o3 | 86.8 |
| o4-mini | 84.3 |
ChartXiv-Reasoning — Scientific Figure Reasoning
| Model | Accuracy (%) |
|---|---|
| o1 | 55.1 |
| o3 | 78.6 |
| o4-mini | 72.0 |
SWE-Lancer: IC SWE Diamond — Freelance Coding Tasks
| Model | Dollars Earned |
|---|---|
| o4-high | $28,500 |
| o3-mini-high | $17,375 |
| o3-high | $65,250 |
| o4-mini-high | $56,375 |
SWE-Bench Verified — Software Engineering
| Model | Accuracy (%) |
|---|---|
| o1 | 48.9 |
| o3-mini | 49.3 |
| o3 | 69.1 |
| o4-mini | 68.1 |
Aider Polyglot — Code Editing
| Model | Accuracy (%) Whole | Accuracy (%) Diff |
|---|---|---|
| o1-high | 64.4 | 61.7 |
| o3-mini-high | 66.7 | 60.4 |
| o3-high | 81.3 | 79.6 |
| o4-mini-high | 68.9 | 58.2 |
Scale MultiChallenge — Multi-turn Instruction Following
| Model | Accuracy (%) |
|---|---|
| o1 | 44.93 |
| o3-mini | 39.89 |
| o3 | 56.51 |
| o4-mini | 42.99 |
BrowseComp — Agentic Browsing
| Configuration | Accuracy (%) |
|---|---|
| o4 + browsing | 1.9 |
| o3 with python + browsing | 49.7 |
| o4-mini with python + browsing | 28.3 |
| Deep research | 51.5 |
Benchmarks glossary
- AIME – Competition math benchmark from AMC; tests advanced problem solving.
- Codeforces – Real-world competitive programming ELO scores.
- GPQA Diamond – Graduate-level science and engineering multiple-choice questions.
- Humanity’s Last Exam – Expert-level cross-domain reasoning across subjects.
- MMMU – Visual problem-solving tasks from undergraduate STEM fields.
- MathVista – Combines math reasoning with visual understanding.
- ChartXiv-Reasoning – Requires interpreting scientific figures and charts.
- SWE-Lancer – Freelance coding tasks evaluated by money earned.
- SWE-Bench Verified – Software engineering benchmark with GitHub issues.
- Aider Polyglot – Code editing benchmark evaluating full vs. diff completions.
- Scale MultiChallenge – Multi-turn instruction following tasks.
- BrowseComp – Measures agentic tool use (browsing + Python) in open-ended tasks.