AI Model Benchmarks Nov 2025

2,500 of the toughest, subject-diverse, multi-modal questions designed to test for both depth of reasoning and breadth of knowledge. Created in partnership with the Center for AI Safety, HLE includes questions across mathematics, humanities, and natural sciences from nearly 1,000 expert contributors.

METR's time horizon is the human task duration at which an AI model reaches 50% success. Tasks are drawn from RE-Bench, HCAST and SWAA, which cover machine-learning research engineering, general software engineering and software operations respectively.

A human-curated subset of 500 GitHub issues from the SWE-bench dataset tests whether models can implement valid code fixes. The model interacts with a Python repository and must modify the correct files to fix the issue. The solution is judged by running unit tests.

A multiple-choice set of 198 PhD-level science questions in biology, chemistry and physics. It focuses on "Diamond" items for which domain experts answered correctly while non-experts often failed; random guessing yields ~25%.

GDPval is a new OpenAI-led benchmark spanning 44 knowledge work occupations, selected from the top 9 industries contributing to U.S. GDP, from software developers and lawyers to registered nurses and mechanical engineers. These occupations represent the types of day-to-day work where AI can meaningfully assist professionals.

Assesses models' ability to optimize software performance. Each task requires making code changes within a limited number of attempts. Performance is measured using OPT@K - the percentage of tasks where the model achieves at least 95% of the human-achieved speedup.

Tasks require gathering and synthesizing information from the web. Models must plan search queries and extract data from static web snapshots, covering categories like "Finding numbers", "Compiling data" and "Causal inference".

Pits two models against each other to build the best website for a given prompt. Voters rate which site looks nicer and functions better. The results are combined using the Bradley-Terry model, producing a score for each model.

The Level 5 subset of the MATH dataset contains the hardest competition-style problems from AMC 10, AMC 12 and AIME. Answers are scored using a combination of normalized string match, symbolic equivalence and model-graded equivalence.

Asks models to write code that trains machine-learning models to solve non-standard tasks (e.g., recognizing shapes, classifying digits, predicting chess outcomes). Models iterate on code, training and evaluating within a constrained environment.

Inspired by GeoGuessr. For each of 100 street-level photos (sampled from five community maps), models must guess the country and the precise latitude/longitude. Performance is measured using geographic distance and country accuracy; tasks require visual recognition, text extraction and geographic reasoning.

Problems are drawn from Exercism's challenging exercises in C++, Go, Java, JavaScript, Python and Rust. Each model gets two attempts per problem; failing the first attempt yields test errors, allowing the model to edit its code. Pass rate after the second attempt is the primary metric.