smplmark

HELM Capabilities

Language models scored on five scenarios by the Stanford HELM Capabilities leaderboard.

Aggregated scores from the HELM Capabilities leaderboard, published by Stanford's Center for Research on Foundation Models (crfm.stanford.edu/helm). Each model row carries the release's mean score and per-scenario scores for MMLU-Pro, GPQA, IFEval, WildBench, and Omni-MATH on a 0-1 scale (higher is better), exactly as published. HELM entered maintenance mode in June 2026, so these results are final.

Metrics

  • mean_score
  • mmlu_pro
  • gpqa
  • ifeval
  • wildbench
  • omni_math

Targets (68)

  • OLMo 2 32B Instruct March 2025
  • OLMo 2 13B Instruct November 2024
  • OLMo 2 7B Instruct November 2024
  • OLMoE 1B-7B Instruct January 2025
  • Amazon Nova Lite
  • Amazon Nova Micro
  • Amazon Nova Premier
  • Amazon Nova Pro
  • Claude 3.5 Haiku (20241022)
  • Claude 3.5 Sonnet (20241022)
  • Claude 3.7 Sonnet (20250219)
  • Claude 4.5 Haiku (20251001)
  • Claude 4 Opus (20250514)
  • Claude 4 Opus (20250514, extended thinking)
  • Claude 4 Sonnet (20250514)
  • Claude 4 Sonnet (20250514, extended thinking)
  • Claude 4.5 Sonnet (20250929)
  • DeepSeek-R1-0528
  • DeepSeek v3
  • Gemini 1.5 Flash (002)
  • Gemini 1.5 Pro (002)
  • Gemini 2.0 Flash
  • Gemini 2.0 Flash Lite (02-05 preview)
  • Gemini 2.5 Flash-Lite
  • Gemini 2.5 Flash (04-17 preview)
  • Gemini 2.5 Pro (03-25 preview)
  • Gemini 3 Pro (Preview)
  • IBM Granite 3.3 8B Instruct
  • IBM Granite 4.0 Small
  • IBM Granite 4.0 Micro
  • Marin 8B Instruct
  • Llama 3.1 Instruct Turbo (405B)
  • Llama 3.1 Instruct Turbo (70B)
  • Llama 3.1 Instruct Turbo (8B)
  • Llama 4 Maverick (17Bx128E) Instruct FP8
  • Llama 4 Scout (17Bx16E) Instruct
  • Mistral Instruct v0.3 (7B)
  • Mistral Large (2411)
  • Mistral Small 3.1 (2503)
  • Mixtral Instruct (8x22B)
  • Mixtral Instruct (8x7B)
  • Kimi K2 Instruct
  • GPT-4.1 (2025-04-14)
  • GPT-4.1 mini (2025-04-14)
  • GPT-4.1 nano (2025-04-14)
  • GPT-4o (2024-11-20)
  • GPT-4o mini (2024-07-18)
  • GPT-5.1 (2025-11-13)
  • GPT-5 (2025-08-07)
  • GPT-5 mini (2025-08-07)

…and 18 more.

Published by Stanford HELM.