Open LLM Leaderboard (archived)

The final leaderboard table of Hugging Face's Open LLM Leaderboard, which evaluated open-weight language models on six benchmarks (IFEval, BBH, MATH level 5, GPQA, MuSR, MMLU-Pro) with one standardized harness until its retirement in March 2025. Scores are the leaderboard's published normalized values on a 0-100 scale (higher is better); the average is the mean of the six. Community-flagged submissions are excluded, mirroring the leaderboard's own default view.

Metrics

• average
• ifeval
• bbh
• math_lvl_5
• gpqa
• musr
• mmlu_pro

Targets (753)

• 01-ai/Yi-1.5-34B-32K
• 01-ai/Yi-1.5-34B
• 01-ai/Yi-1.5-34B-Chat-16K
• 01-ai/Yi-1.5-34B-Chat
• 01-ai/Yi-1.5-6B
• 01-ai/Yi-1.5-6B-Chat
• 01-ai/Yi-1.5-9B-32K
• 01-ai/Yi-1.5-9B
• 01-ai/Yi-1.5-9B-Chat-16K
• 01-ai/Yi-1.5-9B-Chat
• 01-ai/Yi-34B-200K
• 01-ai/Yi-34B
• 01-ai/Yi-34B-Chat
• 01-ai/Yi-6B-200K
• 01-ai/Yi-6B
• 01-ai/Yi-6B-Chat
• 01-ai/Yi-9B-200K
• 01-ai/Yi-9B
• 01-ai/Yi-Coder-9B-Chat
• 1024m/QWEN-14B-B100
• abacusai/bigstral-12b-32k
• abacusai/bigyi-15b
• abacusai/Dracarys-72B-Instruct
• abacusai/Liberated-Qwen1.5-14B
• abacusai/Llama-3-Smaug-8B
• abacusai/Smaug-34B-v0.1
• abacusai/Smaug-72B-v0.1
• abacusai/Smaug-Llama-3-70B-Instruct-32K
• abacusai/Smaug-Mixtral-v0.1
• abacusai/Smaug-Qwen2-72B-Instruct
• ai21labs/Jamba-v0.1
• allenai/Llama-3.1-Tulu-3-70B (bfloat16)
• allenai/Llama-3.1-Tulu-3-70B-DPO
• allenai/Llama-3.1-Tulu-3-70B (float16)
• allenai/Llama-3.1-Tulu-3-70B-SFT
• allenai/Llama-3.1-Tulu-3-8B (bfloat16)
• allenai/Llama-3.1-Tulu-3-8B-DPO
• allenai/Llama-3.1-Tulu-3-8B (float16)
• allenai/Llama-3.1-Tulu-3-8B-RM
• allenai/Llama-3.1-Tulu-3-8B-SFT
• allenai/OLMo-1.7-7B-hf
• allenai/OLMo-1B-hf
• allenai/OLMo-2-1124-7B-Instruct
• allenai/OLMo-7B-hf
• allenai/OLMo-7B-Instruct-hf
• allenai/OLMoE-1B-7B-0125-Instruct
• allenai/OLMoE-1B-7B-0924
• allenai/OLMoE-1B-7B-0924-Instruct
• alpindale/magnum-72b-v1
• amd/AMD-Llama-135m (bfloat16)

…and 703 more.

Published by Hugging Face Open LLM Leaderboard.