Composite
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Composite
66.2Kendall-c Tau_c
161
Composite
139.532RMSE (Yield)
24
Composite (test)
0.6Kendall's Tau
21
Composite (test)
57.6Kendall's Tau-c
18
COMPOSITE (COM) (test)
64.2Kendall's tau-c
17
Composite 1 (test)
57.3Kendall Tau-c
15
Composite-9
94.4Correctness
10
Composite 37 (test)
65Kendall's Tau-c
9
COMPOSITE cultural
-1.9Median % Change vs Neutral
5
COMPOSITE
-0.6Median % Change vs Neutral
5
Composite
0.009Relative L2 Error
5
Composite IFEval-strict-prompt, BFCL v3, CodeIF-Bench, Nexus FC
86.9IFEval Strict Prompt Score
4
Composite (GSM8K, MATH, OlympiadBench, AIME 2025, HARDMath2, Omni-MATH, GSM-Plus, CMATH)
94.62GSM8K
4
Composite CRUXEval-O, MBPP, MBPP+, MultiPL-E, HumanEval, HumanEval+, HumanEvalFix, HumanEval-cn, BigCodeBench-Full, LiveCodeBench, Aider, BIRD-SQL, Spider
76.12CRUXEval-O Score
4
Composite (BIG-Bench Hard, BIG-Bench Extra Hard, bbh-zh, MuSR, ZebraLogic, PrOntoQA, PIQA, OCNLI, HellaSwag, KOR-Bench, DROP, SQuAD 2.0)
83.7BBH
4
Composite (MMLU, MMLU-Pro, CMMLU, C-EVAL, GAOKAO-Bench, ARC-c, GPQA, SciBench, PHYBench, TriviaQA)
65.77Overall Average Score
4
Composite held-out set (Switzerland)
95.4CR
3
Composite Small Towns (held-out set)
95.6CR
3
Composite Yosemite (held-out set)
96.5CR
3
Composite held-out set (Yellowstone)
95.3CR
3
Composite (New York City)
97.3Completion Rate (CR)
3
Composite Los Angeles (held-out set)
96.8CR
3