HLE
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
HLE Verified
73.6Score
8
HLE
29.7HLE Score (Text)
8
HLE
33.3Score
8
HLE
60Success Rate
8
HLE
17.6Performance Score
8
HLE
98Coverage
7
HLE
3.61Accuracy
7
HLE (Humanity’s Last Exam) (official)
42Exact Match
7
HLE (test)
0.031ECE
7
HLE
41.6Overall Score
7
HLE (test)
16.6EM
6
HLE Text-only
13.7Accuracy
6
HLE
45.8Pass@1
6
HLE MC
17.4Pass@1
6
HLE
57Last Epoch Success Rate
6
HLE
26.5Accuracy (HLE)
5
HLE
83.3WR (%)
5
HLE Verified (Gold)
60Accuracy (HLE Verified Gold)
5
HLE
20.5Accuracy
5
HLE text-only
45.8Pass@1
5
HLE
37.7Accuracy
5
HLE
27Performance
5
HLE no tool, text only
31.7Score
4
HLE no tool
22.4Pass@1
4
HLE-Med
36.91Pass@1
4