BrUMO
Benchmarks
TaskDatasetSOTA resultTrendResults
BRUMO 2025
93.3Accuracy
52
BrUMO '25
1Rank
40
BRUMO (DEF.)
66.67Pass@128
30
BRUMO 2025
60.83Accuracy
21
BRUMO25
78.75Avg@k Score
20
BRUMO
826Trace Count
20
Brumo 25
613Trace Count
20
BRUMO25
54.4Pass@1
18
Brumo
81.9Pass@1 Accuracy
15
BrUMO25
94.58Pass@1
14
BRUMO
80Accuracy
12
BRUMO 2025
69.48PASS@1
11
BRUMO 2025 (test)
75.7Pass@1 Accuracy
11
Brumo 2025
54.79Accuracy
10
BRUMO 2025
30.4Pass@1
8
Brumo 2025
96.4Pass@1
8
BRUMO 2025
63.3Avg@32 Score
8
BRUMO'25
47.39Pass@1
4
BRUMO 2025
23.3Majority@2048
4
BRUMO
73.3REST Score
4
BRUMO 2025
86.2Pass@1
3
BRUMO 2025
4.79Pass@1
2
BRUMO 2025
51.42Pass@4
2
BrUMO 25
0.954Mean Kendall's tau_b
1
BrUMO'25
0.858Kendall's tau_b (vs. Gold Standard)
1