MuSR
Benchmarks
TaskDatasetSOTA resultTrendResults
MuSR
73.33Accuracy
53
MuSR 0-shot
48.82Reasoning Score (0-shot)
46
MuSR
69Accuracy
41
MuSR
71.89Accuracy
32
MUSR
61.67Accuracy
31
MUSR
50.77Accuracy (Multi-choice)
27
MUSR
42.12MUSR Performance
24
MUSR
65.86Accuracy
22
MuSR (test)
73.9Accuracy
17
MuSR
65.3Accuracy
12
MUSR murder (test)
75Accuracy
12
MuSR n=756 (test)
98.8Activation Score
11
MuSR n=756 (test)
100Activation (Act)
11
MuSR
56.48MuSR Score
10
MuSR
43.12Accuracy
10
MuSR
37.14MuSR Score
9
MuSR 90-trace
83.66AUROC (Self-doubt)
7
MuSR
0.731AOC
7
MuSR
33.6AOC
7
MuSR
68.76Exact Match
6
MUSR team (test)
69Accuracy
6
MUSR object (test)
68.9Accuracy
6
MUSR team (test)
69Accuracy
6
MUSR object (test)
68.9Accuracy
6
MUSR-fr
33.79Average Score
6