LSAT
Benchmarks
TaskDatasetSOTA resultTrendResults
LSAT
85.57Pass@1
48
LSAT
54Accuracy
21
LSAT HELM
24.38Balanced Accuracy
17
LSAT
0.29Pass@1
17
LSAT
0.038Mean Improvement
9
LSAT
70.7AUC
9
LSAT PT 150–159
99.1LR Accuracy
8
LSAT Official (test N=77)
100LR Accuracy
8
LSAT
81.1Accuracy
7
LSAT (OOD)
26.58Accuracy
5
LSAT In-Distribution (test)
80.14CPT (90%)
4
LSAT OOD
70.54CPT 85%
4
LSAT
90.01CPT (95%)
4
LSAT
80.07CPT (90%)
4
LSAT (ID)
60.9CPT (80%)
4
LSAT ID queries
70.35CPT (85%)
4
LSAT
90CPT (95%)
4
LSAT
91.75Hypervolume
4
LSAT OOD
61CPT Score (80%)
4
LSAT (test)
0.919Hypervolume
4
LSAT
163Score
3
LSAT (test)
0.707AUC
2