Logical Reasoning and Reading Comprehension on LSAT Official (test N=77)
100LR AccuracyDeepSeek-R1
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DeepSeek-R1system prompt=none, answer reordering=none, sampling=single sample2026.04 | 100 | 96.3 | 98.7 | |
| Kimi K2 Thinkingsystem prompt=none, answer reordering=none, sampling=single sample2026.04 | 100 | 100 | 100 | |
| GPT-5system prompt=none, answer reordering=none, sampling=single sample2026.04 | 98 | 100 | 98.7 | |
| Claude Opus 4system prompt=none, answer reordering=none, sampling=single sample2026.04 | 98 | 100 | 98.7 | |
| Gemini 2.5 Prosystem prompt=none, answer reordering=none, sampling=single sample2026.04 | 94 | 100 | 96.1 | |
| QwQ-32Bsystem prompt=none, answer reordering=none, sampling=single sample2026.04 | 92 | 92.6 | 92.2 | |
| DeepSeek-R1 Distill 7Bsystem prompt=none, answer reordering=none, sampling=single sample2026.04 | 54 | 59.3 | 55.8 | |
| DeepSeek-R1 Distill Llama 8Bsystem prompt=none, answer reordering=none, sampling=single sample2026.04 | 46 | 66.7 | 53.2 |