Logical Reasoning and Reading Comprehension on LSAT PT 150–159
99.1LR AccuracyGPT-5
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-5system prompt=none, answer reordering=none, sampling=single sample2026.04 | 99.1 | 98.5 | 98.8 | |
| Gemini 2.5 Prosystem prompt=none, answer reordering=none, sampling=single sample2026.04 | 99.1 | 97.8 | 98.6 | |
| Claude Opus 4system prompt=none, answer reordering=none, sampling=single sample2026.04 | 98 | 97.8 | 97.9 | |
| DeepSeek-R1system prompt=none, answer reordering=none, sampling=single sample2026.04 | 97.8 | 97.5 | 97.7 | |
| Kimi K2 Thinkingsystem prompt=none, answer reordering=none, sampling=single sample2026.04 | 97.8 | 96.3 | 97.2 | |
| QwQ-32Bsystem prompt=none, answer reordering=none, sampling=single sample2026.04 | 92.9 | 93.8 | 93.2 | |
| DeepSeek-R1 Distill Llama 8Bsystem prompt=none, answer reordering=none, sampling=single sample2026.04 | 57 | 72.8 | 63.2 | |
| DeepSeek-R1 Distill 7Bsystem prompt=none, answer reordering=none, sampling=single sample2026.04 | 54 | 60.9 | 56.7 |