Legal Reasoning on BarExam MBE (test)
82.1AccuracyReasoning Only
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Reasoning OnlyBackbone=Llama-3.3-70B2026.03 | 82.1 | 13.9 | 49.5 | 50.4 | 1.7 | 30.8 | 6 | 61.5 | |
| Reasoning OnlyBackbone=GPT-oss-120B2026.03 | 77.8 | 1 | 7.7 | 33.3 | 9.4 | 31.6 | 17.9 | 42.9 | |
| Reasoning OnlyBackbone=Gemma-3-27B2026.03 | 65.8 | 4.5 | 20.2 | 28.2 | 7.7 | 45.3 | 7.7 | 42.9 | |
| Reasoning OnlyBackbone=Gemma-3-12B2026.03 | 60.7 | 4.2 | 21 | 22.2 | 5.1 | 43.6 | 12 | 36.6 | |
| EvidenceRL with GRPOBackbone=Llama-3.1-8B2026.03 | 60.7 | 18.9 | 51.9 | 41 | 6 | 26.5 | 4.3 | 67.6 | |
| EvidenceRL with GRPOBackbone=Gemma-3-12B2026.03 | 60.7 | 11 | 33.6 | 37.6 | 6.8 | 23.1 | 9.4 | 62 | |
| Reasoning OnlyBackbone=Llama-3.1-8B2026.03 | 57.3 | -1.7 | 3.3 | 18.8 | 10.3 | 51.3 | 12.8 | 32.8 | |
| Reasoning OnlyBackbone=GPT-oss-20B2026.03 | 54.7 | -8.3 | -1.5 | 15.4 | 12.8 | 53.8 | 9.4 | 28.1 | |
| EvidenceRL with GRPOBackbone=Llama-3.2-3B2026.03 | 50.4 | 7.7 | 29.3 | 26.5 | 10.3 | 36.8 | 6 | 55.9 | |
| EvidenceRL with GRPOBackbone=Gemma-3-4B2026.03 | 48.7 | 7.8 | 33.8 | 26.5 | 6 | 39.3 | 6.8 | 54.4 | |
| Reasoning OnlyBackbone=Gemma-3-4B2026.03 | 47.9 | 5.3 | 14.5 | 19.7 | 10.3 | 44.4 | 6.8 | 41.1 | |
| Reasoning OnlyBackbone=Llama-3.2-3B2026.03 | 42.7 | -3.8 | 5.3 | 13.7 | 15.4 | 46.2 | 9.4 | 32 |