Legal task processing on Harvey LAB
11.99All-Pass RatePARTHENON
Evaluation results
| Method | Links | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PARTHENONModel=Sonnet 4.62026.06 | 11.99 | 90.2 | 89.9 | 90.1 | 92.4 | 90.3 | 90.6 | 86.5 | 88.6 | 88.4 | 84.1 | 85.9 | 91.4 | 91.5 | 89.9 | 89.4 | 91.2 | 90.6 | 92.9 | 90.7 | 89.5 | 94.8 | 86.5 | 89.1 | 91.5 | 93.8 | |
| Claude CodeModel=Sonnet 4.62026.06 | 11.83 | 82.8 | 76.5 | 74.5 | 76.5 | 80.2 | 86.4 | 75.2 | 82 | 82.4 | 83.9 | 79 | 80.9 | 85.7 | 85.2 | 81.6 | 82.1 | 81.9 | 87.2 | 86.8 | 87 | 86.4 | 87.3 | 83.8 | 87 | 90.8 | |
| PARTHENONModel=GPT-5.52026.06 | 10.95 | 89.9 | 89.6 | 90.9 | 89.7 | 91.1 | 91.2 | 86.1 | 89.9 | 87.7 | 87 | 87.8 | 92.2 | 91.8 | 88.1 | 89.6 | 89.6 | 89.1 | 92.2 | 89.5 | 89.2 | 90.3 | 86.3 | 89.3 | 91.3 | 92.8 | |
| CodexModel=GPT-5.52026.06 | 3.76 | 79.8 | 69.8 | 72.3 | 74.6 | 75.4 | 76 | 77.8 | 78.3 | 78.7 | 78.9 | 79.5 | 79.5 | 80.6 | 80.7 | 80.8 | 81.4 | 81.5 | 81.6 | 82 | 82.8 | 83.2 | 83.3 | 83.7 | 84.6 | 88.6 | |
| PARTHENONModel=GPT-5.4-mini2026.06 | 3.36 | 82 | 82.3 | 84.8 | 82 | 82.9 | 81.1 | 77.7 | 80.8 | 77.7 | 79.8 | 81.7 | 82.9 | 84.1 | 78 | 79.7 | 84.3 | 81.9 | 84.8 | 81.1 | 81.8 | 85.4 | 78.8 | 80.5 | 83 | 85.6 | |
| PARTHENONModel=Haiku 4.52026.06 | 3.04 | 77.8 | 79.9 | 80 | 81.3 | 79.7 | 75.1 | 79.6 | 74.2 | 75.4 | 69.8 | 80.6 | 81.9 | 79.2 | 75.7 | 70.8 | 76.6 | 78.9 | 85.5 | 79.7 | 76.3 | 80.9 | 68.1 | 75 | 75.3 | 76.9 | |
| CodexModel=GPT-5.4-mini2026.06 | 1.12 | 68.2 | 56.7 | 65.1 | 65.2 | 66.2 | 68.9 | 63.1 | 68.8 | 64.4 | 68.7 | 67.4 | 66.2 | 68.9 | 69 | 70.2 | 64 | 71.4 | 69.3 | 70.7 | 68.3 | 72.6 | 73.5 | 72.8 | 73.3 | 64.2 | |
| Claude CodeModel=Haiku 4.52026.06 | 1.12 | 71.6 | 66.3 | 67.6 | 71.7 | 70.5 | 67.7 | 59.5 | 64.8 | 68.6 | 66.4 | 72.7 | 70.9 | 73.9 | 72.3 | 70.4 | 69.3 | 71.2 | 77.4 | 74.6 | 72.1 | 83.9 | 63.9 | 73.5 | 74.8 | 75.3 | |
| Direct Prompting (API)Model=GPT-5.4-mini2026.06 | 0.4 | 56.5 | 47.4 | 44.1 | 55.1 | 63.3 | 54.2 | 57.2 | 44.4 | 58 | 50.4 | 59.9 | 61.2 | 61.4 | 62.4 | 53.1 | 58 | 58.3 | 66.9 | 60.6 | 61 | 70 | 51.4 | 52.1 | 60.5 | 58.8 | |
| Basic Legal-Native HarnessModel=GPT-5.4-mini2026.06 | 0.16 | 60.7 | 61 | 67.9 | 62.8 | 62.8 | 62.9 | 54.2 | 64.2 | 59.8 | 48 | 54.7 | 60.8 | 64.3 | 56.2 | 59.4 | 59.6 | 53.6 | 65.4 | 54.8 | 52.2 | 63.1 | 54.1 | 63.2 | 56.3 | 63.3 |