Logical Reasoning on SLR-BENCH
15.5Overall LRL Scoreo3
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| o3Total Tokens (M)=4.30, Total Costs ($)=207.242025.06 | 15.5 | 80 | 99 | 93 | 74 | 45 | |
| gpt-5Total Tokens (M)=16.40, Total Costs ($)=103.132025.06 | 15.4 | 98 | 100 | 90 | 72 | 46 | |
| gpt-5-mini-highTotal Tokens (M)=13.12, Total Costs ($)=27.982025.06 | 14.2 | 0.94 | 99 | 83 | 63 | 38 | |
| gpt-5-miniTotal Tokens (M)=4.90, Total Costs ($)=11.542025.06 | 12 | 95 | 99 | 82 | 41 | 20 | |
| o1Total Tokens (M)=5.19, Total Costs ($)=364.722025.06 | 11.9 | 68 | 92 | 89 | 41 | 15 | |
| gpt-5-mini-lowTotal Tokens (M)=1.17, Total Costs ($)=4.072025.06 | 9.7 | 0.88 | 97 | 71 | 20 | 7 | |
| R1-Llama-70BTotal Tokens (M)=11.61, Total Costs ($)=5.332025.06 | 8.8 | 75 | 98 | 67 | 8 | 4 | |
| Gemini-thinking2025.06 | 8.6 | 83 | 93 | 65 | 13 | 1 | |
| gpt-5-nanoTotal Tokens (M)=6.17, Total Costs ($)=2.812025.06 | 8.5 | 99 | 97 | 61 | 10 | 2 | |
| gpt-4.5-prevTotal Tokens (M)=0.37, Total Costs ($)=576.402025.06 | 7.3 | 100 | 94 | 47 | 5 | 1 | |
| gpt-4oTotal Tokens (M)=0.26, Total Costs ($)=20.032025.06 | 6.2 | 100 | 93 | 29 | 2 | 0 | |
| Llama 3.3-70BTotal Tokens (M)=0.49, Total Costs ($)=0.822025.06 | 5.6 | 100 | 90 | 22 | 1 | 0 | |
| gpt-4-turboTotal Tokens (M)=0.41, Total Costs ($)=81.302025.06 | 5.4 | 100 | 89 | 18 | 2 | 0 | |
| Llama 3.1-8BTotal Tokens (M)=0.20, Total Costs ($)=0.142025.06 | 3.8 | 80 | 70 | 7 | 0 | 0 |