Logical Reasoning on ProofWriter (Accuracy and Toxic Rate)
98.4AccuracyPoT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PoTbackbone=GPT-120B2026.01 | 98.4 | — | |
| PoTbackbone=Sonnet-3.72026.01 | 98.2 | — | |
| PoTbackbone=GPT-20B2026.01 | 94 | — | |
| VERGE Fullbackbone=Sonnet-3.72026.01 | 93 | — | |
| VERGE w/o MCSbackbone=Sonnet-3.72026.01 | 90.2 | — | |
| VERGE Fullbackbone=GPT-120B2026.01 | 89.9 | — | |
| VERGE w/o MCSbackbone=GPT-120B2026.01 | 88.7 | — | |
| VERGE w/o Rtbackbone=Sonnet-3.72026.01 | 88 | — | |
| SCbackbone=Sonnet-3.72026.01 | 86.6 | — | |
| VERGE w/o Rtbackbone=GPT-20B2026.01 | 85.8 | — | |
| VERGE Fullbackbone=GPT-20B2026.01 | 85.2 | — | |
| VERGE w/o Rtbackbone=GPT-120B2026.01 | 84.6 | — | |
| VERGE w/o MCSbackbone=GPT-20B2026.01 | 82.7 | — | |
| CoTBackbone=Llama-3.3-70B, Evaluation Protocol=String Match (SM)2026.03 | 79.7 | — | |
| RIDERSModel=Llama2-13B2024.02 | 78.2 | 6.4 | |
| LINCbackbone=GPT-120B2026.01 | 76.4 | — | |
| CoTBackbone=Qwen2.5-72B, Evaluation Protocol=String Match (SM)2026.03 | 76.3 | — | |
| CoTbackbone=GPT-20B2026.01 | 74.6 | — | |
| SRbackbone=Sonnet-3.72026.01 | 74 | — | |
| CoTbackbone=Sonnet-3.72026.01 | 71.6 | — | |
| SCbackbone=GPT-20B2026.01 | 71.4 | — | |
| LINCbackbone=Sonnet-3.72026.01 | 71.1 | — | |
| ToTBackbone=Qwen2.5-72B, Evaluation Protocol=String Match (SM)2026.03 | 70.7 | — | |
| SRbackbone=GPT-120B2026.01 | 67.2 | — | |
| SCbackbone=GPT-120B2026.01 | 65.6 | — | |
| NoTBackbone=Qwen2.5-72B, Evaluation Protocol=String Match (SM)2026.03 | 65 | — | |
| NoTBackbone=Qwen2.5-72B, Evaluation Protocol=LLM-as-Judge (J)2026.03 | 65 | — | |
| LogicLMbackbone=Sonnet-3.72026.01 | 64.7 | — | |
| ToTBackbone=Llama-3.3-70B, Evaluation Protocol=String Match (SM)2026.03 | 61.7 | — | |
| NoTBackbone=Llama-3.3-70B, Evaluation Protocol=String Match (SM)2026.03 | 57.7 | — | |
| NoTBackbone=Llama-3.3-70B, Evaluation Protocol=LLM-as-Judge (J)2026.03 | 57.7 | — | |
| CoTModel=Llama2-13B2024.02 | 56.8 | 19.9 | |
| SRbackbone=GPT-20B2026.01 | 56.8 | — | |
| CoTbackbone=GPT-120B2026.01 | 52.4 | — | |
| ToTBackbone=GPT-4o-mini, Evaluation Protocol=String Match (SM)2026.03 | 51.7 | — | |
| NoTBackbone=GPT-4o-mini, Evaluation Protocol=LLM-as-Judge (J)2026.03 | 50.3 | — | |
| NoTBackbone=GPT-4o-mini, Evaluation Protocol=String Match (SM)2026.03 | 49 | — | |
| CoTBackbone=GPT-4o-mini, Evaluation Protocol=String Match (SM)2026.03 | 43.3 | — | |
| LINCbackbone=GPT-20B2026.01 | 42.8 | — | |
| DSBbackbone=Sonnet-3.72026.01 | 42.8 | — | |
| DSBbackbone=GPT-20B2026.01 | 38.8 | — | |
| LogicLMbackbone=GPT-20B2026.01 | 35.8 | — | |
| LogicLMbackbone=GPT-120B2026.01 | 32 | — | |
| DSBbackbone=GPT-120B2026.01 | 31.4 | — |