Logical Reasoning on FOLIO
89.2AccuracyVERGE Full
Evaluation results
| Method | Links | |
|---|---|---|
| VERGE Fullbackbone=GPT-20B2026.01 | 89.2 | |
| PACS Llama 3.3 70BMethod=PACS, Model=Llama 3.3, Parameters=70B, Model Type=non-thinking model2026.05 | 88 | |
| VERGE Fullbackbone=Sonnet-3.72026.01 | 87.9 | |
| VERGE w/o Rtbackbone=GPT-20B2026.01 | 86.7 | |
| VERGE w/o MCSbackbone=Sonnet-3.72026.01 | 86.7 | |
| VERGE Fullbackbone=GPT-120B2026.01 | 84.7 | |
| QwQ-32B-Preview*Model Series=QwQ, Size=32B, Source=Shen et al. (2025)2025.02 | 84.2 | |
| VERGE w/o Rtbackbone=Sonnet-3.72026.01 | 83 | |
| SC OSS-GPT-120BMethod=SC, Model=OSS-GPT, Parameters=120B, Model Type=Thinking model2026.05 | 82 | |
| PACS Llama 3 8BMethod=PACS, Model=Llama 3, Parameters=8B, Model Type=non-thinking model2026.05 | 82 | |
| VERGE w/o Rtbackbone=GPT-120B2026.01 | 81.6 | |
| VERGE w/o MCSbackbone=GPT-120B2026.01 | 80.7 | |
| VERGE w/o MCSbackbone=GPT-20B2026.01 | 73.9 | |
| SCbackbone=Sonnet-3.72026.01 | 72.4 | |
| LogicLMbackbone=Sonnet-3.72026.01 | 71.6 | |
| CoTbackbone=Sonnet-3.72026.01 | 70.4 | |
| Qwen2.5-Math-72B-InstructModel Series=Qwen2.5-Math, Size=72B2025.02 | 69.5 | |
| OpenMath2-Llama3.1-70B*Model Series=OpenMath2, Base=Llama-3.1-70B, Source=Shen et al. (2025)2025.02 | 68.5 | |
| LINCbackbone=Sonnet-3.72026.01 | 65.2 | |
| Llama-3.1-70B-Instruct*Model Series=Llama-3.1, Size=70B, Source=Shen et al. (2025)2025.02 | 65 | |
| SRbackbone=Sonnet-3.72026.01 | 62.1 | |
| Qwen2.5-Math-7B-InstructModel Series=Qwen2.5-Math, Size=7B2025.02 | 61.6 | |
| Qwen2.5-Math-7B-S2R-ORLModel Series=Qwen2.5-Math, Size=7B, RL Type=Outcome-level RL (ORL)2025.02 | 61.6 | |
| CPMIReward Type=CPMI, Zero-shot BoN@8=true2026.04 | 59.61 | |
| CPMI_MergeReward Type=CPMI_Merge, Zero-shot BoN@8=true2026.04 | 58.62 | |
| Qwen2.5-Math-7B-S2R-BIModel Series=Qwen2.5-Math, Size=7B, RL Type=Binary Reward (BI)2025.02 | 58.1 | |
| PoTbackbone=Sonnet-3.72026.01 | 58.1 | |
| Eurus-2-7B-PRIMEModel Series=Eurus-2, Size=7B2025.02 | 56.7 | |
| PoTbackbone=GPT-120B2026.01 | 54.2 | |
| MCReward Type=MC, Zero-shot BoN@8=true2026.04 | 54.19 | |
| DSBbackbone=GPT-20B2026.01 | 53.7 | |
| SCbackbone=GPT-20B2026.01 | 52.2 | |
| Full2025.12 | 51 | |
| SnapKVBackbone=DeepSeek-R1-Distill-Llama-8B, Cache Budget=3842025.12 | 49 | |
| PoTbackbone=GPT-20B2026.01 | 48.8 | |
| Llama2prompting_strategy=CoT w. logic., fine-tuned_on_LLM-ERL=true2023.10 | 48 | |
| LINCbackbone=GPT-120B2026.01 | 47.5 | |
| FullBackbone=DeepSeek-R1-Distill-Llama-8B2025.12 | 47 | |
| Llama2prompting_strategy=vanilla CoT, fine-tuned_on_LLM-ERL=true2023.10 | 46 | |
| Llama2prompting_strategy=CoT w. logic., fine-tuned_on_LLM-ERL=false2023.10 | 46 | |
| SnapKVKV cache budget=5122025.12 | 46 | |
| Vicunaprompting_strategy=CoT w. logic., fine-tuned_on_LLM-ERL=true2023.10 | 45 | |
| Llama2prompting_strategy=vanilla ICL, fine-tuned_on_LLM-ERL=true2023.10 | 45 | |
| SnapKVKV cache budget=3842025.12 | 45 | |
| DSBbackbone=Sonnet-3.72026.01 | 44.5 | |
| Llama2prompting_strategy=vanilla CoT, fine-tuned_on_LLM-ERL=false2023.10 | 44 | |
| RKVKV cache budget=3842025.12 | 44 | |
| SnapKVKV cache budget=1282025.12 | 44 | |
| Vicunaprompting_strategy=vanilla ICL, fine-tuned_on_LLM-ERL=true2023.10 | 43 | |
| H2OKV cache budget=2562025.12 | 43 | |
| H2OKV cache budget=5122025.12 | 43 | |
| SRbackbone=GPT-120B2026.01 | 42.9 | |
| Rand_MergeReward Type=Rand_Merge, Zero-shot BoN@8=true2026.04 | 42.86 | |
| Vicunaprompting_strategy=CoT w. logic., fine-tuned_on_LLM-ERL=false2023.10 | 42 | |
| Llama2prompting_strategy=vanilla ICL, fine-tuned_on_LLM-ERL=false2023.10 | 42 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 42 | |
| H2OKV cache budget=3842025.12 | 41 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 41 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 41 | |
| CoTbackbone=GPT-20B2026.01 | 40.4 | |
| Vicunaprompting_strategy=vanilla ICL, fine-tuned_on_LLM-ERL=false2023.10 | 40 | |
| Vicunaprompting_strategy=vanilla CoT, fine-tuned_on_LLM-ERL=true2023.10 | 40 | |
| SnapKVKV cache budget=2562025.12 | 40 | |
| KnormKV cache budget=3842025.12 | 39 | |
| Vicunaprompting_strategy=vanilla CoT, fine-tuned_on_LLM-ERL=false2023.10 | 38 | |
| KnormKV cache budget=5122025.12 | 38 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 38 | |
| Qwen2.5-Math-7BModel Series=Qwen2.5-Math, Size=7B2025.02 | 37.9 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 37 | |
| FullBackbone=Deepseek-R1-Distill-Qwen-7B2025.12 | 36 | |
| RKVKV cache budget=2562025.12 | 36 | |
| FullBackbone=Nemotron-Nano-8B2025.12 | 36 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 36 | |
| SCbackbone=GPT-120B2026.01 | 35.5 | |
| StreamingLLMKV cache budget=5122025.12 | 35 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 35 | |
| SRbackbone=GPT-20B2026.01 | 34 | |
| RKVKV cache budget=5122025.12 | 34 | |
| ShadowKVBackbone=Deepseek-R1-Distill-Qwen-7B2025.12 | 33 | |
| ShadowKV2025.12 | 33 | |
| ShadowKVBackbone=Nemotron-Nano-8B2025.12 | 33 | |
| CoTbackbone=GPT-120B2026.01 | 32 | |
| SnapKVBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=3842025.12 | 31 | |
| SnapKVBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=1282025.12 | 30 | |
| LogicLMbackbone=GPT-20B2026.01 | 29.9 | |
| SnapKVBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=5122025.12 | 29 | |
| KnormKV cache budget=2562025.12 | 28 | |
| LogicLMbackbone=GPT-120B2026.01 | 27.5 | |
| SnapKVBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=2562025.12 | 25 | |
| StreamingLLMKV cache budget=3842025.12 | 25 | |
| H2OBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=3842025.12 | 23 | |
| H2OBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=5122025.12 | 23 | |
| H2OKV cache budget=1282025.12 | 22 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 22 | |
| H2OBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=2562025.12 | 21 | |
| LINCbackbone=GPT-20B2026.01 | 18.6 | |
| StreamingLLMBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 15 | |
| DSBbackbone=GPT-120B2026.01 | 14 | |
| RKVBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 14 | |
| KnormBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 13 |