Knowledge Evaluation on Natural Questions (NQ) (Evaluation)
83AccuracyGRADEpre
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRADEpreBackbone=Llama3.1-8b2026.04 | 83 | 0.888 | |
| GRADEpreBackbone=Gemma2-9b2026.04 | 81.4 | 0.907 | |
| GRADEposBackbone=Llama3.1-8b2026.04 | 79.4 | 0.841 | |
| GRADEpreBackbone=Qwen2.5-7b2026.04 | 78.1 | 0.814 | |
| Align-PBackbone=Llama3.1-8b2026.04 | 77.1 | 0.793 | |
| GRADEposBackbone=Gemma2-9b2026.04 | 75.9 | 0.823 | |
| GRADEposBackbone=Qwen2.5-7b2026.04 | 74.5 | 0.802 | |
| Align-PBackbone=Gemma2-9b2026.04 | 73.4 | 0.73 | |
| Align-PBackbone=Qwen2.5-7b2026.04 | 68 | 0.704 | |
| JudgeBackbone=Qwen2.5-7b2026.04 | 67.1 | — | |
| ICBackbone=Llama3.1-8b2026.04 | 64 | 0.669 | |
| ICBackbone=Qwen2.5-7b2026.04 | 62.5 | 0.67 | |
| P-EntropyBackbone=Llama3.1-8b2026.04 | 61.6 | 0.643 | |
| P-EntropyBackbone=Qwen2.5-7b2026.04 | 59.6 | 0.616 | |
| GPT-4oEvaluation Protocol=Closed-Source2026.01 | 59.4 | — | |
| GPT-5Evaluation Protocol=Closed-Source2026.01 | 59.3 | — | |
| Gemini2.5-ProEvaluation Protocol=Closed-Source2026.01 | 59.2 | — | |
| P-EntropyBackbone=Gemma2-9b2026.04 | 58.8 | 0.607 | |
| ICBackbone=Gemma2-9b2026.04 | 58.8 | 0.65 | |
| JudgeBackbone=Gemma2-9b2026.04 | 57.4 | — | |
| GPT-4.1Evaluation Protocol=Closed-Source2026.01 | 54.5 | — | |
| JudgeBackbone=Llama3.1-8b2026.04 | 52.3 | — | |
| CoT2-MetaStrategy=Ours (CoT2-Meta), Inference Budget=C=162026.03 | 45.4 | — | |
| ATLAS (RL)Evaluation Protocol=Out-of-Distribution2026.01 | 44.1 | — | |
| ATLAS (cluster)Evaluation Protocol=In-Distribution2026.01 | 43.8 | — | |
| ATLAS (cluster)Evaluation Protocol=Out-of-Distribution2026.01 | 43.8 | — | |
| ReST-MCTS*Strategy=ReST-MCTS*, Inference Budget=C=162026.03 | 43.2 | — | |
| Vanilla ToTStrategy=Vanilla ToT, Inference Budget=C=162026.03 | 41.5 | — | |
| RouterDCEvaluation Protocol=In-Distribution2026.01 | 41.2 | — | |
| RouterDCEvaluation Protocol=Out-of-Distribution2026.01 | 40.1 | — | |
| Best-of-16Strategy=Best-of-16, Inference Budget=C=162026.03 | 39.8 | — | |
| BertRouterEvaluation Protocol=In-Distribution2026.01 | 38.9 | — | |
| BertRouterEvaluation Protocol=Out-of-Distribution2026.01 | 38.9 | — | |
| MLPRouterEvaluation Protocol=Out-of-Distribution2026.01 | 37.3 | — | |
| Greedy CoTStrategy=Greedy CoT, Inference Budget=C=162026.03 | 36.5 | — | |
| MLPRouterEvaluation Protocol=In-Distribution2026.01 | 32.1 | — | |
| ZS RouterEvaluation Protocol=Training-free, Prompting Strategy=Zero-shot2026.01 | 29.2 | — | |
| FS RouterEvaluation Protocol=Training-free, Prompting Strategy=Few-shot2026.01 | 27.3 | — | |
| Random RouterEvaluation Protocol=Training-free2026.01 | 25.3 | — | |
| GAME-LoRAProtocol=Training-time2026.01 | 6.7 | — | |
| ActDecProtocol=Inference-time2026.01 | 6.7 | — | |
| BaselineArchitecture=LoRA, Loss=Vanilla CE, Protocol=Training-time2026.01 | 6.6 | — | |
| CADProtocol=Inference-time2026.01 | 6.6 | — | |
| DisagreementProtocol=Training-time2026.01 | 6.3 | — | |
| MEProtocol=Training-time2026.01 | 5.7 | — |