Accuracy Improvement on OpenBookQA
7.2Accuracy Improvement DeltaPASf
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PASfModel=DeepSeek-R1-Distill-8B2025.09 | 7.2 | 6.4 | 0 | |
| Ultra-Dense PromptingModel=Gemini 2.02026.04 | 6 | — | — | |
| Ultra-Dense PromptingModel=Claude 3.72026.04 | 5.3 | — | — | |
| Ultra-Dense PromptingModel=Average across Models2026.04 | 5.1 | — | — | |
| iPASaModel=DeepSeek-R1-Distill-8B2025.09 | 5 | 4.2 | 0 | |
| Ultra-Dense PromptingModel=GPT-4o2026.04 | 4.9 | — | — | |
| Ultra-Dense PromptingModel=GPT-4o-mini2026.04 | 4.4 | — | — | |
| iPASwoModel=DeepSeek-R1-Distill-8B2025.09 | 3.7 | 3.1 | 0 | |
| iPASwoModel=Nous-Hermes-2-Mistral-7B-DPO2025.09 | 1.1 | 0.8 | 0 | |
| PASfModel=Nous-Hermes-2-Mistral-7B-DPO2025.09 | 0.7 | 0.4 | 0 | |
| iPASaModel=Nous-Hermes-2-Mistral-7B-DPO2025.09 | 0.3 | -0.3 | 0.13 | |
| iPASaModel=Llama-3.1-8B-Instruct2025.09 | 0 | 0 | 0.15 | |
| iPASwoModel=Llama-3.1-8B-Instruct2025.09 | 0 | 0 | 0.11 | |
| PASfModel=Llama-3.1-8B-Instruct2025.09 | 0 | 0 | 0 |