Question Answering on MultiArith
74.3AccuracyGCoT-decoding
Evaluation Results
| Method | Links | |
|---|---|---|
| GCoT-decodingModel=Llama-3.1-8B, Spec Ans=No2026.04 | 74.3 | |
| CoT-decodingModel=Llama-3.1-8B, Spec Ans=Yes2026.04 | 72.3 | |
| GCoT-decoding + SpanAlignModel=Llama-3.1-8B, Spec Ans=No2026.04 | 69.3 | |
| Self-consistencyModel=Llama-3.1-8B, Spec Ans=Yes2026.04 | 46.9 | |
| CoT-decodingModel=Gemma-7B, Spec Ans=Yes2026.04 | 43.8 | |
| CoT-decodingModel=Mistral-7B, Spec Ans=Yes2026.04 | 40.6 | |
| GreedyModel=Llama-3.1-8B, Spec Ans=No2026.04 | 38.8 | |
| Beam searchModel=Llama-3.1-8B, Spec Ans=No2026.04 | 38.1 | |
| Top-k samplingModel=Llama-3.1-8B, Spec Ans=No2026.04 | 37 | |
| Temperature samplingModel=Llama-3.1-8B, Spec Ans=No2026.04 | 36.2 | |
| GCoT-decodingModel=Mistral-7B, Spec Ans=No2026.04 | 31.3 | |
| Self-consistencyModel=Gemma-7B, Spec Ans=Yes2026.04 | 22.9 | |
| GCoT-decodingModel=Gemma-7B, Spec Ans=No2026.04 | 22.8 | |
| Self-consistencyModel=Mistral-7B, Spec Ans=Yes2026.04 | 21.7 | |
| GCoT-decoding + SpanAlignModel=Gemma-7B, Spec Ans=No2026.04 | 19.7 | |
| Temperature samplingModel=Gemma-7B, Spec Ans=No2026.04 | 18.8 | |
| GreedyModel=Gemma-7B, Spec Ans=No2026.04 | 18.7 | |
| Beam searchModel=Gemma-7B, Spec Ans=No2026.04 | 17.9 | |
| Top-k samplingModel=Gemma-7B, Spec Ans=No2026.04 | 17.3 | |
| GCoT-decoding + SpanAlignModel=Mistral-7B, Spec Ans=No2026.04 | 16.8 | |
| GreedyModel=Mistral-7B, Spec Ans=No2026.04 | 16 | |
| Beam searchModel=Mistral-7B, Spec Ans=No2026.04 | 15.5 | |
| Temperature samplingModel=Mistral-7B, Spec Ans=No2026.04 | 15.2 | |
| Top-k samplingModel=Mistral-7B, Spec Ans=No2026.04 | 13.3 |