List your company
ResearchFormerly WizWand

Question Answering on MultiArith

74.3AccuracyGCoT-decoding
10.8627.3343.860.27Apr 8, 2026
Updated 6mo ago

Evaluation results

MethodLinks
GCoT-decodingModel=Llama-3.1-8B, Spec Ans=No2026.0474.3
CoT-decodingModel=Llama-3.1-8B, Spec Ans=Yes2026.0472.3
GCoT-decoding + SpanAlignModel=Llama-3.1-8B, Spec Ans=No2026.0469.3
Self-consistencyModel=Llama-3.1-8B, Spec Ans=Yes2026.0446.9
CoT-decodingModel=Gemma-7B, Spec Ans=Yes2026.0443.8
CoT-decodingModel=Mistral-7B, Spec Ans=Yes2026.0440.6
GreedyModel=Llama-3.1-8B, Spec Ans=No2026.0438.8
Beam searchModel=Llama-3.1-8B, Spec Ans=No2026.0438.1
Top-k samplingModel=Llama-3.1-8B, Spec Ans=No2026.0437
Temperature samplingModel=Llama-3.1-8B, Spec Ans=No2026.0436.2
GCoT-decodingModel=Mistral-7B, Spec Ans=No2026.0431.3
Self-consistencyModel=Gemma-7B, Spec Ans=Yes2026.0422.9
GCoT-decodingModel=Gemma-7B, Spec Ans=No2026.0422.8
Self-consistencyModel=Mistral-7B, Spec Ans=Yes2026.0421.7
GCoT-decoding + SpanAlignModel=Gemma-7B, Spec Ans=No2026.0419.7
Temperature samplingModel=Gemma-7B, Spec Ans=No2026.0418.8
GreedyModel=Gemma-7B, Spec Ans=No2026.0418.7
Beam searchModel=Gemma-7B, Spec Ans=No2026.0417.9
Top-k samplingModel=Gemma-7B, Spec Ans=No2026.0417.3
GCoT-decoding + SpanAlignModel=Mistral-7B, Spec Ans=No2026.0416.8
GreedyModel=Mistral-7B, Spec Ans=No2026.0416
Beam searchModel=Mistral-7B, Spec Ans=No2026.0415.5
Temperature samplingModel=Mistral-7B, Spec Ans=No2026.0415.2
Top-k samplingModel=Mistral-7B, Spec Ans=No2026.0413.3