Question Answering on MuSiQue (held-out)
57.7F1 ScoreIRCOT*
Evaluation results
| Method | Links | ||
|---|---|---|---|
| IRCOT*Reader=Gemma-3-31B, N=3002026.06 | 57.7 | 3,800 | |
| a-PRUNEReader=GPT-OSS-120B, N=5002026.06 | 52.8 | 4,800 | |
| RASER-3Reader=GPT-OSS-120B, N=5002026.06 | 51 | 2,200 | |
| a-PRUNEReader=Gemma-3-31B, N=3002026.06 | 50.3 | 3,700 | |
| RASER-2Reader=GPT-OSS-120B, N=5002026.06 | 50.2 | 2,100 | |
| RASER-3Reader=Gemma-3-31B, N=3002026.06 | 50.2 | 2,400 | |
| IRCOT*Reader=Mistral-S-119B, N=3002026.06 | 49.9 | 4,300 | |
| ONE-SHOT RAGReader=GPT-OSS-120B, N=5002026.06 | 48.8 | 1,500 | |
| SELF-ASK*Reader=GPT-OSS-120B, N=5002026.06 | 48.2 | 4,600 | |
| ChainRAG†Reader=GPT-OSS-120B, N=5002026.06 | 46.5 | 4,600 | |
| ChainRAG†Reader=Mistral-S-119B, N=3002026.06 | 46.4 | 2,300 | |
| RASER-2Reader=Gemma-3-31B, N=3002026.06 | 45.8 | 2,100 | |
| IRCOT*Reader=GPT-OSS-120B, N=5002026.06 | 45.5 | 4,300 | |
| ChainRAG†Reader=Gemma-3-31B, N=3002026.06 | 44.5 | 2,300 | |
| RASER-3Reader=Mistral-S-119B, N=3002026.06 | 40.8 | 2,400 | |
| SELF-ASK*Reader=Phi-4-mini, N=3002026.06 | 40.7 | 3,000 | |
| a-PRUNEReader=Mistral-S-119B, N=3002026.06 | 39.7 | 3,800 | |
| ONE-SHOT RAGReader=Gemma-3-31B, N=3002026.06 | 39.3 | 1,200 | |
| SELF-ASK*Reader=Mistral-S-119B, N=3002026.06 | 38 | 3,300 | |
| IRCOT*Reader=Llama-3.1-8B, N=3002026.06 | 37.6 | 5,100 | |
| IRCOT*Reader=Phi-4-mini, N=3002026.06 | 37.2 | 5,400 | |
| SELF-ASK*Reader=Gemma-3-31B, N=3002026.06 | 36 | 3,000 | |
| RASER-3Reader=Phi-4-mini, N=3002026.06 | 35.7 | 2,700 | |
| IRCOT*Reader=Llama-3-8B, N=3002026.06 | 35.4 | 4,700 | |
| a-PRUNEReader=Phi-4-mini, N=3002026.06 | 35.3 | 3,600 | |
| RASER-2Reader=Phi-4-mini, N=3002026.06 | 33.5 | 1,400 | |
| ONE-SHOT RAGReader=Phi-4-mini, N=3002026.06 | 33.1 | 1,100 | |
| RASER-3Reader=Llama-3.1-8B, N=3002026.06 | 33 | 3,000 | |
| ChainRAG†Reader=Llama-3.1-8B, N=3002026.06 | 31.2 | 2,600 | |
| RASER-3Reader=Llama-3-8B, N=3002026.06 | 30.9 | 2,800 | |
| RASER-2Reader=Mistral-S-119B, N=3002026.06 | 30.6 | 1,900 | |
| ChainRAG†Reader=Phi-4-mini, N=3002026.06 | 29.3 | 1,900 | |
| SELF-ASK*Reader=Llama-3-8B, N=3002026.06 | 29.2 | 3,200 | |
| ONE-SHOT RAGReader=Mistral-S-119B, N=3002026.06 | 28.3 | 1,200 | |
| SELF-ASK*Reader=Llama-3.1-8B, N=3002026.06 | 27.6 | 3,200 | |
| a-PRUNEReader=Llama-3.1-8B, N=3002026.06 | 26.4 | 3,800 | |
| a-PRUNEReader=Llama-3-8B, N=3002026.06 | 25.6 | 3,700 | |
| RASER-2Reader=Llama-3.1-8B, N=3002026.06 | 25.3 | 1,500 | |
| ONE-SHOT RAGReader=Llama-3.1-8B, N=3002026.06 | 23.6 | 1,200 | |
| RASER-2Reader=Llama-3-8B, N=3002026.06 | 22.9 | 1,600 | |
| ONE-SHOT RAGReader=Llama-3-8B, N=3002026.06 | 22.3 | 1,200 | |
| ChainRAG†Reader=Llama-3-8B, N=3002026.06 | 20.9 | 2,800 |