Question Answering on Pacific (dev)
65.24RewardSGP-Oracle
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SGP-Oraclebase_model=Gemma-2-9b, averaging=averaged across rollouts2025.12 | 65.24 | 73.89 | 43.46 | 14.05 | |
| SGPbase_model=Gemma-2-9b, averaging=averaged across rollouts2025.12 | 54.32 | 65.73 | 42.86 | 14.6 | |
| Clarifybase_model=Gemma-2-9b, averaging=averaged across rollouts2025.12 | 45.48 | 62.7 | 100 | 0 | |
| Answerbase_model=Gemma-2-9b, averaging=averaged across rollouts2025.12 | 42.44 | 52.25 | 0 | 0 | |
| Prompted-COTbase_model=Gemma-2-9b, averaging=averaged across rollouts2025.12 | 33 | 50.04 | 16.45 | 16.17 | |
| Promptedbase_model=Gemma-2-9b, averaging=averaged across rollouts2025.12 | 11.46 | 55.28 | 17.99 | 47.18 | |
| Clarify-and-MAbase_model=Gemma-2-9b, averaging=averaged across rollouts2025.12 | -16.18 | 56.66 | 100 | 100 | |
| Multi-answer (MA)base_model=Gemma-2-9b, averaging=averaged across rollouts2025.12 | -21.39 | 54.63 | 0 | 100 |