Multi-hop QA on FRAMES Wiki18 index (test)
37.5Pass@4Gemma3-12B
Evaluation results
| Method | Links | |
|---|---|---|
| Gemma3-12BSampling Method=DivInit (DI)2026.06 | 37.5 | |
| Gemma3-12BSampling Method=Standard parallel sampling (S)2026.06 | 31 | |
| Qwen3-8BSampling Method=DivInit (DI)2026.06 | 30.8 | |
| Qwen3-8BSampling Method=Standard parallel sampling (S)2026.06 | 24.8 | |
| Qwen3-4BSampling Method=DivInit (DI)2026.06 | 20.4 | |
| Qwen3-4BSampling Method=Standard parallel sampling (S)2026.06 | 15.5 | |
| Gemma3-4BSampling Method=DivInit (DI)2026.06 | 14.7 | |
| Qwen3-1.7BSampling Method=DivInit (DI)2026.06 | 13.6 | |
| Qwen3-1.7BSampling Method=Standard parallel sampling (S)2026.06 | 13.1 | |
| Gemma3-4BSampling Method=Standard parallel sampling (S)2026.06 | 12.3 |