Social Interaction Question Answering on SIQA (Normalized PLL score)
50.4Normalized PLL ScoreDecoupled DiLoCo
Evaluation results
| Method | Links | |
|---|---|---|
| Decoupled DiLoCoModel Scale=Gemma 9B, Method=DiLoCo2026.04 | 50.4 | |
| DPModel Scale=Gemma 9B, Method=DP2026.04 | 49.8 | |
| DPModel Scale=Gemma 5B, Method=DP2026.04 | 48.9 | |
| Decoupled DiLoCoModel Scale=Gemma 5B, Method=DiLoCo2026.04 | 46.9 | |
| Decoupled DiLoCoModel Scale=Gemma 2B, Method=DiLoCo2026.04 | 46.5 | |
| DPModel Scale=Gemma 2B, Method=DP2026.04 | 46 | |
| DualRepetitions=32, Alpha (α)=3/42025.12 | 15.4 | |
| DualRepetitions=128, Alpha (α)=1/82025.12 | 15.2 | |
| Masked-DiffusionRepetitions=32, Alpha (α)=02025.12 | 14.4 | |
| Masked-DiffusionRepetitions=128, Alpha (α)=0:12025.12 | 12 |