Question Answering on BoolQ (Delta Metrics)
2.16Delta AccuracyAUTOMIXER-350M
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AUTOMIXER-350MNumber of Parameters=350M2025.06 | 2.16 | — | |
| AUTOMIXER-350MNumber of Parameters=1.5B2025.06 | 1.35 | — | |
| AUTOMIXER-350MNumber of Parameters=3B2025.06 | 1.12 | — | |
| N-GRAM SAMPLINGNumber of Parameters=1.5B2025.06 | 1.02 | — | |
| N-GRAM SAMPLINGNumber of Parameters=350M2025.06 | 0.79 | — | |
| N-GRAM SAMPLINGNumber of Parameters=3B2025.06 | 0.57 | — | |
| PPL SAMPLINGNumber of Parameters=350M2025.06 | 0.44 | — | |
| PPL SAMPLINGNumber of Parameters=1.5B2025.06 | 0.32 | — | |
| PPL SAMPLINGNumber of Parameters=3B2025.06 | 0.1 | — | |
| AutoPromptTarget Model=Qwen2-1.5B, Suffix length (K)=42025.12 | -0.01 | 0.8 | |
| AUTOMIXER-75MNumber of Parameters=3B2025.06 | -0.03 | — | |
| AUTOMIXER-75MNumber of Parameters=1.5B2025.06 | -0.04 | — | |
| Universal TriggerTarget Model=Qwen2-1.5B, Suffix length (K)=42025.12 | -0.08 | 0.6 | |
| Proposed methodTarget Model=Qwen2-1.5B, Suffix length (K)=42025.12 | -0.11 | 0.23 | |
| AUTOMIXER-75MNumber of Parameters=350M2025.06 | -0.14 | — |