Question Answering on MMLU (Subject-Specific Accuracy)
89.2MMLU AccuracyANN
Evaluation results
| Method | Links | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ANNTraining Phase Model=GPT-4o-mini, Validation Phase Model=GPT-4o2025.06 | 89.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TextGrad2025.06 | 88.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoT2025.06 | 85.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gating InversionMitigation Strategy=Realignment Methods, Training Free=true2026.06 | 71.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| re-SFTMitigation Strategy=Realignment Methods, Training Free=false2026.06 | 71.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Misaligned LoRAMitigation Strategy=w/o Mitigation2026.06 | 66.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Activation SteerMitigation Strategy=Realignment Methods, Training Free=true2026.06 | 61.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BLADETrain Toks.=3B, Backbone=TinyLlama-1.1B2026.06 | 27.26 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RHO-1Train Toks.=3B, Backbone=TinyLlama-1.1B2026.06 | 25.49 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Base-CTTrain Toks.=5B, Backbone=TinyLlama-1.1B2026.06 | 25.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RandomTrain Toks.=3B, Backbone=TinyLlama-1.1B2026.06 | 25.35 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseTrain Toks.=-, Backbone=TinyLlama-1.1B2026.06 | 25.31 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MATES*Train Toks.=3B, Backbone=TinyLlama-1.1B2026.06 | 24.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BLOOMShots=5, Domain=out-of-domain2022.11 | — | 25 | 25.7 | 28.5 | 29.4 | 19 | 23.2 | 6 | 25 | 23.7 | 32.4 | 27.6 | 26.2 | 25 | 25 | 27 | 36 | 18.6 | 25.2 | 19.4 | — | |
| ChinchillaShots=5, Domain=out-of-domain2022.11 | — | 31 | 73 | 79.9 | 80.3 | 51 | 58.1 | 51 | 58 | 38.6 | 62.1 | 41.5 | 33.3 | 41.1 | 32 | 31.9 | 69 | 46.1 | 36.4 | 58.8 | — | |
| GalacticaModel Size=120B, Shots=0, Domain=out-of-domain2022.11 | — | 27 | 65.1 | 68.8 | 69.4 | 46 | 47.8 | 49 | 70 | 42.1 | 62.8 | 38.1 | 32.5 | 38.4 | 43 | 32.6 | 68 | 42.2 | 33.8 | 41.2 | — | |
| GalacticaModel Size=30B, Shots=0, Domain=out-of-domain2022.11 | — | 33.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 70 | — | — | — | — | |
| GoABackbone=gpt-oss-120b, N=5, k=2, LLM calls=11, Variant=GoAMean (3-model pool)2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 89.3 | |
| GopherModel Size=280B, Shots=5, Domain=out-of-domain2022.11 | — | 25 | 65.8 | 70.8 | 71.3 | 45 | 47.8 | 49 | 54 | 43 | 60 | 33.6 | 35.7 | 41.1 | 37 | 23.7 | 69 | 34.3 | 33.8 | 50 | — | |
| MoABackbone=gpt-oss-120b, N=5, k=2, LLM calls=11, Proposers=4 heterogeneous proposers2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 85.7 | |
| MoA (para)Backbone=gpt-oss-120b, N=5, k=2, LLM calls=11, Proposers=SPUQ paraphrases with a single model2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 86.6 | |
| OPTShots=5, Domain=out-of-domain2022.11 | — | 21 | 23 | 30.6 | 27.7 | 30 | 21.7 | 17 | 30 | 21 | 36.6 | 25.7 | 29.4 | 28.6 | 33 | 24.4 | 35 | 21.6 | 29.8 | 43.5 | — | |
| SCBackbone=gpt-oss-120b, N=5, k=2, LLM calls=112026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 90.2 | |
| SC-MoABackbone=gpt-oss-120b, N=5, k=2, LLM calls=112026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 92 | |
| Self-MoABackbone=gpt-oss-120b, N=5, k=2, LLM calls=112026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 85.7 | |
| TextGradBackbone=gpt-oss-120b, N=5, k=2, LLM calls=112026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 90.2 | |
| Zero-shot CoTBackbone=gpt-oss-120b, N=5, k=2, LLM calls=112026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.9 |