Multiple-choice Question Answering on MMLU, ARC, CSQA, OBQA, PIQA, SIQA, Winogrande (zero-shot)
30.81MMLU Accuracy (zero-shot)CMA (EMA + A-T)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CMA (EMA + A-T)WA=EMA, Order=A-T, LRS=Const2025.11 | 30.81 | 36.29 | 57.89 | 50.29 | 43.82 | 44.5 | 70.62 | 44.68 | 54.46 | 48.69 | |
| CMA (SMA + A-T)WA=SMA, Order=A-T, LRS=Const2025.11 | 30.65 | 36.79 | 57.37 | 50.78 | 43.9 | 43.6 | 70.89 | 44.73 | 54.74 | 48.69 | |
| CMA (EMA + A,A)WA=EMA, Order=A,A, LRS=Const2025.11 | 30.31 | 36.45 | 57.54 | 50.12 | 43.61 | 41.4 | 72.14 | 45.09 | 56.43 | 48.69 | |
| A,A + WSDWA=✗, Order=A,A, LRS=WSD2025.11 | 30.22 | 33.11 | 56.84 | 47.34 | 41.88 | 39.4 | 71.55 | 44.78 | 56.67 | 47.49 | |
| A-T + WSDWA=✗, Order=A-T, LRS=WSD2025.11 | 29.93 | 37.12 | 54.39 | 49.47 | 42.73 | 39 | 72.2 | 45.14 | 56.83 | 48.01 | |
| EMA + U,U (Constant LR)WA=EMA, Order=U,U, LRS=Const2025.11 | 29.84 | 32.78 | 52.28 | 51.52 | 41.6 | 42 | 71.6 | 44.68 | 56.99 | 47.71 | |
| EMA + U,A (Constant LR)WA=EMA, Order=U,A, LRS=Const2025.11 | 29.75 | 35.12 | 51.75 | 48.57 | 41.3 | 42.2 | 70.51 | 44.83 | 56.91 | 47.45 | |
| U,A + WSDWA=✗, Order=U,A, LRS=WSD2025.11 | 29.44 | 34.45 | 52.63 | 50.12 | 41.66 | 41 | 71.76 | 44.42 | 56.75 | 47.57 | |
| Baseline (U,U + WSD)WA=✗, Order=U,U, LRS=WSD2025.11 | 29.23 | 33.78 | 53.86 | 49.55 | 41.61 | 40.4 | 71.87 | 44.78 | 56.43 | 47.49 |