Language Understanding on LLM Evaluation Harness
22.9ARC-Challenge AccBased
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BasedArchitecture=Based, Shot=1-shot2024.02 | 22.9 | — | 40.4 | — | — | — | — | 50.4 | 62.1 | — | 30.8 | — | — | — | 41.3 | |
| ReBasedArchitecture=ReBased, Shot=1-shot2024.02 | 21.5 | — | 41 | — | — | — | — | 54.6 | 62.8 | — | 30.7 | — | — | — | 42.1 | |
| MH-MoEExperts Number (N)=322024.04 | 21.4 | 26.8 | 50.6 | 44.8 | 53.4 | 18.8 | 31.6 | 53.8 | 69.3 | 56.6 | 35 | 42.1 | 24.8 | 39.5 | 40.6 | |
| MH-MoEExperts Number (N)=82024.04 | 19.6 | 25.2 | 50.2 | 42.2 | 53 | 18.2 | 30.3 | 51.1 | 68.7 | 59.6 | 33.2 | 40.3 | 24.7 | 40.9 | 39.8 | |
| X-MoEExperts Number (N)=322024.04 | 19.4 | 24.8 | 50.4 | 42.5 | 52.7 | 17.8 | 30 | 51.3 | 68.8 | 52.8 | 33.4 | 40.1 | 24.3 | 39.1 | 39.1 | |
| X-MoEExperts Number (N)=82024.04 | 19 | 24.7 | 48.3 | 42 | 52.7 | 17.4 | 29.8 | 50.3 | 67.9 | 58.4 | 31.4 | 35.7 | 24.3 | 40.2 | 38.7 | |
| Dense2024.04 | 18.1 | 23.3 | 44.9 | 39.7 | 51.5 | 17.1 | 29 | 48.2 | 66.6 | 55 | 29.7 | 34.1 | 24.1 | 39.3 | 37.2 |