# General Language Understanding on 8 Sub-Tasks (test)

Source: [General Language Understanding on 8 Sub-Tasks (test)](<https://www.sota2.com/research/sota/general-language-understanding-on-8-sub-tasks-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-16T09:49:25.748Z

Evaluation of general language understanding capabilities across 8 specific sub-tasks, measured on the test split.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | General Language Understanding |
| Dataset (including any stated split) | 8 Sub-Tasks (test) |
| Metric | Performance on 8 Sub-Tasks |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Performance on 8 Sub-Tasks](<https://www.sota2.com/research/sota/general-language-understanding-on-8-sub-tasks-test.md?metric=Performance%20on%208%20Sub-Tasks>): Higher is better; unit: PERCENT.

- [Average Score](<https://www.sota2.com/research/sota/general-language-understanding-on-8-sub-tasks-test.md?metric=Average%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Performance on 8 Sub-Tasks | Result date | Report paper |
| --- | --- | --- | --- |
| LoRA | 62.3 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| Original | 61.8 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| MoBE | 59.7 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| LightMoE | 59.2 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| HC-SMoE | 57.8 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| MoBE | 57.6 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| LightMoE | 57.3 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| MC-SMoE | 57 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| HC-SMoE | 55.1 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| MC-SMoE | 54.7 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| LightMoE | 54.3 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| HC-SMoE | 53 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| MC-SMoE | 51.3 | Not recorded | 69b78e5e0636e5bf4afbda39 |
| MoBE | 50.7 | Not recorded | 69b78e5e0636e5bf4afbda39 |

### LoRA

Performance on 8 Sub-Tasks: 62.3. Result ID: 69b7d0fa47f34ce9249fe9ab.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b121e209131cea2bf59.md>)

[Implementation](<https://github.com/microsoft/LoRA>)

Recorded evaluation configuration: {"Ratio":"0%","# Params":"0.96B"}. This record does not assert independent reproduction.

### Original

Performance on 8 Sub-Tasks: 61.8. Result ID: 69b7d0fa47f34ce9249fe9aa.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa773e1e209131cea71f14.md>)

Recorded evaluation configuration: {"Ratio":"0%","# Params":"-"}. This record does not assert independent reproduction.

### MoBE

Performance on 8 Sub-Tasks: 59.7. Result ID: 69b7d0fa47f34ce9249fe9ac.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a6e1e209131cea83265.md>)

Recorded evaluation configuration: {"Ratio":"30%","# Params":"0.96B"}. This record does not assert independent reproduction.

### LightMoE

Performance on 8 Sub-Tasks: 59.2. Result ID: 69b7d0fa47f34ce9249fe9af.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

Recorded evaluation configuration: {"Ratio":"30%","# Params":"0.96B"}. This record does not assert independent reproduction.

### HC-SMoE

Performance on 8 Sub-Tasks: 57.8. Result ID: 69b7d0fa47f34ce9249fe9ad.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80afe.md>)

Recorded evaluation configuration: {"Ratio":"30%","# Params":"0.96B"}. This record does not assert independent reproduction.

### MoBE

Performance on 8 Sub-Tasks: 57.6. Result ID: 69b7d0fa47f34ce9249fe9b0.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a6e1e209131cea83265.md>)

Recorded evaluation configuration: {"Ratio":"40%","# Params":"0.96B"}. This record does not assert independent reproduction.

### LightMoE

Performance on 8 Sub-Tasks: 57.3. Result ID: 69b7d0fa47f34ce9249fe9b3.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

Recorded evaluation configuration: {"Ratio":"40%","# Params":"0.96B"}. This record does not assert independent reproduction.

### MC-SMoE

Performance on 8 Sub-Tasks: 57. Result ID: 69b7d0fa47f34ce9249fe9ae.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72841e209131cea56f90.md>)

Recorded evaluation configuration: {"Ratio":"30%","# Params":"0.96B"}. This record does not assert independent reproduction.

### HC-SMoE

Performance on 8 Sub-Tasks: 55.1. Result ID: 69b7d0fa47f34ce9249fe9b1.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80afe.md>)

Recorded evaluation configuration: {"Ratio":"40%","# Params":"0.96B"}. This record does not assert independent reproduction.

### MC-SMoE

Performance on 8 Sub-Tasks: 54.7. Result ID: 69b7d0fa47f34ce9249fe9b2.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72841e209131cea56f90.md>)

Recorded evaluation configuration: {"Ratio":"40%","# Params":"0.96B"}. This record does not assert independent reproduction.

### LightMoE

Performance on 8 Sub-Tasks: 54.3. Result ID: 69b7d0fa47f34ce9249fe9b7.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

Recorded evaluation configuration: {"Ratio":"50%","# Params":"0.96B"}. This record does not assert independent reproduction.

### HC-SMoE

Performance on 8 Sub-Tasks: 53. Result ID: 69b7d0fa47f34ce9249fe9b5.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80afe.md>)

Recorded evaluation configuration: {"Ratio":"50%","# Params":"0.96B"}. This record does not assert independent reproduction.

### MC-SMoE

Performance on 8 Sub-Tasks: 51.3. Result ID: 69b7d0fa47f34ce9249fe9b6.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72841e209131cea56f90.md>)

Recorded evaluation configuration: {"Ratio":"50%","# Params":"0.96B"}. This record does not assert independent reproduction.

### MoBE

Performance on 8 Sub-Tasks: 50.7. Result ID: 69b7d0fa47f34ce9249fe9b4.

[Reporting paper](<https://www.sota2.com/research/paper/69b78e5e0636e5bf4afbda39.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a6e1e209131cea83265.md>)

Recorded evaluation configuration: {"Ratio":"50%","# Params":"0.96B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
