Question Answering on ARC-E (Accuracy, Token, Speed)
91.2AccuracyLatentMAS
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| LatentMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Sequential2025.11 | 91.2 | 228 | 516 | |
| LatentMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Hierarchical2025.11 | 90.4 | 206 | 669 | |
| TextMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Sequential2025.11 | 89.6 | 1,981 | 2,708 | |
| TextMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Hierarchical2025.11 | 89.3 | 1,118 | 1,832 | |
| SingleBackbone=Llama-3.1-8B-Instruct, MAS Setting=Sequential2025.11 | 85.7 | 219 | 352 | |
| SingleBackbone=Llama-3.1-8B-Instruct, MAS Setting=Hierarchical2025.11 | 85.7 | 219 | 352 | |
| LatentMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Hierarchical2025.11 | 82.3 | 177 | 192 | |
| TextMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Hierarchical2025.11 | 81.6 | 956 | 615 | |
| LatentMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Sequential2025.11 | 80.4 | 229 | 392 | |
| TextMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Sequential2025.11 | 80.1 | 1,889 | 1,670 | |
| SingleBackbone=Llama-3.2-3B-Instruct, MAS Setting=Sequential2025.11 | 79.2 | 236 | 155 | |
| SingleBackbone=Llama-3.2-3B-Instruct, MAS Setting=Hierarchical2025.11 | 79.2 | 236 | 155 |