Multi-task Language Understanding on MMLU (Inference Latency)
2.07Average Inference Time (s)CoT
Evaluation Results
| Method | Links | |
|---|---|---|
| CoTModel=GPT-3.52025.09 | 2.07 | |
| CoTModel=Mixtral 8x22b2025.09 | 2.29 | |
| CoTModel=Llama3-70b2025.09 | 2.73 | |
| CoTModel=GPT-4o-mini2025.09 | 3.48 | |
| Self-reflectionModel=Mixtral 8x22b2025.09 | 3.9 | |
| Self-reflectionModel=GPT-3.52025.09 | 4.28 | |
| Self-consistencyModel=GPT-3.52025.09 | 4.93 | |
| Self-reflectionModel=GPT-4o-mini2025.09 | 5.42 | |
| Self-reflectionModel=Llama3-70b2025.09 | 6.86 | |
| MARSModel=GPT-3.52025.09 | 7.61 | |
| DMADModel=GPT-3.52025.09 | 10.67 | |
| MADModel=GPT-3.52025.09 | 11.38 | |
| Self-consistencyModel=Mixtral 8x22b2025.09 | 14.44 | |
| Self-consistencyModel=GPT-4o-mini2025.09 | 14.55 | |
| MARSModel=Mixtral 8x22b2025.09 | 16.15 | |
| MADModel=Mixtral 8x22b2025.09 | 17.39 | |
| CoTModel=Mixtral 8x7b2025.09 | 20.38 | |
| DMADModel=Mixtral 8x22b2025.09 | 23.89 | |
| MARSModel=GPT-4o-mini2025.09 | 28.14 | |
| Self-consistencyModel=Llama3-70b2025.09 | 29.6 | |
| Self-reflectionModel=Mixtral 8x7b2025.09 | 29.82 | |
| MARSModel=Llama3-70b2025.09 | 36.49 | |
| MADModel=GPT-4o-mini2025.09 | 41.58 | |
| DMADModel=GPT-4o-mini2025.09 | 42.96 | |
| Self-consistencyModel=Mixtral 8x7b2025.09 | 47.76 | |
| MADModel=Llama3-70b2025.09 | 51.78 | |
| MARSModel=Mixtral 8x7b2025.09 | 55.91 | |
| DMADModel=Llama3-70b2025.09 | 64.03 | |
| MADModel=Mixtral 8x7b2025.09 | 77.31 | |
| DMADModel=Mixtral 8x7b2025.09 | 105.87 |