Mathematical Reasoning on GSM8K (test) (Acc, Latency)
90.52AccuracyOracle
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Oracle2024.08 | 90.52 | 3.24 | |
| SELECTLLMPolicy=MLC + WEIGHTEDMAXCONF, s (number of LLMs)=42024.08 | 77.94 | 16.5 | |
| Top-s LLMss (number of LLMs)=42024.08 | 77.48 | 19 | |
| SELECTLLMPolicy=MLC + MAXCONF, s (number of LLMs)=42024.08 | 77.48 | 16.5 | |
| All LLMs2024.08 | 76.04 | 19 | |
| SELECTLLMPolicy=MLC + LABELLEDMAXCONF, s (number of LLMs)=42024.08 | 75.66 | 14.69 | |
| LLM-Blender2024.08 | 75.28 | 19 | |
| Random2024.08 | 69.49 | 9.65 | |
| SELECTLLMPolicy=MLC + WEIGHTEDMAXCONF, s (number of LLMs)=12024.08 | 67.24 | 4.7 |