Reasoning on AQuA, MMLU, StrategyQA, and CommonSenseQA (Average)
74AccuracyLoT
Evaluation results
| Method | Links | |
|---|---|---|
| LoTGenerator Model Scale=70B, Verifier Model Scale=1B2025.03 | 74 | |
| LoTGenerator Model Scale=70B, Verifier Model Scale=3B2025.03 | 73 | |
| LoTGenerator Model Scale=70B, Verifier Model Scale=8B2025.03 | 72.5 | |
| LoTGenerator Model Scale=70B, Verifier Model Scale=70B2025.03 | 72.5 | |
| LoTGenerator Model Scale=8B, Verifier Model Scale=1B2025.03 | 60 | |
| LoTGenerator Model Scale=8B, Verifier Model Scale=3B2025.03 | 60 | |
| LoTGenerator Model Scale=8B, Verifier Model Scale=8B2025.03 | 60 | |
| LoTGenerator Model Scale=8B, Verifier Model Scale=70B2025.03 | 60 | |
| LoTGenerator Model Scale=3B, Verifier Model Scale=8B2025.03 | 51 | |
| LoTGenerator Model Scale=3B, Verifier Model Scale=70B2025.03 | 51 | |
| LoTGenerator Model Scale=3B, Verifier Model Scale=3B2025.03 | 48 | |
| LoTGenerator Model Scale=3B, Verifier Model Scale=1B2025.03 | 45.5 | |
| LoTGenerator Model Scale=1B, Verifier Model Scale=3B2025.03 | 27.5 | |
| LoTGenerator Model Scale=1B, Verifier Model Scale=8B2025.03 | 27.5 | |
| LoTGenerator Model Scale=1B, Verifier Model Scale=70B2025.03 | 27.5 | |
| LoTGenerator Model Scale=1B, Verifier Model Scale=1B2025.03 | 26 |