Mathematical Problem Solving on MMLU STEM (test)
75Maj@kMinerva 540B
Evaluation results
| Method | Links | |
|---|---|---|
| Minerva 540BSize=540B, k (samples)=16, Temperature (T)=0.3, Nucleus sampling (p)=0.952023.10 | 75 | |
| Minerva 62BSize=62B, k (samples)=16, Temperature (T)=0.3, Nucleus sampling (p)=0.952023.10 | 63.5 | |
| LLEMMA 34BSize=34B, k (samples)=16, Temperature (T)=0.3, Nucleus sampling (p)=0.952023.10 | 59.7 | |
| LLEMMA 7BSize=7B, k (samples)=16, Temperature (T)=0.3, Nucleus sampling (p)=0.952023.10 | 49.9 | |
| Minerva 8BSize=8B, k (samples)=16, Temperature (T)=0.3, Nucleus sampling (p)=0.952023.10 | 43.4 |