Language Understanding on MMLU-Pro CoT
71.8Health AccuracyOptimal
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OptimalPRM=VersaPRM, LLM=Llama-3.1-8B-Instruct, Sample size=162025.10 | 71.8 | 78 | 76 | 85 | 79 | 84 | 65.3 | 78 | 62 | 85 | 70.7 | 75.9 | |
| KDE WVPRM=VersaPRM, LLM=Llama-3.1-8B-Instruct, Sample size=162025.10 | 64.7 | 57 | 63 | 66 | 58 | 64 | 42.1 | 58 | 45 | 65 | 46.5 | 57.2 | |
| Vanilla WVPRM=VersaPRM, LLM=Llama-3.1-8B-Instruct, Sample size=162025.10 | 63.5 | 57 | 62 | 66 | 59 | 64 | 43.2 | 60 | 46 | 65 | 48.5 | 57.7 | |
| Linear WVPRM=VersaPRM, LLM=Llama-3.1-8B-Instruct, Sample size=162025.10 | 63.5 | 57 | 64 | 66 | 62 | 67 | 44.2 | 63 | 47 | 66 | 46.5 | 58.7 | |
| MVPRM=VersaPRM, LLM=Llama-3.1-8B-Instruct, Sample size=162025.10 | 62.4 | 55 | 61 | 64 | 57 | 62 | 35.8 | 56 | 44 | 61 | 42.4 | 54.6 | |
| Logit WVPRM=VersaPRM, LLM=Llama-3.1-8B-Instruct, Sample size=162025.10 | 62.4 | 58 | 64 | 65 | 60 | 66 | 45.3 | 63 | 44 | 66 | 53.5 | 58.8 | |
| BoNPRM=VersaPRM, LLM=Llama-3.1-8B-Instruct, Sample size=162025.10 | 61.2 | 57 | 62 | 64 | 61 | 62 | 45.3 | 55 | 46 | 63 | 50.5 | 57 |