Reasoning and Classification on BBH (unseen)
98.8BBH Temporal SequencesRLHFlow-Llama3.1-8B-D
Evaluation results
| Method | Links | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RLHFlow-Llama3.1-8B-DK=72025.05 | 98.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 77.2 | 92.8 | 40 | |
| GPT-4Prompting=Direct, Setting=Zero-shot2023.05 | 98 | 77.6 | 59.9 | 74.8 | 69.2 | 64.4 | 40.8 | 62.8 | 66.8 | 66 | 94 | 79.5 | 68.8 | 76.7 | 84.8 | 89.1 | 62.4 | 87.6 | 84.4 | 25.2 | 25.2 | 42.4 | 49.6 | 67.4 | — | — | |
| FOVER-Llama3.1-8B-PRMK=72025.05 | 97.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 80 | 96 | 46.8 | |
| FOVER-Qwen2.5-7B-PRMK=72025.05 | 92.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 70.7 | 90 | 29.6 | |
| RLHFlow-Llama3.1-8B-MK=72025.05 | 92 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 74 | 90.8 | 39.2 | |
| Qwen 2.5 7BK=72025.05 | 91.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 69.7 | 89.2 | 28.8 | |
| Llama 3.1 8BK=72025.05 | 90.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 74 | 90 | 41.6 | |
| Qwen2.5-Math-PRM-7BK=72025.05 | 86.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 69.3 | 92.8 | 28.8 | |
| Qwen2.5-7B-Skywork-PRMK=72025.05 | 84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 68.9 | 91.2 | 31.6 | |
| Qwen2.5-Math-7B-PRM800KK=72025.05 | 82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 67.1 | 90.8 | 28.4 | |
| CODEXPrompting=Direct, Setting=Zero-shot2023.05 | 77.6 | 88.4 | 63.6 | 63.6 | 67.2 | 52.4 | 32 | 60.4 | 32.4 | 26 | 52.8 | 84.8 | 50.4 | 66.4 | 67.6 | 75.2 | 62 | 61.2 | 72.8 | 20.4 | 14.4 | 37.6 | 51.6 | 52.8 | — | — | |
| CHATGPTPrompting=Direct, Setting=Zero-shot2023.05 | 35.6 | 82.8 | 57.2 | 42.8 | 57.2 | 53.6 | 25.6 | 69.2 | 38.8 | 39.6 | 60.4 | 55.4 | 55.6 | 45.9 | 47.6 | 56 | 40.8 | 59 | 79.6 | 18.4 | 15.2 | 31.6 | 56 | 48.9 | — | — | |
| COT-T5-11BPrompting=Direct, Setting=Zero-shot, Parameters=11B2023.05 | 28.8 | 59.2 | 60.2 | 51 | 68.2 | 55.2 | 10.4 | 64.2 | 54.4 | 60.6 | 75 | 52.8 | 60 | 41.8 | 33.2 | 41.6 | 49.2 | 66.2 | 66.4 | 13.2 | 9.6 | 31.2 | 51.6 | 48 | — | — | |
| FLAN-T5-11BPrompting=Direct, Setting=Zero-shot, Parameters=11B2023.05 | 28.8 | 56.8 | 61 | 54.8 | 67.2 | 55.2 | 21.2 | 70.8 | 53.6 | 60 | 74.4 | 36.4 | 61.6 | 41.8 | 23.2 | 34.4 | 49.2 | 70.2 | 60 | 15.2 | 12 | 24.4 | 50 | 47.05 | — | — | |
| COT-T5-11BPrompting=CoT, Setting=Zero-shot, Parameters=11B2023.05 | 27.4 | 65.6 | 60.4 | 52 | 63.4 | 51.2 | 22 | 65.2 | 48.2 | 52.4 | 55.4 | 44.6 | 59 | 39.1 | 32.6 | 42.8 | 43.8 | 67.7 | 64.8 | 20 | 18.4 | 41.8 | 57 | 47.6 | — | — | |
| FLAN-T5-11BPrompting=CoT, Setting=Zero-shot, Parameters=11B2023.05 | 24.4 | 51.6 | 58.3 | 46.8 | 63.2 | 54.4 | 12.4 | 55.2 | 51.2 | 57.6 | 66.4 | 32.4 | 60.8 | 41.8 | 22.8 | 31.6 | 35.6 | 59.5 | 56 | 19.6 | 13.2 | 28.8 | 52.8 | 43.32 | — | — | |
| VICUNA-13BPrompting=Direct, Setting=Zero-shot, Parameters=13B2023.05 | 16 | 40.8 | 42.2 | 10 | 18.4 | 47.2 | 3.6 | 44 | 4.8 | 1.2 | 16.8 | 43.4 | 46.4 | 15.1 | 12 | 15.7 | 2 | 28.1 | 48.4 | 9.2 | 5.6 | 23.2 | 41.2 | 23.3 | — | — |