# Commonsense Reasoning on HellaSwag 10-shot Source: [Commonsense Reasoning on HellaSwag 10-shot]() Collection index: [llms.txt]() Data updated: 2026-04-28T09:32:48.448Z Evaluation of model performance on the HellaSwag dataset using a 10-shot prompting methodology for commonsense reasoning, measured by Score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Commonsense Reasoning | | Dataset (including any stated split) | HellaSwag 10-shot | | Metric | Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Score | Result date | Report paper | | --- | --- | --- | --- | | FP16 | 83.3 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | COVERCAL | 82.8 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | FP16 | 82.1 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Max-ActVar | 82.1 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Stratified | 82 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Random | 81.9 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Max-PPL | 81.7 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | COVERCAL | 81.5 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Max-ActVar | 80.8 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Stratified | 80.7 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Random | 80.5 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Max-PPL | 80.3 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | FP16 | 78.6 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | COVERCAL | 78 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Max-ActVar | 77.4 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Stratified | 77.3 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Random | 77.1 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | Max-PPL | 77 | Not recorded | 69f03f2d55a3c32af89fa1b3 | | UM-190k | 62.86 | Not recorded | 691aabe56750be83fe5ce08b | | UM-187k | 62.54 | Not recorded | 691aabe56750be83fe5ce08b | | UM-170k | 62.17 | Not recorded | 691aabe56750be83fe5ce08b | | TuluDPO | 61.97 | Not recorded | 691aabe56750be83fe5ce08b | | ORPO | 61.3 | Not recorded | 691aabe56750be83fe5ce08b | | HelpSteer | 60.55 | Not recorded | 691aabe56750be83fe5ce08b | | UltraFB | 60.24 | Not recorded | 691aabe56750be83fe5ce08b | | CodePref | 60.24 | Not recorded | 691aabe56750be83fe5ce08b | | SFT | 60.14 | Not recorded | 691aabe56750be83fe5ce08b | ### FP16 Score: 83.3. Result ID: 69f07eac47f34ce924d6c394. [Reporting paper]() Recorded evaluation configuration: {"Model":"Mistral-7B","Quantization":"FP16"}. This record does not assert independent reproduction. ### COVERCAL Score: 82.8. Result ID: 69f07eac47f34ce924d6c399. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Mistral-7B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### FP16 Score: 82.1. Result ID: 69f07eac47f34ce924d6c38e. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3-8B","Quantization":"FP16"}. This record does not assert independent reproduction. ### Max-ActVar Score: 82.1. Result ID: 69f07eac47f34ce924d6c397. [Reporting paper]() Recorded evaluation configuration: {"Model":"Mistral-7B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Stratified Score: 82. Result ID: 69f07eac47f34ce924d6c398. [Reporting paper]() Recorded evaluation configuration: {"Model":"Mistral-7B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Random Score: 81.9. Result ID: 69f07eac47f34ce924d6c395. [Reporting paper]() Recorded evaluation configuration: {"Model":"Mistral-7B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Max-PPL Score: 81.7. Result ID: 69f07eac47f34ce924d6c396. [Reporting paper]() Recorded evaluation configuration: {"Model":"Mistral-7B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### COVERCAL Score: 81.5. Result ID: 69f07eac47f34ce924d6c393. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-3-8B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Max-ActVar Score: 80.8. Result ID: 69f07eac47f34ce924d6c391. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3-8B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Stratified Score: 80.7. Result ID: 69f07eac47f34ce924d6c392. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3-8B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Random Score: 80.5. Result ID: 69f07eac47f34ce924d6c38f. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3-8B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Max-PPL Score: 80.3. Result ID: 69f07eac47f34ce924d6c390. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-3-8B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### FP16 Score: 78.6. Result ID: 69f07eac47f34ce924d6c388. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-2-7B","Quantization":"FP16"}. This record does not assert independent reproduction. ### COVERCAL Score: 78. Result ID: 69f07eac47f34ce924d6c38d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA-2-7B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Max-ActVar Score: 77.4. Result ID: 69f07eac47f34ce924d6c38b. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-2-7B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Stratified Score: 77.3. Result ID: 69f07eac47f34ce924d6c38c. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-2-7B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Random Score: 77.1. Result ID: 69f07eac47f34ce924d6c389. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-2-7B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### Max-PPL Score: 77. Result ID: 69f07eac47f34ce924d6c38a. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA-2-7B","Quantization":"AWQ INT4","K (Calibration Samples)":"128"}. This record does not assert independent reproduction. ### UM-190k Score: 62.86. Result ID: 69a824736dbb7075b94e4122. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"OLMo-2-7B-SFT","DPO Training Dataset":"UM-190k"}. This record does not assert independent reproduction. ### UM-187k Score: 62.54. Result ID: 69a824736dbb7075b94e4121. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"OLMo-2-7B-SFT","DPO Training Dataset":"UM-187k"}. This record does not assert independent reproduction. ### UM-170k Score: 62.17. Result ID: 69a824736dbb7075b94e4120. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"OLMo-2-7B-SFT","DPO Training Dataset":"UM-170k"}. This record does not assert independent reproduction. ### TuluDPO Score: 61.97. Result ID: 69a824736dbb7075b94e411b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"OLMo-2-7B-SFT","DPO Training Dataset":"TuluDPO"}. This record does not assert independent reproduction. ### ORPO Score: 61.3. Result ID: 69a824736dbb7075b94e411c. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"OLMo-2-7B-SFT","DPO Training Dataset":"ORPO"}. This record does not assert independent reproduction. ### HelpSteer Score: 60.55. Result ID: 69a824736dbb7075b94e411e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"OLMo-2-7B-SFT","DPO Training Dataset":"HelpSteer"}. This record does not assert independent reproduction. ### UltraFB Score: 60.24. Result ID: 69a824736dbb7075b94e411d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"OLMo-2-7B-SFT","DPO Training Dataset":"UltraFB"}. This record does not assert independent reproduction. ### CodePref Score: 60.24. Result ID: 69a824736dbb7075b94e411f. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"OLMo-2-7B-SFT","DPO Training Dataset":"CodePref"}. This record does not assert independent reproduction. ### SFT Score: 60.14. Result ID: 69a824736dbb7075b94e411a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"OLMo-2-7B-SFT","DPO Training Dataset":"SFT"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.