# Commonsense on HellaSwag 10-shot Source: [Commonsense on HellaSwag 10-shot]() Collection index: [llms.txt]() Data updated: 2026-03-04T12:24:40.254Z Evaluation of commonsense reasoning capabilities on the HellaSwag dataset using a 10-shot prompting methodology, measured by Score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Commonsense | | Dataset (including any stated split) | HellaSwag 10-shot | | Metric | Accuracy (10-shot) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy (10-shot)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy (10-shot) | Result date | Report paper | | --- | --- | --- | --- | | TuluDPO | 64.85 | Not recorded | 691aabe56750be83fe5ce08b | | UltraMix-190k | 64.82 | Not recorded | 691aabe56750be83fe5ce08b | | UltraMix-187k | 64.02 | Not recorded | 691aabe56750be83fe5ce08b | | UltraMix-190k | 63.89 | Not recorded | 691aabe56750be83fe5ce08b | | UltraMix-170k | 63.59 | Not recorded | 691aabe56750be83fe5ce08b | | UltraMix-187k | 63.54 | Not recorded | 691aabe56750be83fe5ce08b | | UltraMix-170k | 63.43 | Not recorded | 691aabe56750be83fe5ce08b | | TuluDPO | 62.7 | Not recorded | 691aabe56750be83fe5ce08b | | SFT | 60.85 | Not recorded | 691aabe56750be83fe5ce08b | | SFT | 60.41 | Not recorded | 691aabe56750be83fe5ce08b | | Instella-3B-SFT (UM-190k) | 58.74 | Not recorded | 691aabe56750be83fe5ce08b | | Instella-3B-SFT (UM-187k) | 58.56 | Not recorded | 691aabe56750be83fe5ce08b | | Instella-3B-SFT (UM-170k) | 58.23 | Not recorded | 691aabe56750be83fe5ce08b | | Instella-3B-SFT (TuluDPO) | 57.51 | Not recorded | 691aabe56750be83fe5ce08b | | Instella-3B-SFT (ORPO) | 56.72 | Not recorded | 691aabe56750be83fe5ce08b | | Instella-3B-SFT (UltraFB) | 56.3 | Not recorded | 691aabe56750be83fe5ce08b | | Instella-3B-SFT (HelpSteer) | 56.24 | Not recorded | 691aabe56750be83fe5ce08b | | Instella-3B-SFT (SFT) | 56.2 | Not recorded | 691aabe56750be83fe5ce08b | | Instella-3B-SFT (CodePref) | 55.99 | Not recorded | 691aabe56750be83fe5ce08b | ### TuluDPO Accuracy (10-shot): 64.85. Result ID: 69a824736dbb7075b94e3f58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B-TuluSFT","Training Mixture/Protocol":"TuluDPO"}. This record does not assert independent reproduction. ### UltraMix-190k Accuracy (10-shot): 64.82. Result ID: 69a824736dbb7075b94e3f5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B-TuluSFT","Training Mixture/Protocol":"UM-190k"}. This record does not assert independent reproduction. ### UltraMix-187k Accuracy (10-shot): 64.02. Result ID: 69a824736dbb7075b94e3f5a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B-TuluSFT","Training Mixture/Protocol":"UM-187k"}. This record does not assert independent reproduction. ### UltraMix-190k Accuracy (10-shot): 63.89. Result ID: 69a824736dbb7075b94e3f60. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen-2.5-7B-TuluSFT","Training Mixture/Protocol":"UM-190k"}. This record does not assert independent reproduction. ### UltraMix-170k Accuracy (10-shot): 63.59. Result ID: 69a824736dbb7075b94e3f5e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen-2.5-7B-TuluSFT","Training Mixture/Protocol":"UM-170k"}. This record does not assert independent reproduction. ### UltraMix-187k Accuracy (10-shot): 63.54. Result ID: 69a824736dbb7075b94e3f5f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen-2.5-7B-TuluSFT","Training Mixture/Protocol":"UM-187k"}. This record does not assert independent reproduction. ### UltraMix-170k Accuracy (10-shot): 63.43. Result ID: 69a824736dbb7075b94e3f59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B-TuluSFT","Training Mixture/Protocol":"UM-170k"}. This record does not assert independent reproduction. ### TuluDPO Accuracy (10-shot): 62.7. Result ID: 69a824736dbb7075b94e3f5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen-2.5-7B-TuluSFT","Training Mixture/Protocol":"TuluDPO"}. This record does not assert independent reproduction. ### SFT Accuracy (10-shot): 60.85. Result ID: 69a824736dbb7075b94e3f5c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen-2.5-7B-TuluSFT","Training Mixture/Protocol":"SFT"}. This record does not assert independent reproduction. ### SFT Accuracy (10-shot): 60.41. Result ID: 69a824736dbb7075b94e3f57. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B-TuluSFT","Training Mixture/Protocol":"SFT"}. This record does not assert independent reproduction. ### Instella-3B-SFT (UM-190k) Accuracy (10-shot): 58.74. Result ID: 69a824736dbb7075b94e3e38. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"DPO Training Dataset":"UM-190k","Evaluation Protocol":"10-shot"}. This record does not assert independent reproduction. ### Instella-3B-SFT (UM-187k) Accuracy (10-shot): 58.56. Result ID: 69a824736dbb7075b94e3e37. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"DPO Training Dataset":"UM-187k","Evaluation Protocol":"10-shot"}. This record does not assert independent reproduction. ### Instella-3B-SFT (UM-170k) Accuracy (10-shot): 58.23. Result ID: 69a824736dbb7075b94e3e36. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"DPO Training Dataset":"UM-170k","Evaluation Protocol":"10-shot"}. This record does not assert independent reproduction. ### Instella-3B-SFT (TuluDPO) Accuracy (10-shot): 57.51. Result ID: 69a824736dbb7075b94e3e31. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"DPO Training Dataset":"TuluDPO","Evaluation Protocol":"10-shot"}. This record does not assert independent reproduction. ### Instella-3B-SFT (ORPO) Accuracy (10-shot): 56.72. Result ID: 69a824736dbb7075b94e3e32. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"DPO Training Dataset":"ORPO","Evaluation Protocol":"10-shot"}. This record does not assert independent reproduction. ### Instella-3B-SFT (UltraFB) Accuracy (10-shot): 56.3. Result ID: 69a824736dbb7075b94e3e33. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"DPO Training Dataset":"UltraFB","Evaluation Protocol":"10-shot"}. This record does not assert independent reproduction. ### Instella-3B-SFT (HelpSteer) Accuracy (10-shot): 56.24. Result ID: 69a824736dbb7075b94e3e34. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"DPO Training Dataset":"HelpSteer","Evaluation Protocol":"10-shot"}. This record does not assert independent reproduction. ### Instella-3B-SFT (SFT) Accuracy (10-shot): 56.2. Result ID: 69a824736dbb7075b94e3e30. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"DPO Training Dataset":"None (SFT Baseline)","Evaluation Protocol":"10-shot"}. This record does not assert independent reproduction. ### Instella-3B-SFT (CodePref) Accuracy (10-shot): 55.99. Result ID: 69a824736dbb7075b94e3e35. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"DPO Training Dataset":"CodePref","Evaluation Protocol":"10-shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.