# General Reasoning on BIG-Bench Hard (val) Source: [General Reasoning on BIG-Bench Hard (val)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:29:59.721Z Evaluation of model performance on general reasoning tasks within the BIG-Bench Hard (BBH) suite, specifically using the validation (val) split, measured by Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | General Reasoning | | Dataset (including any stated split) | Big-Bench Hard (BBH) (val) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 36 of 36 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | TAIA | 43.46 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 42.54 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 37.35 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 36.09 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 35.85 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 35.05 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 33.19 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 32.64 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 32.54 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 31.47 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 31.3 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 30.96 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 30.93 | Not recorded | 68fa75401e209131cea66d49 | | Qwen1.5-7B | 30.76 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 30.2 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 30.03 | Not recorded | 68fa75401e209131cea66d49 | | LLaMA3-8B | 29.58 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 29.1 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 28.8 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 28.63 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 27.71 | Not recorded | 68fa75401e209131cea66d49 | | LLaMA2-7B | 26.36 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 26 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 24.07 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 23.67 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 23.24 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 22.5 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 22.49 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 22.24 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 21.93 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 20.21 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 19.09 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 18.86 | Not recorded | 68fa75401e209131cea66d49 | | Qwen1.5-1.8B | 16.8 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 13.9 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 12.99 | Not recorded | 68fa75401e209131cea66d49 | ### TAIA Accuracy: 43.46. Result ID: 6993978b3a6c3148491718fa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-7B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 42.54. Result ID: 6993978b3a6c3148491718fc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-7B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### LoRA Accuracy: 37.35. Result ID: 6993978b3a6c314849171901. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA3-8B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### LoRA Accuracy: 36.09. Result ID: 6993978b3a6c3148491718f9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-7B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 35.85. Result ID: 6993978b3a6c314849171903. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA3-8B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 35.05. Result ID: 6993978b3a6c3148491718fb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-7B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### LoRA Accuracy: 33.19. Result ID: 6993978b3a6c3148491718fd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA2-7B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 32.64. Result ID: 6993978b3a6c3148491718ff. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA2-7B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 32.54. Result ID: 6993978b3a6c31484917190a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-7B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 31.47. Result ID: 6993978b3a6c3148491718fe. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA2-7B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 31.3. Result ID: 6993978b3a6c314849171902. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA3-8B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 30.96. Result ID: 6993978b3a6c31484917190c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-7B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 30.93. Result ID: 6993978b3a6c314849171900. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA2-7B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### Qwen1.5-7B Accuracy: 30.76. Result ID: 6993978b3a6c3148491718f2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Base Model","Model":"Qwen1.5-7B"}. This record does not assert independent reproduction. ### TAIA Accuracy: 30.2. Result ID: 6993978b3a6c314849171910. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA2-7B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 30.03. Result ID: 6993978b3a6c3148491718f6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-1.8B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### LLaMA3-8B Accuracy: 29.58. Result ID: 6993978b3a6c3148491718f4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Base Model","Model":"LLaMA3-8B"}. This record does not assert independent reproduction. ### TAIA Accuracy: 29.1. Result ID: 6993978b3a6c314849171904. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA3-8B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### LoRA Accuracy: 28.8. Result ID: 6993978b3a6c3148491718f5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-1.8B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 28.63. Result ID: 6993978b3a6c314849171914. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA3-8B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 27.71. Result ID: 6993978b3a6c3148491718f8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-1.8B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### LLaMA2-7B Accuracy: 26.36. Result ID: 6993978b3a6c3148491718f3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Base Model","Model":"LLaMA2-7B"}. This record does not assert independent reproduction. ### TAIA Accuracy: 26. Result ID: 6993978b3a6c31484917190e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA2-7B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### LoRA Accuracy: 24.07. Result ID: 6993978b3a6c314849171909. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-7B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 23.67. Result ID: 6993978b3a6c3148491718f7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-1.8B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### LoRA Accuracy: 23.24. Result ID: 6993978b3a6c314849171911. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA3-8B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 22.5. Result ID: 6993978b3a6c31484917190b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-7B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 22.49. Result ID: 6993978b3a6c314849171908. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-1.8B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 22.24. Result ID: 6993978b3a6c314849171913. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA3-8B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 21.93. Result ID: 6993978b3a6c314849171906. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-1.8B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 20.21. Result ID: 6993978b3a6c31484917190f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA2-7B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### LoRA Accuracy: 19.09. Result ID: 6993978b3a6c31484917190d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA2-7B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 18.86. Result ID: 6993978b3a6c314849171912. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA3-8B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### Qwen1.5-1.8B Accuracy: 16.8. Result ID: 6993978b3a6c3148491718f1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Base Model","Model":"Qwen1.5-1.8B"}. This record does not assert independent reproduction. ### LoRA Accuracy: 13.9. Result ID: 6993978b3a6c314849171905. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-1.8B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 12.99. Result ID: 6993978b3a6c314849171907. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-1.8B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.