# Mathematical Reasoning on SVAMP (val) Source: [Mathematical Reasoning on SVAMP (val)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:29:59.741Z Evaluation of model performance on mathematical reasoning tasks using the validation split of the SVAMP dataset, typically measured by Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical Reasoning | | Dataset (including any stated split) | SVAMP (val) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 36 of 36 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | TAIA | 85.1 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 84.9 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 83.1 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 82.8 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 79.3 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 78.6 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 71.2 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 69.7 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 67.2 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 65.6 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 61.9 | Not recorded | 68fa75401e209131cea66d49 | | LLaMA3-8B | 60.33 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 58.81 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 58.33 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 57.1 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 57.08 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 56.8 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 56 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 55.4 | Not recorded | 68fa75401e209131cea66d49 | | Qwen1.5-7B | 54.9 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 53.6 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 52.7 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 52.5 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 51.8 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 45.4 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 44.8 | Not recorded | 68fa75401e209131cea66d49 | | LLaMA2-7B | 44.5 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 43.4 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 40.3 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 39 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 38.2 | Not recorded | 68fa75401e209131cea66d49 | | LoRA | 34.8 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 31.1 | Not recorded | 68fa75401e209131cea66d49 | | TAIA | 29.9 | Not recorded | 68fa75401e209131cea66d49 | | MOLORA | 25.2 | Not recorded | 68fa75401e209131cea66d49 | | Qwen1.5-1.8B | 24.8 | Not recorded | 68fa75401e209131cea66d49 | ### TAIA Accuracy: 85.1. Result ID: 6993978b3a6c31484917196e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA3-8B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 84.9. Result ID: 6993978b3a6c314849171970. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA3-8B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 83.1. Result ID: 6993978b3a6c314849171980. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA3-8B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 82.8. Result ID: 6993978b3a6c31484917197e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA3-8B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 79.3. Result ID: 6993978b3a6c31484917197f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA3-8B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### LoRA Accuracy: 78.6. Result ID: 6993978b3a6c31484917197d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA3-8B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 71.2. Result ID: 6993978b3a6c31484917196f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA3-8B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### LoRA Accuracy: 69.7. Result ID: 6993978b3a6c31484917196d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA3-8B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 67.2. Result ID: 6993978b3a6c314849171966. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-7B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 65.6. Result ID: 6993978b3a6c314849171968. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-7B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 61.9. Result ID: 6993978b3a6c314849171976. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-7B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### LLaMA3-8B Accuracy: 60.33. Result ID: 6993978b3a6c314849171960. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Base Model","Model":"LLaMA3-8B"}. This record does not assert independent reproduction. ### TAIA Accuracy: 58.81. Result ID: 6993978b3a6c314849171978. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-7B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 58.33. Result ID: 6993978b3a6c31484917197a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA2-7B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### LoRA Accuracy: 57.1. Result ID: 6993978b3a6c314849171965. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-7B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 57.08. Result ID: 6993978b3a6c31484917196a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA2-7B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 56.8. Result ID: 6993978b3a6c314849171967. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-7B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 56. Result ID: 6993978b3a6c31484917197b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA2-7B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 55.4. Result ID: 6993978b3a6c31484917197c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA2-7B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### Qwen1.5-7B Accuracy: 54.9. Result ID: 6993978b3a6c31484917195e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Base Model","Model":"Qwen1.5-7B"}. This record does not assert independent reproduction. ### LoRA Accuracy: 53.6. Result ID: 6993978b3a6c314849171975. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-7B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 52.7. Result ID: 6993978b3a6c314849171977. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-7B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### LoRA Accuracy: 52.5. Result ID: 6993978b3a6c314849171979. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"LLaMA2-7B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 51.8. Result ID: 6993978b3a6c31484917196c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA2-7B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 45.4. Result ID: 6993978b3a6c31484917196b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA2-7B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 44.8. Result ID: 6993978b3a6c314849171974. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-1.8B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### LLaMA2-7B Accuracy: 44.5. Result ID: 6993978b3a6c31484917195f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Base Model","Model":"LLaMA2-7B"}. This record does not assert independent reproduction. ### LoRA Accuracy: 43.4. Result ID: 6993978b3a6c314849171969. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"LLaMA2-7B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 40.3. Result ID: 6993978b3a6c314849171972. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-1.8B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### LoRA Accuracy: 39. Result ID: 6993978b3a6c314849171971. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-1.8B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 38.2. Result ID: 6993978b3a6c314849171973. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"CoT-Collection","Model":"Qwen1.5-1.8B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### LoRA Accuracy: 34.8. Result ID: 6993978b3a6c314849171961. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-1.8B","FT Method":"LoRA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### TAIA Accuracy: 31.1. Result ID: 6993978b3a6c314849171964. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-1.8B","FT Method":"MOLORA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### TAIA Accuracy: 29.9. Result ID: 6993978b3a6c314849171962. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-1.8B","FT Method":"LoRA","Infer Mode":"TAIA"}. This record does not assert independent reproduction. ### MOLORA Accuracy: 25.2. Result ID: 6993978b3a6c314849171963. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Alpaca-GPT4","Model":"Qwen1.5-1.8B","FT Method":"MOLORA","Infer Mode":"Vanilla"}. This record does not assert independent reproduction. ### Qwen1.5-1.8B Accuracy: 24.8. Result ID: 6993978b3a6c31484917195d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Dataset":"Base Model","Model":"Qwen1.5-1.8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.