Mathematical Reasoning on GSM-Symbolic (test)
88.1AccuracyCCA w mask
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| CCA w maskModel=Gemma-2-9B-Instruct, Training Dataset=Branching, Dataset Size=512, Mask %=0.17%2025.12 | 88.1 | 0.015 | 7.4 | |
| CCA w/o maskModel=Gemma-2-9B-Instruct, Training Dataset=Branching, Dataset Size=5122025.12 | 87.5 | 0.01 | 6.8 | |
| LoRA FinetuningModel=Gemma-2-9B-Instruct, Training Dataset=GSym-Train, Dataset Size=6762025.12 | 85 | — | 4.3 | |
| CCA w/o maskModel=Gemma-2-9B-Instruct, Training Dataset=Prefix, Dataset Size=5102025.12 | 84.9 | 0.011 | 4.2 | |
| CCA w maskModel=Gemma-2-9B-Instruct, Training Dataset=Prefix, Dataset Size=510, Mask %=0.13%2025.12 | 84.8 | 0.006 | 4.1 | |
| Original ModelModel=Gemma-2-9B-Instruct2025.12 | 80.7 | — | — | |
| CCA w/o maskModel=OLMo-2-1124-7B-Instruct, Training Dataset=Branching, Dataset Size=9832025.12 | 80.6 | 0.012 | 6.7 | |
| LoRA FinetuningModel=OLMo-2-1124-13B-Instruct, Training Dataset=GSym-Train, Dataset Size=11182025.12 | 79.7 | — | 5.5 | |
| CCA w maskModel=OLMo-2-1124-7B-Instruct, Training Dataset=Branching, Dataset Size=983, Mask %=0.25%2025.12 | 79.4 | 0.006 | 5.5 | |
| CCA w maskModel=OLMo-2-1124-13B-Instruct, Training Dataset=Branching, Dataset Size=845, Mask %=0.44%2025.12 | 78.6 | 0.005 | 4.4 | |
| CCA w/o maskModel=OLMo-2-1124-13B-Instruct, Training Dataset=Branching, Dataset Size=8452025.12 | 78.4 | 0.006 | 4.2 | |
| CCA w/o maskModel=OLMo-2-1124-7B-Instruct, Training Dataset=Prefix, Dataset Size=9742025.12 | 77.7 | 0.006 | 3.8 | |
| CCA w maskModel=OLMo-2-1124-7B-Instruct, Training Dataset=Prefix, Dataset Size=974, Mask %=0.19%2025.12 | 77.2 | 0.018 | 3.3 | |
| CCA w maskModel=OLMo-2-1124-13B-Instruct, Training Dataset=Prefix, Dataset Size=864, Mask %=0.37%2025.12 | 76.8 | 0.018 | 2.6 | |
| CCA w/o maskModel=OLMo-2-1124-13B-Instruct, Training Dataset=Prefix, Dataset Size=8642025.12 | 76.2 | 0.002 | 2 | |
| LoRA FinetuningModel=OLMo-2-1124-7B-Instruct, Training Dataset=GSym-Train, Dataset Size=12222025.12 | 74.6 | — | 0.7 | |
| Original ModelModel=OLMo-2-1124-13B-Instruct2025.12 | 74.2 | — | — | |
| Original ModelModel=OLMo-2-1124-7B-Instruct2025.12 | 73.9 | — | — | |
| LoRA FinetuningModel=Gemma-2-2B-Instruct, Training Dataset=GSym-Train, Dataset Size=20282025.12 | 57.9 | — | 16.8 | |
| CCA w/o maskModel=Gemma-2-2B-Instruct, Training Dataset=Branching, Dataset Size=12442025.12 | 53.2 | 0.009 | 12.1 | |
| CCA w maskModel=Gemma-2-2B-Instruct, Training Dataset=Branching, Dataset Size=1244, Mask %=1.59%2025.12 | 52.5 | 0.01 | 11.4 | |
| CCA w/o maskModel=Gemma-2-2B-Instruct, Training Dataset=Prefix, Dataset Size=12832025.12 | 50.2 | 0.018 | 9.1 | |
| CCA w maskModel=Gemma-2-2B-Instruct, Training Dataset=Prefix, Dataset Size=1283, Mask %=0.92%2025.12 | 44 | 0.011 | 2.9 | |
| Original ModelModel=Gemma-2-2B-Instruct2025.12 | 41.1 | — | — |