List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM-Symbolic (test)

88.1AccuracyCCA w mask
39.2251.9164.677.29Dec 18, 2025
Updated 8mo ago

Evaluation results

MethodLinks
CCA w maskModel=Gemma-2-9B-Instruct, Training Dataset=Branching, Dataset Size=512, Mask %=0.17%2025.1288.10.0157.4
CCA w/o maskModel=Gemma-2-9B-Instruct, Training Dataset=Branching, Dataset Size=5122025.1287.50.016.8
LoRA FinetuningModel=Gemma-2-9B-Instruct, Training Dataset=GSym-Train, Dataset Size=6762025.1285—4.3
CCA w/o maskModel=Gemma-2-9B-Instruct, Training Dataset=Prefix, Dataset Size=5102025.1284.90.0114.2
CCA w maskModel=Gemma-2-9B-Instruct, Training Dataset=Prefix, Dataset Size=510, Mask %=0.13%2025.1284.80.0064.1
Original ModelModel=Gemma-2-9B-Instruct2025.1280.7——
CCA w/o maskModel=OLMo-2-1124-7B-Instruct, Training Dataset=Branching, Dataset Size=9832025.1280.60.0126.7
LoRA FinetuningModel=OLMo-2-1124-13B-Instruct, Training Dataset=GSym-Train, Dataset Size=11182025.1279.7—5.5
CCA w maskModel=OLMo-2-1124-7B-Instruct, Training Dataset=Branching, Dataset Size=983, Mask %=0.25%2025.1279.40.0065.5
CCA w maskModel=OLMo-2-1124-13B-Instruct, Training Dataset=Branching, Dataset Size=845, Mask %=0.44%2025.1278.60.0054.4
CCA w/o maskModel=OLMo-2-1124-13B-Instruct, Training Dataset=Branching, Dataset Size=8452025.1278.40.0064.2
CCA w/o maskModel=OLMo-2-1124-7B-Instruct, Training Dataset=Prefix, Dataset Size=9742025.1277.70.0063.8
CCA w maskModel=OLMo-2-1124-7B-Instruct, Training Dataset=Prefix, Dataset Size=974, Mask %=0.19%2025.1277.20.0183.3
CCA w maskModel=OLMo-2-1124-13B-Instruct, Training Dataset=Prefix, Dataset Size=864, Mask %=0.37%2025.1276.80.0182.6
CCA w/o maskModel=OLMo-2-1124-13B-Instruct, Training Dataset=Prefix, Dataset Size=8642025.1276.20.0022
LoRA FinetuningModel=OLMo-2-1124-7B-Instruct, Training Dataset=GSym-Train, Dataset Size=12222025.1274.6—0.7
Original ModelModel=OLMo-2-1124-13B-Instruct2025.1274.2——
Original ModelModel=OLMo-2-1124-7B-Instruct2025.1273.9——
LoRA FinetuningModel=Gemma-2-2B-Instruct, Training Dataset=GSym-Train, Dataset Size=20282025.1257.9—16.8
CCA w/o maskModel=Gemma-2-2B-Instruct, Training Dataset=Branching, Dataset Size=12442025.1253.20.00912.1
CCA w maskModel=Gemma-2-2B-Instruct, Training Dataset=Branching, Dataset Size=1244, Mask %=1.59%2025.1252.50.0111.4
CCA w/o maskModel=Gemma-2-2B-Instruct, Training Dataset=Prefix, Dataset Size=12832025.1250.20.0189.1
CCA w maskModel=Gemma-2-2B-Instruct, Training Dataset=Prefix, Dataset Size=1283, Mask %=0.92%2025.12440.0112.9
Original ModelModel=Gemma-2-2B-Instruct2025.1241.1——