Instruction Fine-tuning on MetaMathQA Suite (ARC-C, PIQA, MMLU, HE, GSM8K) (test)

51.45ARC-C AccuracyPre-trained

Evaluation results

Method
Pre-trained2026.05	51.45	79.16	59.61	35.37	56.4	37	46.7
DualSFT2026.05	51.38	79.03	59.61	35.45	56.37	66.58	61.47
DualSFT-Data2026.05	51.32	78.61	59.44	35.12	56.12	66.94	61.53
Random2026.05	51.26	79.03	59.37	35.11	56.19	61.59	58.89
DualSFT-Param2026.05	51.06	78.88	59.79	34.98	56.18	67.14	61.66
LESS + S2FT2026.05	50.91	78.54	56.48	31.54	54.37	63.85	59.11
NanoAdam2026.05	50.67	78.16	58.25	33.57	55.16	65.89	60.53
SPICE2026.05	50.33	78.42	57.52	33.94	55.05	64.73	59.89
LIFT2026.05	50.14	76.59	55.21	30.36	53.08	64.73	58.9
SMT2026.05	49.82	77.29	56.17	31.94	53.81	65.51	59.66
FisherSFT2026.05	49.81	77.61	56.49	32.49	54.1	63.81	58.96
SPICE + NanoAdam2026.05	49.73	78.66	58.21	33.69	55.07	63.47	59.27
S2FT2026.05	49.51	77.68	55.73	31.22	53.54	64.34	58.94
LoRA2026.05	48.91	77.94	55.8	30.47	53.28	62.48	57.88
LESS2026.05	48.65	77.53	56.07	30.94	53.3	65.28	59.29
Standard SFT2026.05	43.36	74.48	53	29.55	50.1	67.79	58.94
Pre-trained2026.05	43	76.61	56.52	28.66	51.2	27.45	39.32
DualSFT-Data2026.05	42.88	76.13	56.03	28.57	50.9	65.12	58.01
DualSFT2026.05	42.88	76.42	56.34	29.88	51.38	64.51	57.95
NanoAdam2026.05	42.55	74.51	53.74	26.61	49.35	65.17	57.26
SPICE + NanoAdam2026.05	42.47	76.02	54.83	26.71	50.01	62.64	56.32
DualSFT-Param2026.05	42.43	75.82	55.96	28.01	50.56	65.84	58.2
Random2026.05	42.19	76.22	55.92	28.66	50.75	59.67	55.21
LESS + S2FT2026.05	41.75	75.36	53.67	25.69	49.12	61.53	55.32
LESS2026.05	41.52	74.66	53.58	26.35	49.03	63.57	56.3
SPICE2026.05	41.26	74.89	55.42	27.29	49.72	62.36	56.04
SMT2026.05	41.22	72.33	52.74	26.13	48.11	65.06	56.58
LIFT2026.05	40.94	73.58	52.61	25.07	48.05	64.72	56.39
S2FT2026.05	40.78	75.96	52.88	25.34	48.74	64.94	56.84
FisherSFT2026.05	40.69	74.25	55.12	27.04	49.28	61.69	55.48
LoRA2026.05	40.61	74.21	52.11	25.05	48	61.08	54.54
Standard SFT2026.05	36.6	74.48	48.51	14.98	43.64	65.96	54.8