Instruction Following on MulDimIF (test) benchmark leaderboard · SOTA2 Research

Instruction Following on MulDimIF (test)

73.5Example ScoreGemini1.5 Pro

Evaluation Results

Method
Gemini1.5 Pro2025.05	73.5	61.75	65.25	87.4	81.81	88.28	75.67	86.67	70	59	51.67	66.83
Claude3.5 Sonnet2025.05	72.5	69	61	86.62	83.44	84.23	76.21	82.67	71.67	60.67	55	67.5
Qwen3 14B-Reason.2025.05	71.5	64	57.75	87.79	77.29	92.47	72.31	86	72.33	58.67	40.67	64.42
Gemini1.5 Flash2025.05	71.5	61	64.5	88.7	80.68	87.84	74.06	86	68	57.67	51	65.67
GPT 4-Turbo2025.05	70.75	59.25	52.25	89.09	79.17	77.57	73.25	82.67	68	55	37.33	60.75
Qwen3 32B-Reason.2025.05	70.5	69.5	59.5	87.14	83.06	90.3	74.19	86.33	72.67	61	46	66.5
GPT 4o2025.05	70.5	62.5	59	87.92	84.44	82.78	69.35	84.33	69.33	57.33	45	64
Qwen3 32B-Direct.2025.05	70	59	49	87.14	78.29	78.29	68.01	85	65.67	51	34	58.92
LLaMA3.1 Instruct-70B2025.05	68	54.25	48.25	86.62	73.65	80.9	65.73	78	61.33	51.33	36.67	56.83
Qwen3 8B-Reason.2025.05	67.75	61.25	54	85.84	71.02	87.7	72.58	87	68.33	52.67	36	61
GPT 4o-Mini2025.05	67.5	67	59	85.71	84.57	84.37	72.04	84.33	70	59	44.67	64.5
Qwen3 14B-Direct.2025.05	64	63	52	87.92	77	89.87	65.86	88	66.67	50.67	33.67	60
Qwen3 8B-Direct.2025.05	63	55	51	84.03	72.27	83	69.89	87.67	60.33	45.67	31	56.17
DeepSeek-R1-Distill-LLaMA Instruct-70B2025.05	59.5	64	57.5	84.55	83.69	84.8	65.32	77	63	57	44.33	60.33
GPT 3.5-Turbo2025.05	49	41.25	38	84.94	70.39	42.98	66.26	77.33	48	30.33	15.33	42.75
Claude3.5 Haiku2025.05	44.25	53.5	52	84.29	82.81	49.06	68.01	71.33	51.67	41.67	35	49.92
DeepSeek-R1-Distill-LLaMA Instruct-8B2025.05	42	28	28	80	50.44	44.72	53.09	59.67	40.33	18.33	12.33	32.67
LLaMA3.1 Instruct-8B2025.05	40.25	36	32.25	80.13	64.74	44.28	49.6	64.67	40.67	27.33	12	36.17