Conflict Resolution on CALCONFLICTBENCH 1.0 (test) benchmark leaderboard · SOTA2 Research

Conflict Resolution on CALCONFLICTBENCH 1.0 (test)

0.3Avg Error Rate (N=1)Qwen3-8B

Evaluation Results

Method	Links
Qwen3-8B2026.01		0.3	0.38	0.36	0.37	0.37	0.76	0.78	0.79	0.79	0.79	0.026
GPT-5-nano2026.01		0.3	0.42	0.41	0.43	0.41	0.85	0.77	0.78	0.77	0.78	0.122
Gemini-2.5-flash2026.01		0.3	0.4	0.39	0.4	0.38	0.84	0.79	0.79	0.79	0.81	0.088
Qwen3-30B2026.01		0.34	0.39	0.39	0.39	0.38	0.79	0.79	0.79	0.78	0.78	0.069
ReAct2026.01		0.34	0.4	0.39	0.39	0.39	0.78	0.78	0.79	0.79	0.8	0.007
Qwen3-30B-Think2026.01		0.36	0.38	0.34	0.36	0.35	0.8	0.79	0.81	0.81	0.82	0.161
Mem+ReAct2026.01		0.36	0.37	0.39	0.39	0.4	0.84	0.81	0.81	0.8	0.79	-0.162
Qwen3-14B2026.01		0.38	0.42	0.41	0.4	0.41	0.82	0.75	0.75	0.74	0.75	-0.039
OLMo3-32B-Think2026.01		0.4	0.45	0.46	0.46	0.45	0.72	0.72	0.72	0.72	0.72	0.05
GPT-52026.01		0.42	0.39	0.36	0.36	0.35	0.83	0.81	0.82	0.82	0.83	0.092
Qwen3-4B2026.01		0.44	0.46	0.44	0.45	0.45	0.73	0.73	0.75	0.75	0.76	-0.029
LLaMA-3.1-8B2026.01		0.66	0.66	0.67	0.65	0.65	0.58	0.58	0.6	0.61	0.62	-0.027
OLMo3-7B-Instruct2026.01		0.98	1	1	1	1	0.01	0	0	0	0	-0.004