Classification on Analysis N-shot benchmark leaderboard · SOTA2 Research

Classification on Analysis N-shot

88AccuracyCRISTAL

Evaluation Results

Method	Links
CRISTAL2026.06		88	0.81
CRISTAL2026.06		86	0.79
CRISTAL2026.06		84	0.75
CRISTAL2026.06		83	0.74
CRISTAL2026.06		79	0.67
GPT-4o2026.06		66	0.49
GPT-4o2026.06		58	0.41
GPT-4o2026.06		57	0.39
CRISTAL2026.06		55	0.29
Deepseek-R1 70B2026.06		43	0.25
Mistral Nemo 32026.06		41	0.02
Mistral Nemo 32026.06		41	0.02
Qwen2.5-Coder 32B2026.06		41	0.03
Qwen2.5-Coder 32B2026.06		39	-0.01
Mistral Nemo 32026.06		39	0
Mistral Nemo 32026.06		38	-0.02
Mistral Nemo 32026.06		37	-0.06
Mistral Nemo 32026.06		37	-0.04
Llama 3.1 8B2026.06		36	0.01
Llama 3.1 8B2026.06		35	0
Deepseek-R1 14B2026.06		35	0.05
Qwen2.5-Coder 32B2026.06		35	-0.05
Llama 3.1 8B2026.06		35	-0.01
Llama 3.1 8B2026.06		35	0
Llama 3.1 8B2026.06		34	-0.01
Deepseek-R1 70B2026.06		34	0.12
Deepseek-R1 70B2026.06		33	0.11
Deepseek-R1 14B2026.06		32	0.02
Deepseek-R1 70B2026.06		31	0.06
Llama 3 70B2026.06		31	0
Llama 3.1 8B2026.06		31	-0.07
Llama 3 70B2026.06		30	0.01
Mistral Small 32026.06		30	0.03
Qwen2 72B2026.06		30	0.02
Llama 3 70B2026.06		30	0.01
Deepseek-R1 14B2026.06		29	-0.05
Mistral Small 32026.06		29	0.02
Qwen2 72B2026.06		29	0.01
Qwen2 72B2026.06		29	-0.01
Mistral Small 32026.06		29	0.02
Deepseek-R1 70B2026.06		28	0.05
Llama 3 70B2026.06		28	-0.03
Mistral Small 32026.06		28	-0.01
Mistral Small 32026.06		28	-0.01
Deepseek-R1 70B2026.06		27	0.01
Llama 3 70B2026.06		26	-0.05
Mistral Small 32026.06		26	-0.02
Qwen2 72B2026.06		26	-0.07
Llama 3 70B2026.06		25	-0.03