Message Parsing on CBRS (bn) benchmark leaderboard · SOTA2 Research

Message Parsing on CBRS (bn)

0.0002Average CostLLaMA-3.2-3B

Evaluation Results

Method	Links
LLaMA-3.2-3B2026.04		0.0002	2,663.64	93.97	2,757.61	1.65
LLaMA-3.2-3B2026.04		0.0002	2,919.22	315.24	3,234.46	3.36
Gemini-2.02026.04		0.0003	2,740.46	131.37	2,871.83	2.5
GPT-4o-mini2026.04		0.0004	2,216.08	106.26	2,322.33	3.11
GPT-4o-mini2026.04		0.0004	2,284.57	147.36	2,431.93	3.9
Gemini-2.02026.04		0.0004	2,845.1	388.46	3,233.56	3.87
LLaMA-3.1-8B2026.04		0.0005	2,805.85	130.25	2,936.09	1.02
LLaMA-3.1-8B2026.04		0.0006	3,040.99	481.06	3,522.05	2.41
Mistral-7B2026.04		0.0007	3,307.23	163.34	3,470.57	2.24
Mistral-7B2026.04		0.0008	3,524.8	215.26	3,740.06	2.67
Qwen-2.5-7B2026.04		0.0009	2,908.61	125.06	3,033.67	1.57
Claude-3-Haiku2026.04		0.0009	2,818.61	191.32	3,009.93	2.49
Qwen-2.5-7B2026.04		0.001	3,113.89	212.39	3,326.28	2.27
Claude-3-Haiku2026.04		0.0011	2,991.29	258.67	3,249.96	2.89
Gemma-2-27B2026.04		0.0023	2,752.46	127.94	2,880.4	2.95
Gemma-2-27B2026.04		0.0025	2,857.1	279.11	3,136.21	5.42
LLaMA-3.3-70B2026.04		0.0026	2,807.41	110.11	2,917.52	3.28
LLaMA-3.3-70B2026.04		0.0029	3,042.21	236.08	3,278.28	4.86