Message Parsing on CBRS (bn)
0.0002Average CostLLaMA-3.2-3B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LLaMA-3.2-3Bmode=zero-shot2026.04 | 0.0002 | 2,663.64 | 93.97 | 2,757.61 | 1.65 | |
| LLaMA-3.2-3Bmode=zero-shot2026.04 | 0.0002 | 2,919.22 | 315.24 | 3,234.46 | 3.36 | |
| Gemini-2.0mode=zero-shot2026.04 | 0.0003 | 2,740.46 | 131.37 | 2,871.83 | 2.5 | |
| GPT-4o-minimode=zero-shot2026.04 | 0.0004 | 2,216.08 | 106.26 | 2,322.33 | 3.11 | |
| GPT-4o-minimode=zero-shot2026.04 | 0.0004 | 2,284.57 | 147.36 | 2,431.93 | 3.9 | |
| Gemini-2.0mode=zero-shot2026.04 | 0.0004 | 2,845.1 | 388.46 | 3,233.56 | 3.87 | |
| LLaMA-3.1-8Bmode=zero-shot2026.04 | 0.0005 | 2,805.85 | 130.25 | 2,936.09 | 1.02 | |
| LLaMA-3.1-8Bmode=zero-shot2026.04 | 0.0006 | 3,040.99 | 481.06 | 3,522.05 | 2.41 | |
| Mistral-7Bmode=zero-shot2026.04 | 0.0007 | 3,307.23 | 163.34 | 3,470.57 | 2.24 | |
| Mistral-7Bmode=zero-shot2026.04 | 0.0008 | 3,524.8 | 215.26 | 3,740.06 | 2.67 | |
| Qwen-2.5-7Bmode=zero-shot2026.04 | 0.0009 | 2,908.61 | 125.06 | 3,033.67 | 1.57 | |
| Claude-3-Haikumode=zero-shot2026.04 | 0.0009 | 2,818.61 | 191.32 | 3,009.93 | 2.49 | |
| Qwen-2.5-7Bmode=zero-shot2026.04 | 0.001 | 3,113.89 | 212.39 | 3,326.28 | 2.27 | |
| Claude-3-Haikumode=zero-shot2026.04 | 0.0011 | 2,991.29 | 258.67 | 3,249.96 | 2.89 | |
| Gemma-2-27Bmode=zero-shot2026.04 | 0.0023 | 2,752.46 | 127.94 | 2,880.4 | 2.95 | |
| Gemma-2-27Bmode=zero-shot2026.04 | 0.0025 | 2,857.1 | 279.11 | 3,136.21 | 5.42 | |
| LLaMA-3.3-70Bmode=zero-shot2026.04 | 0.0026 | 2,807.41 | 110.11 | 2,917.52 | 3.28 | |
| LLaMA-3.3-70Bmode=zero-shot2026.04 | 0.0029 | 3,042.21 | 236.08 | 3,278.28 | 4.86 |