Message Parsing on CBRS (total)
3,396.22Average Input TokensMistral-7B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Mistral-7Bmode=zero-shot2026.04 | 3,396.22 | 0.0007 | 184.25 | 3,580.46 | 2.4 | |
| Qwen-2.5-7Bmode=zero-shot2026.04 | 2,992.45 | 0.001 | 164.97 | 3,157.41 | 1.88 | |
| LLaMA-3.3-70Bmode=zero-shot2026.04 | 2,902.88 | 0.0027 | 164.21 | 3,067.09 | 4 | |
| LLaMA-3.1-8Bmode=zero-shot2026.04 | 2,901.48 | 0.0006 | 267.19 | 3,168.67 | 1.63 | |
| Claude-3-Haikumode=zero-shot2026.04 | 2,889.91 | 0.001 | 220.56 | 3,110.48 | 2.65 | |
| Gemma-2-27Bmode=zero-shot2026.04 | 2,795.67 | 0.0024 | 191.66 | 2,987.33 | 4.06 | |
| Gemini-2.0mode=zero-shot2026.04 | 2,783.67 | 0.0004 | 236.2 | 3,019.87 | 3.07 | |
| LLaMA-3.2-3Bmode=zero-shot2026.04 | 2,772.65 | 0.0002 | 173.62 | 2,946.27 | 2.27 | |
| GPT-4o-minimode=zero-shot2026.04 | 2,245.31 | 0.0004 | 125.55 | 2,370.86 | 3.47 | |
| LoRA finetuned LLama-3.2-3Bmode=LoRA-finetuned2026.04 | 78.5 | — | 189.08 | 267.58 | 1.35 |