Multi-label compliance evaluation on 53-example (blind test)
2Latency (s)LLaMA 3.1 8B (LoRA fine-tuned)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LLaMA 3.1 8B (LoRA fine-tuned)Hardware=A100, Data On-Prem=true, Fine-tuning=LoRA (2.05% trainable), Examples=219 examples2026.06 | 2 | 83 | 100 | 0.013 | 4,020 | |
| GPT-4o APIData On-Prem=false, Deployment=API2026.06 | 4 | — | — | 0.025 | 7,500 | |
| Claude Sonnet APIData On-Prem=false, Deployment=API2026.06 | 6 | — | — | 0.033 | 9,900 | |
| Claude Opus APIData On-Prem=false, Deployment=API2026.06 | 10 | — | — | 0.055 | 16,500 | |
| Few-Shot PromptingData On-Prem=false, Context window=250K token context2026.06 | 30 | — | 85 | 0.65 | 195 |