Policy-trajectory compliance evaluation on POLICYGUARDBENCH
88.83F1 ScoreLLAMA-3.3-70B-INSTRUCT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LLAMA-3.3-70B-INSTRUCTSIZE=70B, LATENCY=305.0, FLOPS=45.732025.10 | 88.83 | 2.9125 | |
| GEMMA-3-12B-ITSIZE=12B, LATENCY=51.3, FLOPS=7.872025.10 | 87.73 | 17.1014 | |
| POLICYGUARD-4BSIZE=4B, LATENCY=22.5, FLOPS=2.572025.10 | 87.59 | 38.9289 | |
| QWEN3-235B-A22B-INSTRUCT-2507SIZE=235B, LATENCY=3640.0, FLOPS=13.802025.10 | 86.9 | 0.2387 | |
| CLAUDE-SONNET-4LATENCY=1238.02025.10 | 86.78 | 0.701 | |
| QWEN2.5-72B-INSTRUCTSIZE=72B, LATENCY=205.0, FLOPS=48.022025.10 | 86.07 | 4.1985 | |
| GEMMA-3-27B-ITSIZE=27B, LATENCY=73.6, FLOPS=17.992025.10 | 85.2 | 11.5761 | |
| GEMINI-1.5-PROLATENCY=596.12025.10 | 85.02 | 1.4263 | |
| DEEPSEEK-V3.1 (NON-THINKING)SIZE=685B, LATENCY=3270.02025.10 | 84.07 | 0.2571 | |
| LLAMA-4-SCOUT-17B-16E-INSTRUCTSIZE=109B, LATENCY=265.0, FLOPS=63.922025.10 | 81.98 | 3.0936 |