Compliance Classification on Compliance domain (test)
100AccuracyOracle
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| OracleRuntime (ms)=0.0032026.03 | 100 | 100 | 100 | 0 | 0 | 0 | |
| Groq-qwen3-32bRuntime (ms)=3008.6, Evaluation context=50 test samples due to API cost2026.03 | 98 | 98 | 98 | — | — | 79.7 | |
| Groq-kimi-k2Runtime (ms)=764.2, Evaluation context=50 test samples due to API cost2026.03 | 98 | 98 | 98 | — | — | 80.5 | |
| LPF-SPNRuntime (ms)=14.82026.03 | 97.8 | 97.2 | 97.8 | 0.125 | 0.015 | 1.4 | |
| BERTRuntime (ms)=134.72026.03 | 97 | 95.2 | 97 | 0.069 | 0.011 | 3.2 | |
| Groq-llama-3.3-70bRuntime (ms)=1578.7, Evaluation context=50 test samples due to API cost2026.03 | 95.9 | 95.9 | 95.9 | — | — | 81.6 | |
| VAE-OnlyRuntime (ms)=6.92026.03 | 95.6 | 94.8 | 95.5 | 0.265 | 0.039 | 9.6 | |
| SPN-OnlyRuntime (ms)=2.42026.03 | 94.8 | 92.5 | 94.6 | 0.471 | 0.075 | 30.9 | |
| Groq-gpt-oss-120bRuntime (ms)=1541.7, Evaluation context=50 test samples due to API cost2026.03 | 93.9 | 93.9 | 93.9 | — | — | 81.3 | |
| LPF-LearnedRuntime (ms)=37.42026.03 | 91.1 | 90.5 | 90.7 | 0.273 | 0.04 | 6.6 | |
| EDL-AggregatedRuntime (ms)=1.12026.03 | 43 | 44 | 32.6 | 0.698 | 0.137 | 21.4 | |
| EDL-IndividualRuntime (ms)=3.82026.03 | 28.1 | 14.6 | 12.4 | 1.87 | 0.32 | 18.6 | |
| R-GCNRuntime (ms)=0.0012026.03 | 15.6 | 9 | 4.2 | 1.099 | 0.222 | 17.8 |