Inference Efficiency on 1024-token sequences (inference summary)
499.49Throughput (Samples/s)Opir-edge
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Opir-edgeBackend=gliclass, Input Tokens=10242026.05 | 499.49 | 9.25 | 9.52 | |
| Opir-edge-multilangBackend=gliclass, Input Tokens=10242026.05 | 306.81 | 15.6 | 15.69 | |
| Opir-multitask-multilangBackend=gliclass, Input Tokens=10242026.05 | 123.67 | 13.3 | 14.03 | |
| PolyGuard-Qwen-SmolBackend=vllm, Input Tokens=10242026.05 | 81.48 | 71.77 | 73.46 | |
| Qwen3Guard-Gen-8BBackend=vllm, Input Tokens=10242026.05 | 65.45 | 91.3 | 91.8 | |
| Llama-3.1-Nemotron-Safety-Guard-8B v3Backend=vllm, Input Tokens=10242026.05 | 62.19 | 97.63 | 98.31 | |
| Opir-multitask-largeBackend=gliclass, Input Tokens=10242026.05 | 50.51 | 25.65 | 26.09 | |
| GLiGuard-LLMGuardrails-300MBackend=gliner2, Input Tokens=10242026.05 | 42.98 | 28.99 | 30.09 | |
| Gliner-Guard-OmniBackend=gliner2, Input Tokens=10242026.05 | 34.49 | 34.04 | 34.58 | |
| WildGuardBackend=vllm, Input Tokens=10242026.05 | 28.79 | 243 | 243.86 | |
| PolyGuard-QwenBackend=vllm, Input Tokens=10242026.05 | 23.51 | 308.59 | 309.86 |