Inference Throughput on NQ for Hallucination Detection
5,224Throughput (Samples/sec)Mean Pooling
Evaluation Results
| Method | Links | |
|---|---|---|
| Mean PoolingModel=LLaMA-3.1-8B2026.05 | 5,224 | |
| Max PoolingModel=LLaMA-3.1-8B2026.05 | 4,992 | |
| HaMI (Original)Model=LLaMA-3.1-8B2026.05 | 4,096 | |
| Max PoolingModel=Mistral-Nemo-Instruct (12B)2026.05 | 3,720 | |
| Mean PoolingModel=Mistral-Nemo-Instruct (12B)2026.05 | 3,706 | |
| HaMI (Original)Model=Mistral-Nemo-Instruct (12B)2026.05 | 2,886 | |
| Mean PoolingModel=LLaMA-3.3-Instruct-70B2026.05 | 2,625 | |
| Max PoolingModel=LLaMA-3.3-Instruct-70B2026.05 | 2,602 | |
| HaMI (Original)Model=LLaMA-3.3-Instruct-70B2026.05 | 2,315 | |
| HaMI (SP)Model=Mistral-Nemo-Instruct (12B)2026.05 | 0.22 | |
| HaMI (SP)Model=LLaMA-3.3-Instruct-70B2026.05 | 0.18 | |
| HaMI (SP)Model=LLaMA-3.1-8B2026.05 | 0.1 |