Question Answering on QA benchmarks
54.6TTFT (ms)Quantized (4-bit)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Quantized (4-bit)Model Pair=Mistral Pair (7B)2026.06 | 54.6 | 4.34 | 15.1 | 7.16 | |
| Raw KV (BF16)Model Pair=Mistral Pair (7B)2026.06 | 61.3 | 3.87 | 19.2 | 4.68 | |
| Quantized (4-bit)Model Pair=Qwen Pair (32B)2026.06 | 115.3 | 4.37 | 21.2 | 10.27 | |
| SCDModel Pair=Mistral Pair (7B)2026.06 | 119.7 | 1.98 | 76.9 | 0.26 | |
| Raw KV (BF16)Model Pair=Qwen Pair (32B)2026.06 | 128.7 | 3.91 | 28.3 | 8.16 | |
| DroidSpeak-styleModel Pair=Mistral Pair (7B)2026.06 | 139.4 | 1.7 | 75.4 | 0.31 | |
| OracleModel Pair=Mistral Pair (7B)2026.06 | 237.1 | 1 | 80.7 | 0 | |
| SCDModel Pair=Qwen Pair (32B)2026.06 | 243.7 | 2.07 | 84.7 | 0.31 | |
| DroidSpeak-styleModel Pair=Qwen Pair (32B)2026.06 | 278.4 | 1.81 | 81.3 | 0.49 | |
| OracleModel Pair=Qwen Pair (32B)2026.06 | 503.6 | 1 | 87.6 | 0 |