Time to First Token (TTFT) on GSM8K
19.99TTFT (ms)Behavior-Equivalent Token
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Behavior-Equivalent TokenBackbone=Llama3.2-1B, Prompt=[BE] token + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 19.99 | -7.72 | -27.9 | |
| Full PromptBackbone=Llama3.2-1B, Prompt=Sys prompt + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 27.71 | — | — | |
| Behavior-Equivalent TokenBackbone=Llama3.2-3B, Prompt=[BE] token + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 32.56 | -34.58 | -51.5 | |
| Behavior-Equivalent TokenBackbone=Llama3.1-8B, Prompt=[BE] token + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 40.36 | -49.31 | -55 | |
| Behavior-Equivalent TokenBackbone=Qwen3-4B, Prompt=[BE] token + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 47.9 | -70.17 | -59.4 | |
| Full PromptBackbone=Llama3.2-3B, Prompt=Sys prompt + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 67.14 | — | — | |
| Full PromptBackbone=Llama3.1-8B, Prompt=Sys prompt + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 89.67 | — | — | |
| Full PromptBackbone=Qwen3-4B, Prompt=Sys prompt + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 118.07 | — | — |