Time to First Token (TTFT) on RoleLLM
19.17TTFT (ms)Behavior-Equivalent Token
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Behavior-Equivalent TokenBackbone=Llama3.2-1B, Prompt=[BE] token + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 19.17 | -4.72 | -19.8 | |
| Full PromptBackbone=Llama3.2-1B, Prompt=Sys prompt + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 23.89 | — | — | |
| Behavior-Equivalent TokenBackbone=Llama3.2-3B, Prompt=[BE] token + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 32.14 | -4.47 | -12.2 | |
| Full PromptBackbone=Llama3.2-3B, Prompt=Sys prompt + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 36.61 | — | — | |
| Behavior-Equivalent TokenBackbone=Qwen3-4B, Prompt=[BE] token + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 38.51 | -11.4 | -22.8 | |
| Behavior-Equivalent TokenBackbone=Llama3.1-8B, Prompt=[BE] token + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 40.25 | -4.27 | -9.6 | |
| Full PromptBackbone=Llama3.1-8B, Prompt=Sys prompt + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 44.52 | — | — | |
| Full PromptBackbone=Qwen3-4B, Prompt=Sys prompt + query, Hardware=NVIDIA A100 GPU, Attention Implementation=FlashAttention v2.7.4, Precision=bfloat162025.11 | 49.91 | — | — |