LLM Inference Efficiency on Sequence prompts Medium
0.37TTFT (s)FedRAG
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FedRAGModel=Qwen/Qwen3-4B-Instruct-2507, Protected Layers=22026.05 | 0.37 | 36.51 | 619.11 | 139 | |
| FedRAGModel=meta-llama/Llama-3.1-8B-Instruct, Protected Layers=22026.05 | 0.42 | 46.99 | 555.65 | 139 | |
| FedRAGModel=mistralai/Ministral-3-14B-Instruct-2512-BF16, Protected Layers=22026.05 | 0.43 | 33.05 | 683.78 | 139 | |
| FedRAGModel=openai/gpt-oss-20b, Protected Layers=22026.05 | 0.44 | 20.66 | 160.84 | 139 | |
| FedRAGModel=openai/gpt-oss-20b, Protected Layers=full2026.05 | 2.45 | 10.28 | 804.73 | 1,635 | |
| FedRAGModel=mistralai/Ministral-3-14B-Instruct-2512-BF16, Protected Layers=full2026.05 | 3.51 | 9.23 | 1,316.92 | 2,707 | |
| FedRAGModel=meta-llama/Llama-3.1-8B-Instruct, Protected Layers=full2026.05 | 4.83 | 11.79 | 1,107.61 | 2,171 | |
| FedRAGModel=Qwen/Qwen3-4B-Instruct-2507, Protected Layers=full2026.05 | 5.39 | 9.96 | 1,210.85 | 2,439 | |
| PermLLMModel=meta-llama/Llama-3.1-8B-Instruct, Protected Layers=22026.05 | 5.93 | 16.94 | 3,592.9 | 1,323 | |
| PermLLMModel=Qwen/Qwen3-4B-Instruct-2507, Protected Layers=22026.05 | 6.02 | 15.32 | 3,650.17 | 1,323 | |
| PermLLMModel=mistralai/Ministral-3-14B-Instruct-2512-BF16, Protected Layers=22026.05 | 6.07 | 14.62 | 3,721.02 | 1,323 | |
| PermLLMModel=openai/gpt-oss-20b, Protected Layers=22026.05 | 6.21 | 12.56 | 3,295.09 | 1,323 | |
| SCXModel=openai/gpt-oss-20b, Protected Layers=22026.05 | 9.29 | 17.9 | 3,643.9 | 676 | |
| SCXModel=openai/gpt-oss-20b, Protected Layers=full2026.05 | 10.13 | 5.98 | 4,454.75 | 4,922 | |
| SCXModel=Qwen/Qwen3-4B-Instruct-2507, Protected Layers=22026.05 | 14.44 | 25.84 | 5,494.15 | 820 | |
| SCXModel=meta-llama/Llama-3.1-8B-Instruct, Protected Layers=full2026.05 | 15.09 | 5.48 | 6,752.25 | 6,530 | |
| SCXModel=Qwen/Qwen3-4B-Instruct-2507, Protected Layers=full2026.05 | 15.2 | 4.84 | 6,345.42 | 7,334 | |
| SCXModel=meta-llama/Llama-3.1-8B-Instruct, Protected Layers=22026.05 | 17.44 | 28.67 | 5,891.63 | 772 | |
| SCXModel=mistralai/Ministral-3-14B-Instruct-2512-BF16, Protected Layers=22026.05 | 19.91 | 24.56 | 8,172.84 | 868 | |
| SCXModel=mistralai/Ministral-3-14B-Instruct-2512-BF16, Protected Layers=full2026.05 | 21.65 | 4.18 | 9,083.67 | 8,138 | |
| PermLLMModel=openai/gpt-oss-20b, Protected Layers=full2026.05 | 73.47 | 1.62 | 38,419.13 | 15,843 | |
| PermLLMModel=meta-llama/Llama-3.1-8B-Instruct, Protected Layers=full2026.05 | 89.23 | 1.46 | 45,815.71 | 21,091 | |
| PermLLMModel=Qwen/Qwen3-4B-Instruct-2507, Protected Layers=full2026.05 | 92.52 | 1.28 | 49,936.56 | 23,715 | |
| PermLLMModel=mistralai/Ministral-3-14B-Instruct-2512-BF16, Protected Layers=full2026.05 | 106.02 | 1.52 | 53,448.83 | 20,139 |