Multi-step LLM Interaction on Qwen3-4B Inference Performance (test)
100High-Precision RatioBF16
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BF16Precision=BF162026.02 | 100 | 93.3 | 15.7 | |
| Random@80%Sampling Ratio=80%2026.02 | 80 | 90.3 | 15.88 | |
| Random@60%Sampling Ratio=60%2026.02 | 60 | 88.8 | 18.67 | |
| Random@40%Sampling Ratio=40%2026.02 | 40 | 81.1 | 8.75 | |
| Random@20%Sampling Ratio=20%2026.02 | 20 | 78.1 | 2.5 | |
| Router(Ours)Routing=Dynamic Mix Precision2026.02 | 8.6 | 81.3 | 43.02 | |
| GPTQ-3BitQuantization=3-bit2026.02 | 0 | 77.6 | — |