Multi-step LLM Interaction on Qwen3-1.7B Inference Performance (test)
100High-Precision RatioBF16
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BF16Precision=BF162026.02 | 100 | 69.4 | 14.9 | |
| Random@80%Sampling Ratio=80%2026.02 | 80 | 65.7 | 14 | |
| Random@60%Sampling Ratio=60%2026.02 | 60 | 61.9 | 12.33 | |
| Random@40%Sampling Ratio=40%2026.02 | 40 | 60.5 | 15 | |
| Router(Ours)Routing=Dynamic Mix Precision2026.02 | 20.5 | 59 | 21.95 | |
| Random@20%Sampling Ratio=20%2026.02 | 20 | 56.2 | 8.5 | |
| GPTQ-Int4Quantization=4-bit2026.02 | 0 | 54.5 | — |