Representation Alignment on Qwen3-4B-Thinking Internal Layers 2507
1Average CKABF16 (Teacher)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BF16 (Teacher)Model=Qwen3-4B-Thinking-2507, Precision=BF162026.06 | 1 | 1 | |
| NVFP4 PTQModel=Qwen3-4B-Thinking-2507, Precision=NVFP42026.06 | 1 | 0.99 | |
| CKA-QADModel=Qwen3-4B-Thinking-2507, Precision=NVFP42026.06 | 0.99 | 0.97 | |
| QAD (KL-only)Model=Qwen3-4B-Thinking-2507, Precision=NVFP42026.06 | 0.98 | 0.86 |