Entity counting on Entity counting Qwen3-8B prompts N=200x3 seeds (test)
97Greedy Generation AccuracyLoRA Q/V rank-16 (entity-counting only)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LoRA Q/V rank-16 (entity-counting only)Task scenario=entity-only stability2026.05 | 97 | — | — | |
| Probe-round (oracle UB)Note=injects probe at each step in generation, Seeds=32026.05 | 96 | 98.7 | — | |
| LoRA Q/V rank-16Rank=16, Trainable parameters=7.67M, Seeds=32026.05 | 83.1 | 96 | 91.7 | |
| Hard DPSProtocol=Diagnostic Probe Steering (DPS), Type=hard bypass, Seeds=32026.05 | 72.4 | 98.7 | — | |
| LoRA Q/V rank-16 (per-seed, multi-task)Task scenario=multi-task: entity+char+add, Metric variation=task-mix-driven variance2026.05 | 71.5 | — | — | |
| Baseline (entity counting)Intervention=no intervention, Seeds=3 (42, 11, 77)2026.05 | 7.2 | 13.7 | 0 | |
| 9-row lm_head repairParameters rewritten=36,864, Seeds=32026.05 | 0 | 60.7 | 60.3 | |
| Norm rescaling (digit ×3)Scaling Factor=32026.05 | — | — | 26.5 | |
| Soft DPSProtocol=Diagnostic Probe Steering (DPS), Seeds=32026.05 | — | 13.2 | — |