Entity counting on Entity counting 200 prompts Qwen3-8B (test)
98.7AccuracyHard DPS
Evaluation results
| Method | Links | |
|---|---|---|
| Hard DPSIntervention type=Hard DPS, Argmax restriction=digit-restricted, Number of seeds=32026.05 | 98.7 | |
| Probe-roundIntervention type=Probe-round, Argmax restriction=digit-restricted, Number of seeds=32026.05 | 98.7 | |
| 9-row repairIntervention type=9-row repair, Argmax restriction=digit-restricted, Number of seeds=3, Training protocol=9-row repair2026.05 | 60.7 | |
| Fullvocab repairIntervention type=Fullvocab repair, Argmax restriction=full-vocabulary, Number of seeds=3, Training protocol=optimized training2026.05 | 60.3 | |
| BaselineIntervention type=None, Argmax restriction=digit-restricted, Number of seeds=32026.05 | 13.7 |