Span-level uncertainty estimation on SPANUQ-BENCH v1.0 (test)
0.944AUROCSPANUQ
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SPANUQBackbone=Qwen3-4B2026.07 | 0.944 | 0.126 | 0.754 | |
| SPANUQBackbone=Qwen3-14B2026.07 | 0.939 | 0.11 | 0.685 | |
| SPANUQBackbone=Qwen3-30B-A3B2026.07 | 0.936 | 0.11 | 0.647 | |
| SPANUQBackbone=Qwen3-8B2026.07 | 0.93 | 0.129 | 0.692 | |
| SPANUQBackbone=Mistral-7B2026.07 | 0.908 | 0.126 | 0.637 | |
| MLP ProbeBackbone=Qwen3-30B-A3B, Oracle span boundaries=true2026.07 | 0.893 | 0.12 | 0.575 | |
| MLP ProbeBackbone=Qwen3-8B, Oracle span boundaries=true2026.07 | 0.884 | 0.149 | 0.612 | |
| MLP ProbeBackbone=Qwen3-14B, Oracle span boundaries=true2026.07 | 0.881 | 0.139 | 0.575 | |
| MLP ProbeBackbone=Qwen3-4B, Oracle span boundaries=true2026.07 | 0.873 | 0.179 | 0.626 | |
| MLP ProbeBackbone=Mistral-7B, Oracle span boundaries=true2026.07 | 0.863 | 0.146 | 0.562 | |
| SelfCheckGPT-NLIBackbone=Mistral-7B2026.07 | 0.816 | 0.154 | 0.499 | |
| SelfCheckGPT-NLIBackbone=Qwen3-30B-A3B2026.07 | 0.813 | 0.147 | 0.455 | |
| SelfCheckGPT-NLIBackbone=Qwen3-14B2026.07 | 0.808 | 0.158 | 0.471 | |
| SelfCheckGPT-NLIBackbone=Qwen3-8B2026.07 | 0.801 | 0.182 | 0.473 | |
| SelfCheckGPT-NLIBackbone=Qwen3-4B2026.07 | 0.782 | 0.215 | 0.479 | |
| FActScoreBackbone=Qwen3-8B2026.07 | 0.762 | 0.197 | 0.405 | |
| FActScoreBackbone=Qwen3-4B2026.07 | 0.714 | 0.233 | 0.39 | |
| FActScoreBackbone=Qwen3-30B-A3B2026.07 | 0.701 | 0.169 | 0.306 | |
| FActScoreBackbone=Mistral-7B2026.07 | 0.661 | 0.182 | 0.297 | |
| FActScoreBackbone=Qwen3-14B2026.07 | 0.625 | 0.178 | 0.181 | |
| Token EntropyBackbone=Qwen3-14B2026.07 | 0.603 | 0.23 | 0.097 | |
| Verbalized Conf.Backbone=Qwen3-30B-A3B2026.07 | 0.598 | 0.167 | 0.117 | |
| Verbalized Conf.Backbone=Qwen3-4B2026.07 | 0.595 | 0.267 | 0.141 | |
| Verbalized Conf.Backbone=Qwen3-14B2026.07 | 0.593 | 0.187 | 0.131 | |
| Token EntropyBackbone=Qwen3-30B-A3B2026.07 | 0.592 | 0.196 | 0.097 | |
| Token EntropyBackbone=Qwen3-8B2026.07 | 0.58 | 0.226 | 0.111 | |
| Token EntropyBackbone=Qwen3-4B2026.07 | 0.575 | 0.272 | 0.119 | |
| Verbalized Conf.Backbone=Qwen3-8B2026.07 | 0.574 | 0.217 | 0.155 | |
| Token EntropyBackbone=Mistral-7B2026.07 | 0.569 | 0.231 | 0.109 | |
| P(True)Backbone=Qwen3-8B2026.07 | 0.546 | 0.233 | 0.121 | |
| P(True)Backbone=Qwen3-30B-A3B2026.07 | 0.541 | 0.194 | 0.123 | |
| P(True)Backbone=Qwen3-4B2026.07 | 0.529 | 0.273 | 0.122 | |
| P(True)Backbone=Qwen3-14B2026.07 | 0.526 | 0.197 | 0.087 | |
| P(True)Backbone=Mistral-7B2026.07 | 0.505 | 0.199 | 0.005 | |
| Verbalized Conf.Backbone=Mistral-7B2026.07 | 0.498 | 0.171 | -0.026 |