Ambiguity Detection on AmbigQA
79.5F1 ScoreH0 topology
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| H0 topologyModel=Gemma-7B-it2026.06 | 79.5 | — | — | 79.5 | |
| H0 topologyModel=Llama-3.1-8B-Instruct2026.06 | 79.3 | — | — | 79.3 | |
| H0 topologyModel=Mistral-7B-Instruct-v0.32026.06 | 77.8 | — | — | 77.9 | |
| AENModel=Mistral-7B-Instruct-v0.32026.06 | 77.3 | — | — | 77.3 | |
| ShaQ (Max Span)Backbone=GPT-42026.05 | 70.38 | 64.2 | 59.95 | — | |
| ShaQBackbone=GPT-42026.05 | 70.38 | 65.77 | 61.52 | — | |
| LOOBackbone=GPT-42026.05 | 67.77 | 50.43 | 54.35 | — | |
| LOO (Max Span)Backbone=GPT-42026.05 | 67.77 | 52.36 | 55.8 | — | |
| ICEBackbone=GPT-42026.05 | 65.54 | 60.66 | 59.98 | — | |
| AENModel=Llama-3.1-8B-Instruct2026.06 | 63.2 | — | — | 63.4 | |
| AENModel=Gemma-7B-it2026.06 | 61.6 | — | — | 61.6 | |
| InfoGainModel=Mistral-7B-Instruct-v0.32026.06 | 59.8 | — | — | 59.8 | |
| Deep EnsemblesBackbone=GPT-42026.05 | 59.7 | 60.29 | 60.31 | — | |
| Few-shotModel=Mistral-7B-Instruct-v0.32026.06 | 58.4 | — | — | 58.4 | |
| Sample RepetitionBackbone=GPT-42026.05 | 56.98 | 60.9 | 59.24 | — | |
| Ask4Conf-DBackbone=GPT-42026.05 | 56.36 | 52.44 | 53.97 | — | |
| Sample DiversityBackbone=GPT-42026.05 | 55.24 | 60.86 | 59.35 | — | |
| Self-ConsistencyBackbone=GPT-42026.05 | 53.4 | 59.33 | 57.59 | — | |
| InfoGainModel=Gemma-7B-it2026.06 | 50.7 | — | — | 52.5 | |
| Zero-shotModel=Llama-3.1-8B-Instruct2026.06 | 45.9 | — | — | 50 | |
| Few-shot CoTModel=Llama-3.1-8B-Instruct2026.06 | 45.8 | — | — | 49.1 | |
| Semantic EntropyBackbone=GPT-42026.05 | 44.58 | 57.16 | 57 | — | |
| Few-shot CoTModel=Mistral-7B-Instruct-v0.32026.06 | 43.1 | — | — | 49.2 | |
| Few-shot CoTModel=Gemma-7B-it2026.06 | 42.7 | — | — | 42.7 | |
| Zero-shotModel=Gemma-7B-it2026.06 | 41.8 | — | — | 50.4 | |
| InfoGainModel=Llama-3.1-8B-Instruct2026.06 | 40.9 | — | — | 52.9 | |
| Few-shotModel=Gemma-7B-it2026.06 | 36.3 | — | — | 50.4 | |
| Few-shotModel=Llama-3.1-8B-Instruct2026.06 | 35.5 | — | — | 52.5 | |
| Zero-shotModel=Mistral-7B-Instruct-v0.32026.06 | 34 | — | — | 49.4 |