Hallucination Detection on FLEURS S2TT (test)
89.66Hallucination Rate (%)LR (Top 75 features)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LR (Top 75 features)Backbone=Qwen-2-Audio, Training Data Source=ASR, Number of Features=300, k=10%2026.04 | 89.66 | 14 | 9 | 5 | 86 | 15 | 30 | |
| LR (Top 75 features)Backbone=Voxtral-3B, Training data source=ASR DATA, Number of features=300, Feature selection type=Top 75, k-value=10%2026.04 | 83.92 | 20 | 10 | 5 | 92 | 8 | 16 | |
| Attention ScoreBackbone=Voxtral-3B, Training data source=S2TT DATA, k-value=10%2026.04 | 83.57 | 20 | 10 | 5 | 90 | 6 | 2 | |
| RandomBackbone=Qwen-2-Audio, Training Data Source=S2TT, k=10%2026.04 | 50.99 | 49 | 9 | 5 | 50 | — | — | |
| RandomBackbone=Voxtral-3B, Training data source=S2TT DATA, k-value=10%2026.04 | 49.04 | 51 | 9 | 5 | 51 | — | — | |
| Attention ScoreBackbone=Qwen-2-Audio, Training Data Source=S2TT, k=10%2026.04 | 14.06 | 84 | 14 | 10 | 27 | 8 | 15 | |
| LR (Top 150 features)Backbone=Voxtral-3B, Training data source=S2TT DATA, Number of features=600, Feature selection type=Top 150, k-value=10%2026.04 | 2.51 | 95 | 37 | 55 | 28 | 44 | 68 | |
| LR (Combined features)Backbone=Voxtral-3B, Training data source=S2TT DATA, Number of features=3840, Feature selection type=Combined, k-value=10%2026.04 | 2.25 | 95 | 37 | 60 | 27 | 43 | 66 | |
| LR (AUDIORATIO features)Backbone=Voxtral-3B, Training data source=S2TT DATA, Number of features=960, Feature selection type=AUDIORATIO, k-value=10%2026.04 | 1.8 | 97 | 35 | 45 | 29 | 38 | 66 | |
| LR (Top 150 features)Backbone=Qwen-2-Audio, Training Data Source=S2TT, Number of Features=600, k=10%2026.04 | 1.51 | 95 | 28 | 61 | 18 | 44 | 67 | |
| LR (Combined features)Backbone=Qwen-2-Audio, Training Data Source=S2TT, Number of Features=4096, k=10%2026.04 | 1.32 | 96 | 29 | 69 | 18 | 43 | 67 | |
| LR (AUDIORATIO features)Backbone=Qwen-2-Audio, Training Data Source=S2TT, Number of Features=1024, k=10%2026.04 | 1.3 | 95 | 23 | 56 | 15 | 39 | 64 | |
| RAUQ EntropyBackbone=Voxtral-3B, Training data source=S2TT DATA, k-value=10%2026.04 | 1.15 | 95 | 13 | 34 | 8 | 16 | 35 | |
| Mean EntropyBackbone=Voxtral-3B, Training data source=S2TT DATA, k-value=10%2026.04 | 1.13 | 95 | 11 | 31 | 7 | 17 | 32 | |
| PerplexityBackbone=Qwen-2-Audio, Training Data Source=S2TT, k=10%2026.04 | 0.23 | 95 | 8 | 90 | 4 | 23 | 39 | |
| Mean EntropyBackbone=Qwen-2-Audio, Training Data Source=S2TT, k=10%2026.04 | 0.21 | 95 | 8 | 100 | 4 | 25 | 45 | |
| RAUQ EntropyBackbone=Qwen-2-Audio, Training Data Source=S2TT, k=10%2026.04 | 0.18 | 95 | 7 | 100 | 4 | 25 | 46 | |
| PerplexityBackbone=Voxtral-3B, Training data source=S2TT DATA, k-value=10%2026.04 | 0.17 | 95 | 5 | 75 | 3 | 15 | 27 |