Audio Attribute Verification on AC-Verify zero-shot
97.76Gender AccuracyQwen3-Omni-30B-A3B-Instruct
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Qwen3-Omni-30B-A3B-InstructSource Type=Open-source, Evaluation Protocol=Zero-shot forced-choice2026.06 | 97.76 | 95.37 | 80.93 | 69.59 | 85.91 | 55.28 | |
| MiMo-Audio-7B-InstructSource Type=Open-source, Evaluation Protocol=Zero-shot forced-choice2026.06 | 97.45 | 70.12 | 67.45 | 70.27 | 76.32 | 51.9 | |
| Gemini 3.5 FlashSource Type=Closed-source, Evaluation Protocol=Zero-shot forced-choice2026.06 | 97.41 | 92.35 | 84.4 | 76.99 | 87.79 | 51.72 | |
| Gemini 2.5 FlashSource Type=Closed-source, Evaluation Protocol=Zero-shot forced-choice2026.06 | 96.73 | 94.62 | 75.18 | 74.74 | 85.32 | 53.41 | |
| Ours (dual-task)Evaluation Protocol=Zero-shot forced-choice2026.06 | 95.93 | 97.43 | 93.33 | 88.66 | 93.84 | 72.53 | |
| Kimi-Audio-7B-InstructSource Type=Open-source, Evaluation Protocol=Zero-shot forced-choice2026.06 | 94.9 | 81.51 | 64.44 | 65.07 | 76.51 | 48.58 | |
| Audio Flamingo 3Source Type=Open-source, Evaluation Protocol=Zero-shot forced-choice2026.06 | 94.59 | 71.88 | 56.06 | 55.26 | 69.45 | 50.05 | |
| GPT audio miniSource Type=Closed-source, Evaluation Protocol=Zero-shot forced-choice2026.06 | 87.5 | 60.92 | 71.41 | 70.26 | 72.52 | 49.32 | |
| Qwen2-Audio-7B-InstructSource Type=Open-source, Evaluation Protocol=Zero-shot forced-choice2026.06 | 53.97 | 46.28 | 52.99 | 49.2 | 50.61 | 49.77 |