Response Consistency Evaluation on TPI-Bench Shared utterances between TPI and Janus (test)
94BLEUKimi-Audio-Instruct-7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Kimi-Audio-Instruct-7BData C (Trained on corpus data)=No, Data VA (Trained on dyadic voice assistant data)=No, Data HN (Trained on hard negatives)=No2026.04 | 94 | 99 | |
| ChatGPT-4o-audioData C (Trained on corpus data)=No, Data VA (Trained on dyadic voice assistant data)=No, Data HN (Trained on hard negatives)=No2026.04 | 89 | 93 | |
| TPI-BaseData C (Trained on corpus data)=Yes, Data VA (Trained on dyadic voice assistant data)=No, Data HN (Trained on hard negatives)=No2026.04 | 46 | 63 | |
| VITA-Audio-Instruct-7BData C (Trained on corpus data)=No, Data VA (Trained on dyadic voice assistant data)=No, Data HN (Trained on hard negatives)=No2026.04 | 42 | 71 | |
| TPI-VAData C (Trained on corpus data)=Yes, Data VA (Trained on dyadic voice assistant data)=Yes, Data HN (Trained on hard negatives)=No2026.04 | 39 | 58 | |
| Qwen2.5-Omni-7BData C (Trained on corpus data)=No, Data VA (Trained on dyadic voice assistant data)=No, Data HN (Trained on hard negatives)=No2026.04 | 31 | 53 | |
| Qwen3-Omni-30B-A3B-InstructData C (Trained on corpus data)=No, Data VA (Trained on dyadic voice assistant data)=No, Data HN (Trained on hard negatives)=No2026.04 | 20 | 39 | |
| TPI-FullData C (Trained on corpus data)=Yes, Data VA (Trained on dyadic voice assistant data)=Yes, Data HN (Trained on hard negatives)=Yes2026.04 | 12 | 34 |