Throughput Evaluation on MELD Emotion
17.3RTFAU-Harness
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| AU-HarnessHF Support=✓, vLLM Support=✓, Multi-task Parallel=✓, Multi-turn=✓, Customizable=✓, Simulated Agentic Dialogue=✗2025.09 | 17.3 | 1.96 | |
| LMMs-EvalHF Support=✓, vLLM Support=✓, Multi-task Parallel=✓, Multi-turn=✗, Customizable=✗, Simulated Agentic Dialogue=✗2025.09 | 40.33 | 0.84 | |
| AHELMHF Support=✓, vLLM Support=✓, Multi-task Parallel=✗, Multi-turn=✓, Customizable=✗, Simulated Agentic Dialogue=✗2025.09 | 41.6 | 0.81 | |
| Kimi-EvalHF Support=✓, vLLM Support=✗, Multi-task Parallel=✗, Multi-turn=✗, Customizable=✗, Simulated Agentic Dialogue=✗2025.09 | 96.23 | 0.35 | |
| VoiceBenchHF Support=✓, vLLM Support=✗, Multi-task Parallel=✗, Multi-turn=✗, Customizable=✗, Simulated Agentic Dialogue=✗2025.09 | 124.48 | 0.27 | |
| AudioBenchHF Support=✓, vLLM Support=✗, Multi-task Parallel=✗, Multi-turn=✗, Customizable=✗, Simulated Agentic Dialogue=✗2025.09 | 162.69 | 0.21 |