Turn-taking prediction on DnD Gesture++
71.5AccuracyText+Audio+Gesture
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Text+Audio+GestureModality=Text+Audio+Gesture, Fusion=MoE, Semantic gesture embeddings=true2025.10 | 71.5 | 69.9 | 76.7 | 63.1 | |
| Text+Audio+GestureModality=Text+Audio+Gesture, Fusion=MoE, Semantic gesture embeddings=false2025.10 | 70.4 | 68.7 | 75.9 | 61.5 | |
| Text+GestureModality=Text+Gesture, Fusion=MoE, Semantic gesture embeddings=true2025.10 | 70 | 67.8 | 76.2 | 59.4 | |
| Text+AudioModality=Text+Audio, Fusion=MoE2025.10 | 69.7 | 67.9 | 74.9 | 61.1 | |
| TextModality=Text2025.10 | 68.8 | 66.6 | 75.1 | 58.2 | |
| Audio+GestureModality=Audio+Gesture, Fusion=MoE, Semantic gesture embeddings=true2025.10 | 67.7 | 64.2 | 75.4 | 53 | |
| AudioModality=Audio2025.10 | 67.1 | 63.2 | 75.1 | 51.3 | |
| GestureModality=Gesture, Semantic gesture embeddings=true2025.10 | 66.2 | 61.2 | 75.1 | 47.4 | |
| GestureModality=Gesture, Semantic gesture embeddings=false2025.10 | 66.1 | 61.8 | 74.6 | 48.9 | |
| Majority ClassModality=None2025.10 | 60.4 | 37.6 | 75.3 | 0 |