Expert Commentary Generation on Ego-Exo4D 1.0 (test)
45.8BLEU-4ExpertAF w/ full-sup
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| ExpertAF w/ full-supprivileged_input=demo video at inference2024.08 | 45.8 | 50.9 | 55.7 | — | |
| ExpertAF2024.08 | 44.9 | 49.6 | 54.6 | 3.2 | |
| ExpertAF w/o videoablation=no video2024.08 | 44.6 | 49.4 | 54.2 | — | |
| ExpertAF w/o poseablation=no pose2024.08 | 44.2 | 49.3 | 54.2 | — | |
| LLaVA-FT w/ posemode=Fine-tuned, pose_input=true2024.08 | 43.6 | 48.5 | 51.7 | 2.5 | |
| LLaVA-FTmode=Fine-tuned2024.08 | 43.5 | 48.5 | 51.5 | — | |
| ExpertAF w/ global poseablation=global pose instead of sequence2024.08 | 43 | 47.9 | 52.6 | — | |
| InternVideo2-FTmode=Fine-tuned2024.08 | 42.9 | 47.6 | 50 | 1.8 | |
| InternVideo2-NNmode=Nearest Neighbor2024.08 | 42.1 | 46.9 | 49.3 | — | |
| ExpertAF w/o alignmentablation=no temporal alignment2024.08 | 42 | 48.6 | 51.5 | — | |
| LLaVA2024.08 | 28.5 | 44.1 | 44.2 | 1.1 | |
| VideoChat22024.08 | 27.8 | 44.3 | 41.9 | — | |
| PoseScript/Fix2024.08 | 24.1 | 44.5 | 46.3 | 1.3 |