Step Forecasting on COIN
56.2AccuracyVL-JEPA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VL-JEPAModel size (Parameters)=1.6B, Fine-tuning=true2025.12 | 56.2 | — | |
| ProVideLLMModel size (Parameters)=8B/11+, Fine-tuning=true2025.12 | 53.6 | — | |
| GADEvaluation Protocol=fine-tuned2026.03 | 51.6 | — | |
| VideoLLM-MoDModel size (Parameters)=8B, Fine-tuning=true2025.12 | 49.7 | — | |
| VideoLLM-onlineModel size (Parameters)=8B, Fine-tuning=true2025.12 | 49.1 | — | |
| OursDownstream architecture=Transformer2023.07 | 40.2 | — | |
| Transformer w/ KB TransferSegment Model=TimeSformer, Pretraining Supervision=Unsupervised: distant supervision (ours), Pretraining Dataset=HT100M2022.01 | 39.4 | — | |
| DistantSup.Downstream architecture=Transformer2023.07 | 39.4 | — | |
| Basic TransformerSegment Model=TimeSformer, Pretraining Supervision=Unsupervised: distant supervision (ours), Pretraining Dataset=HT100M2022.01 | 38.2 | — | |
| TimeSformerDownstream architecture=Transformer2023.07 | 38.2 | — | |
| OursDownstream architecture=MLP2023.07 | 36.3 | — | |
| DistantSup.Downstream architecture=MLP2023.07 | 35.5 | — | |
| Basic TransformerSegment Model=TimeSformer [8], Pretraining Supervision=Supervised: action labels, Pretraining Dataset=Kinetics2022.01 | 34.7 | — | |
| VideoCLIPDownstream architecture=Transformer2023.07 | 34.6 | — | |
| Basic TransformerSegment Model=TimeSformer [8], Pretraining Supervision=Unsupervised: k-means on ASR, Pretraining Dataset=HT100M2022.01 | 34 | — | |
| TimeSformerDownstream architecture=MLP2023.07 | 32.7 | — | |
| VideoCLIPDownstream architecture=MLP2023.07 | 30 | — | |
| Basic TransformerSegment Model=S3D [39], Pretraining Supervision=Unsupervised: MIL-NCE on ASR, Pretraining Dataset=HT100M2022.01 | 28.1 | — | |
| S3DDownstream architecture=Transformer2023.07 | 28.1 | — | |
| Basic TransformerSegment Model=SlowFast [17], Pretraining Supervision=Supervised: action labels, Pretraining Dataset=Kinetics2022.01 | 25.6 | — | |
| SlowFastDownstream architecture=Transformer2023.07 | 25.6 | — | |
| SlowFastDownstream architecture=MLP2023.07 | 23 | — | |
| S3DDownstream architecture=MLP2023.07 | 19.9 | — | |
| Qwen2.5-VL-7BEvaluation Protocol=open-ended2026.03 | 8.9 | — | |
| Qwen2.5-VL-7BPrompting Strategy=categories in prompt2026.03 | 6.5 | — | |
| Videollm-online-8BEvaluation Protocol=open-ended2026.03 | 3.5 | — | |
| CLIPSupervision=Unsupervised: captions, Pretraining Dataset=CLIP400M, Evaluation Protocol=Zero-shot2023.03 | — | 9.4 | |
| DiscBackbone=Llama3.2-1B-Instruct, Classifier Type=Discriminative2026.03 | — | 50.1 | |
| DiscBackbone=Llama3-8B-Instruct, Classifier Type=Discriminative2026.03 | — | 51 | |
| DistantSupSupervision=Unsupervised: ASR + wikiHow, Pretraining Dataset=HT100M, Evaluation Protocol=Fine-tuning2023.03 | — | 39.4 | |
| GADBackbone=Llama3.2-1B-Instruct, Classifier Type=Generative-Assisted Discriminative2026.03 | — | 51.4 | |
| GADBackbone=Llama3-8B-Instruct, Classifier Type=Generative-Assisted Discriminative2026.03 | — | 51.6 | |
| ProcedureVRLSupervision=Unsupervised: ASR, Pretraining Dataset=HT100M, Evaluation Protocol=Zero-shot2023.03 | — | 11.3 | |
| ProcedureVRLSupervision=Unsupervised: ASR, Pretraining Dataset=HT100M, Evaluation Protocol=Fine-tuning2023.03 | — | 46.8 | |
| ProcedureVRL (oracle-5)Supervision=Unsupervised: ASR, Pretraining Dataset=HT100M, Evaluation Protocol=Zero-shot2023.03 | — | 14.7 | |
| ProcedureVRL (oracle-5)Supervision=Unsupervised: ASR, Pretraining Dataset=HT100M, Evaluation Protocol=Fine-tuning2023.03 | — | 51.8 | |
| Random GuessEvaluation Protocol=Zero-shot2023.03 | — | 0.1 | |
| S3DSupervision=Unsupervised: ASR w. MIL-NCE [39], Pretraining Dataset=HT100M, Evaluation Protocol=Fine-tuning2023.03 | — | 28.1 | |
| SlowFastSupervision=Supervised: action labels, Pretraining Dataset=Kinetics, Evaluation Protocol=Fine-tuning2023.03 | — | 25.6 | |
| StreamMind-8BBackbone=8B2026.03 | — | 49.9 | |
| TimeSformerSupervision=Supervised: action labels, Pretraining Dataset=Kinetics, Evaluation Protocol=Fine-tuning2023.03 | — | 34.7 | |
| TimeSformerSupervision=Unsupervised: ASR w. MIL-NCE [39], Pretraining Dataset=HT100M, Evaluation Protocol=Fine-tuning2023.03 | — | 34 | |
| Videollm-MoD-8BBackbone=8B2026.03 | — | 49.7 | |
| Videollm-online-8BBackbone=8B2026.03 | — | 49.1 | |
| VideoTaskGraph2026.03 | — | 40.2 |