Sign Language Translation on BOBSL SENT (test)
7.3B4LLM-based SLT with Contextual Cues
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| LLM-based SLT with Contextual CuesInput Cues=Vid + GT Previous Sentence + Spotting + BG, Oracle=true2025.01 | 7.3 | 47.1 | 25.1 | 88.9 | 26.5 | 1.85 | |
| Ours (Multitask)Pre-training (PT)=true, Training hours=2.7K2025.12 | 6.8 | 47 | 23.7 | 77 | 21.8 | 1.77 | |
| Ours (SLT)Pre-training (PT)=true, Training hours=2.7K2025.12 | 6.7 | 47.1 | 23.7 | 76.6 | 21.8 | 1.74 | |
| LLM-based SLT with Contextual CuesInput Cues=Vid + GT Previous Sentence + Spotting, Oracle=true2025.01 | 6.5 | 45.9 | 24 | 82.5 | 25.5 | 1.69 | |
| Ours (SLT, w/o PT)Pre-training (PT)=false2025.12 | 6 | 46.4 | 23.1 | 73.4 | 20.9 | 1.67 | |
| LiTFiC (Full)Pre-training (PT)=false2025.12 | 3.3 | 40.3 | 16.9 | 41.9 | 14.8 | 1.2 | |
| LLM-based SLT with Contextual CuesInput Cues=Vid + Predicted Previous Sentence + PG + BG2025.01 | 3.3 | 40.3 | 16.9 | 41.9 | 14.8 | 1.2 | |
| LLM-based SLT with Contextual CuesInput Cues=Vid + GT Previous Sentence, Oracle=true2025.01 | 3.3 | 40.5 | 17.3 | 42.8 | 14.8 | 1.23 | |
| LLM-based SLT with Contextual CuesInput Cues=Vid + Predicted Previous Sentence + PG2025.01 | 2.9 | 39.2 | 16.1 | 38.9 | 14.1 | 1.15 | |
| SincanInput Cues=Vid + GT Previous Sentence + Spotting, Oracle=true2025.01 | 2.9 | 37 | 12.4 | 41 | 12.5 | 0.8 | |
| LLM-based SLT with Contextual CuesInput Cues=Vid + Predicted Previous Sentence2025.01 | 2.8 | 38.9 | 16 | 37.8 | 13.8 | 1.02 | |
| LiTFiC (Vid)Pre-training (PT)=false2025.12 | 2.6 | 37.8 | 15.6 | 37.5 | 13.6 | 0.95 | |
| LLM-based SLT with Contextual CuesInput Cues=Video Only2025.01 | 2.6 | 37.8 | 15.6 | 37.5 | 13.6 | 0.95 | |
| SincanInput Cues=Vid + GT Previous Sentence, Oracle=true2025.01 | 1.5 | 35.8 | 9.7 | 23.9 | 10.4 | 0.56 | |
| SincanPre-training (PT)=false2025.12 | 1.3 | — | 8.9 | — | — | — | |
| SincanInput Cues=Video Only2025.01 | 1.3 | — | 8.9 | — | — | — | |
| AlbaniePre-training (PT)=false2025.12 | 1 | — | 10.2 | — | — | — | |
| AlbanieInput Cues=Video Only2025.01 | 1 | — | 10.2 | — | — | — | |
| Sign2GPTPre-training (PT)=false2025.12 | 0.9 | 35.2 | 11.4 | 16.1 | 8.7 | 0.49 | |
| Sign2GPTInput Cues=Video, PG, Trained on BOBSL=true2025.01 | 0.9 | 35.2 | 11.4 | 16.1 | 8.7 | 0.49 | |
| Sign2GPTInput Cues=Video Only, Trained on BOBSL=true2025.01 | 0.7 | 34.3 | 10.6 | 12.8 | 8.2 | 0.37 | |
| GFSLTPre-training (PT)=false2025.12 | 0.6 | 27.7 | 7.4 | 4.3 | 5.2 | 0.05 | |
| GFSLTInput Cues=Video Only, Trained on BOBSL=true2025.01 | 0.6 | 27.7 | 7.4 | 4.3 | 5.2 | 0.05 |