Sign Language Translation on How2Sign (test)
24.6BLEU-4Sliding Perceiver
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Sliding PerceiverPre-training=true, Pre-training hours=2.7k2025.12 | 24.6 | — | 58.7 | — | — | — | — | — | 48.1 | 219 | 43.7 | 2.92 | |
| SSLTPre-training=true, Pre-training hours=6.6k2025.12 | 21.1 | — | 55.7 | — | — | — | — | — | — | — | — | — | |
| Sliding PerceiverPre-training=false2025.12 | 17.5 | — | 49.5 | — | — | — | — | — | 38.4 | 147.5 | 32.2 | 1.97 | |
| SignDPO2026.04 | 15.58 | — | — | — | 39.93 | 27.44 | 21.07 | — | 38.14 | — | — | — | |
| SSVP-SLTModality=Pose, Protocol=Gloss-free2025.01 | 15.5 | — | 0.496 | — | 43.2 | 28.8 | 20.8 | 38.4 | — | — | — | — | |
| SSVP-SLTModality=Pose, Protocol=Gloss-free SLT2025.01 | 15.5 | — | 0.496 | — | 43.2 | — | — | 38.4 | — | — | — | — | |
| SSVP-SLTPre-training=true, Pre-training hours=1.0k2025.12 | 15.5 | — | 49.6 | — | — | — | — | — | 38.4 | — | — | — | |
| SSVP-SLTExternal Pre-training=YouTube-ASL2025.12 | 15.5 | — | — | — | 43.2 | 28.8 | 20.8 | 38.4 | — | — | — | — | |
| SSVP-SLTPre-training on YouTube-ASL=true2025.01 | 15.5 | — | 0.496 | — | — | — | — | — | 38.4 | — | — | — | |
| SSVP-SLT-LSPModality (RGB)=true, Gloss-free=true, Pre-training=YT-ASL and How2Sign2026.06 | 15.5 | — | 0.496 | — | 43.2 | — | — | — | 38.4 | — | — | — | |
| SIGNETModality (Pose)=true, Gloss-free=true2026.06 | 15.4 | — | 0.495 | — | 41.1 | — | — | — | 36.4 | — | — | — | |
| Geo-SignPre-training=true, Pre-training hours=1.9k2025.12 | 15.1 | — | — | — | — | — | — | — | 35.4 | — | — | — | |
| Geo-SignModality (Pose)=true, Gloss-free=true2026.06 | 15.1 | — | — | — | 40.8 | — | — | — | 35.4 | — | — | — | |
| Sigma2025.09 | 14.92 | — | — | — | 39.89 | 27.12 | 19.81 | — | 36.88 | — | — | — | |
| Uni-SignModality=Pose + RGB, Protocol=Gloss-free2025.01 | 14.9 | — | 0.494 | — | 40.2 | 27.1 | 19.7 | 36 | — | — | — | — | |
| Uni-SignModality=Pose+RGB, Protocol=Gloss-free SLT2025.01 | 14.9 | — | 0.494 | — | 40.2 | — | — | 36 | — | — | — | — | |
| Uni-SignPre-training=true, Pre-training hours=1.9k2025.12 | 14.9 | — | 49.4 | — | — | — | — | — | 36 | — | — | — | |
| Uni-SignExternal Pre-training=YouTube-ASL2025.12 | 14.9 | — | — | — | 40.2 | 27.1 | 19.7 | 36 | — | — | — | — | |
| Uni-SignModality (Pose)=true, Modality (RGB)=true, Gloss-free=true2026.06 | 14.9 | — | 0.494 | — | 40.2 | — | — | — | 36 | — | — | — | |
| SSVP-SLTModality (RGB)=true, Gloss-free=true2026.06 | 14.7 | — | 0.493 | — | 41.9 | — | — | — | 37.8 | — | — | — | |
| Uni-SignModality=Pose, Protocol=Gloss-free2025.01 | 14.5 | — | 0.486 | — | 40.4 | 26.8 | 19.3 | 34.3 | — | — | — | — | |
| Uni-SignModality=Pose, Protocol=Gloss-free SLT2025.01 | 14.5 | — | 0.486 | — | 40.4 | — | — | 34.3 | — | — | — | — | |
| SignMusketeersModality=Pose, Protocol=Gloss-free2025.01 | 14.3 | — | — | — | 41.5 | 27.2 | 19.3 | — | — | — | — | — | |
| SignMusketeersModality=Pose, Protocol=Gloss-free SLT2025.01 | 14.3 | — | — | — | 41.5 | — | — | — | — | — | — | — | |
| SignMusketeersExternal Pre-training=YouTube-ASL2025.12 | 14.3 | — | — | — | 41.5 | 27.2 | 19.3 | — | — | — | — | — | |
| SLT-GRPOExternal Pre-training=None2025.12 | 14.3 | — | — | — | 39.4 | 25.3 | 18.6 | 34.7 | — | — | — | — | |
| SignMusketeersModality (RGB)=true, Gloss-free=true2026.06 | 14.3 | — | — | — | 41.5 | — | — | — | — | — | — | — | |
| PGG-SLTModality (RGB)=true, Gloss-free=true2026.06 | 13.7 | — | — | — | 40.8 | — | — | — | 32.9 | — | — | — | |
| Sliding PerceiverEvaluation Protocol=zero-shot2025.12 | 13.6 | — | 0.513 | — | — | — | — | — | 35.7 | 123.2 | 30.8 | 2.1 | |
| VAPPre-training=false2025.12 | 12.9 | — | — | — | — | — | — | — | 27.8 | — | — | — | |
| VAPExternal Pre-training=None2025.12 | 12.9 | — | — | — | 39.2 | — | — | 27.8 | — | — | — | — | |
| VAP2025.01 | 12.9 | — | — | — | — | — | — | — | 27.8 | — | — | — | |
| LiTFiCPre-training=false2025.12 | 12.7 | — | 45.3 | — | — | — | — | — | 32.5 | 100.8 | 27.9 | 1.59 | |
| LLM-based SLT with Contextual CuesInput cues=Vid+PG+PrevPred2025.01 | 12.7 | — | 0.453 | — | — | — | — | — | 32.5 | 1.008 | 27.9 | 1.59 | |
| Jang et al.Modality (RGB)=true, Gloss-free=true2026.06 | 12.7 | — | 0.453 | — | — | — | — | — | 32.5 | — | — | — | |
| YouTube-ASLModality=Pose, Protocol=Gloss-free2025.01 | 12.4 | — | 0.466 | — | 37.8 | 24.1 | 16.9 | — | — | — | — | — | |
| ConSLTModality=Pose, Protocol=Gloss-free SLT2025.01 | 12.4 | — | — | — | 37.8 | — | — | 46.6 | — | — | — | — | |
| YouTube-ASLExternal Pre-training=YouTube-ASL2025.12 | 12.4 | — | — | — | 37.8 | 24.1 | 16.9 | — | — | — | — | — | |
| YouTube-ASL2025.09 | 12.4 | — | — | — | 37.8 | 24.1 | 16.9 | — | — | — | — | — | |
| YouTube-ASL2026.04 | 12.4 | — | — | — | 37.8 | 24.1 | 16.9 | — | — | — | — | — | |
| YouTube-ASLModality (RGB)=true, Gloss-free=true2026.06 | 12.4 | — | 0.466 | — | 37.8 | — | — | — | — | — | — | — | |
| LLM-based SLT with Contextual CuesInput cues=Vid+PG2025.01 | 12.3 | — | 0.447 | — | — | — | — | — | 31.9 | 0.978 | 27.4 | 1.55 | |
| SLT-SFTExternal Pre-training=None2025.12 | 12.2 | — | — | — | 38.3 | 23.7 | 16.5 | 31.3 | — | — | — | — | |
| T5-baseTraining Data=YT-ASL+H2S, Backbone=T5-base, Number of Parameters=248 M, Frame Rate=12-30 fps2026.05 | 11.89 | — | — | — | 36.35 | 23 | 16.13 | — | — | — | — | — | |
| LLM-based SLT with Contextual CuesInput cues=Vid2025.01 | 11.8 | — | 0.441 | — | — | — | — | — | 31.1 | 0.933 | 26.1 | 1.39 | |
| Back-translation2024.07 | 10.89 | — | — | — | 22.38 | 16.71 | 13.32 | 23.23 | — | — | — | — | |
| T5-smallTraining Data=YT-ASL+H2S, Backbone=T5-small, Number of Parameters=77 M, Frame Rate=24 fps2026.05 | 10.06 | — | — | — | 34.29 | 20.83 | 14.06 | — | — | — | — | — | |
| FLa-LLMModality=Pose, Protocol=Gloss-free2025.01 | 9.7 | — | — | — | 29.8 | 19 | 13.3 | 27.8 | — | — | — | — | |
| FLa-LLMModality=Pose, Protocol=Gloss-free SLT2025.01 | 9.7 | — | — | — | 29.8 | — | — | 27.8 | — | — | — | — | |
| FLa-LLMExternal Pre-training=None2025.12 | 9.7 | — | — | — | 29.8 | 19 | 13.3 | 27.8 | — | — | — | — | |
| Fla-LLM2025.01 | 9.7 | — | — | — | — | — | — | — | 27.8 | — | — | — | |
| FLa-LLM2025.09 | 9.7 | — | — | — | 29.8 | 19 | 13.3 | — | 27.8 | — | — | — | |
| FLa-LLM2026.04 | 9.7 | — | — | — | 29.8 | 19 | 13.3 | — | 27.8 | — | — | — | |
| FLa-LLMModality (RGB)=true, Gloss-free=true2026.06 | 9.7 | — | — | — | 29.8 | — | — | — | 27.8 | — | — | — | |
| T5-smallTraining Data=YT-ASL+H2S, Backbone=T5-small, Number of Parameters=77 M, Frame Rate=12 fps2026.05 | 9.53 | — | — | — | 33.21 | 20.01 | 13.43 | — | — | — | — | — | |
| C2RLModality=Pose, Protocol=Gloss-free2025.01 | 9.4 | — | — | — | 29.1 | 18.6 | 12.9 | 27 | — | — | — | — | |
| C2RLModality=Pose, Protocol=Gloss-free SLT2025.01 | 9.4 | — | — | — | 29.1 | — | — | 27 | — | — | — | — | |
| C2RLExternal Pre-training=None2025.12 | 9.4 | — | — | — | 29.1 | 18.6 | 12.9 | 27 | — | — | — | — | |
| C^2RL2025.09 | 9.4 | — | — | — | 29.1 | 18.6 | 12.9 | — | 27 | — | — | — | |
| C^2RL2026.04 | 9.4 | — | — | — | 29.1 | 18.6 | 12.9 | — | 27 | — | — | — | |
| C2RLModality (RGB)=true, Gloss-free=true2026.06 | 9.4 | — | — | — | 29.1 | — | — | — | 27 | — | — | — | |
| SLT-IVModality=Pose, Protocol=Gloss-free2025.01 | 8 | — | — | — | 34 | 19.3 | 12.2 | — | — | — | — | — | |
| SLT-IVModality=Pose, Protocol=Gloss-free SLT2025.01 | 8 | — | — | — | 34 | — | — | — | — | — | — | — | |
| OpenSLTExternal Pre-training=None2025.12 | 8 | — | — | — | 34 | 19.3 | 12.2 | — | — | — | — | — | |
| SLT-IVModality (RGB)=true, Gloss-free=true2026.06 | 8 | — | — | — | 34 | — | — | — | — | — | — | — | |
| SSVP-SLTEvaluation Protocol=zero-shot2025.12 | 7.1 | — | 0.418 | — | — | — | — | — | 28.3 | — | — | — | |
| SSVP-SLTExternal Pre-training=None2025.12 | 7 | — | — | — | 30.2 | 16.7 | 10.5 | 25.7 | — | — | — | — | |
| SSVP-SLTPre-training on YouTube-ASL=false2025.01 | 7 | — | 0.393 | — | — | — | — | — | 25.7 | — | — | — | |
| SSLTEvaluation Protocol=zero-shot2025.12 | 5.8 | — | 0.371 | — | — | — | — | — | — | — | — | — | |
| FLEURS-SLTEvaluation Protocol=zero-shot2025.12 | 5.1 | — | 0.369 | — | — | — | — | — | — | — | — | — | |
| MS2SLECL strategy=enabled2024.07 | 4.26 | — | — | — | 14.67 | 9.17 | 6.84 | 16.38 | — | — | — | — | |
| T5-baseTraining Data=YT-ASL, Backbone=T5-base, Number of Parameters=248 M, Frame Rate=12-30 fps2026.05 | 3.95 | — | — | — | 20.93 | 10.35 | 6.14 | — | — | — | — | — | |
| MS2SLECL strategy=disabled2024.07 | 3.76 | — | — | — | 14.51 | 8.05 | 6.03 | 15.1 | — | — | — | — | |
| T5-smallTraining Data=YT-ASL, Backbone=T5-small, Number of Parameters=77 M, Frame Rate=24 fps2026.05 | 3.62 | — | — | — | 18.05 | 8.69 | 5.41 | — | — | — | — | — | |
| T2M-GPT2024.07 | 3.53 | — | — | — | 12.87 | 7.92 | 5.14 | 13.99 | — | — | — | — | |
| T5-smallTraining Data=YT-ASL, Backbone=T5-small, Number of Parameters=77 M, Frame Rate=12 fps2026.05 | 3.34 | — | — | — | 17.3 | 7.82 | 5.28 | — | — | — | — | — | |
| Ham2Pose2024.07 | 2.93 | — | — | — | 12.38 | 7.31 | 4.07 | 13.29 | — | — | — | — | |
| MSLUModality=Pose, Protocol=Gloss-free2025.01 | 2.4 | — | — | — | 20.1 | 7.7 | — | 17.2 | — | — | — | — | |
| YouTube-ASLModality=RGB, Protocol=Gloss-free SLT2025.01 | 2.4 | — | — | — | 20.1 | — | — | 17.2 | — | — | — | — | |
| MSLU2025.09 | 2.4 | — | — | — | 20.1 | 7.7 | — | — | 17.2 | — | — | — | |
| MSLU2026.04 | 2.4 | — | — | — | 20.1 | 7.7 | — | — | 17.2 | — | — | — | |
| MOMP2024.07 | 2.34 | — | — | — | 13.68 | 7.63 | 3.92 | 13.83 | — | — | — | — | |
| GloFE-VNanchor_words=verbs and nouns2023.05 | 2.24 | — | 31.65 | — | 14.94 | 7.27 | 3.93 | 12.61 | — | — | — | — | |
| Alvarezvisual_features=CNN pre-trained with GLOSS annotations2023.05 | 2.21 | — | — | — | 17.4 | 7.69 | 3.97 | — | — | — | — | — | |
| GloFE-VNModality=Pose, Protocol=Gloss-free2025.01 | 2.2 | — | 0.317 | — | 14.9 | 7.3 | 3.9 | 12.6 | — | — | — | — | |
| GloFE-VNModality=Pose, Protocol=Gloss-free SLT2025.01 | 2.2 | — | 0.317 | — | 14.9 | — | — | 12.6 | — | — | — | — | |
| GloFE-VNExternal Pre-training=None2025.12 | 2.2 | — | — | — | 14.9 | 7.3 | 3.9 | 12.6 | — | — | — | — | |
| GloFE-VN2025.09 | 2.2 | — | — | — | 14.9 | 7.3 | 3.9 | — | 12.6 | — | — | — | |
| GloFE-VN2026.04 | 2.2 | — | — | — | 14.9 | 7.3 | 3.9 | — | 12.6 | — | — | — | |
| PT2024.07 | 2.01 | — | — | — | 13.69 | 7.04 | 3.86 | 13.81 | — | — | — | — | |
| T5-baseTraining Data=H2S, Backbone=T5-base, Number of Parameters=248 M, Frame Rate=12-30 fps2026.05 | 1.22 | — | — | — | 14.96 | 5.11 | 2.26 | — | — | — | — | — | |
| YouTube-ASLExternal Pre-training=None2025.12 | 1.2 | — | — | — | 15 | 5.1 | 2.3 | — | — | — | — | — | |
| T5-smallTraining Data=H2S, Backbone=T5-small, Number of Parameters=77 M, Frame Rate=24 fps2026.05 | 1.07 | — | — | — | 12.98 | 4.22 | 1.82 | — | — | — | — | — | |
| T5-smallTraining Data=H2S, Backbone=T5-small, Number of Parameters=77 M, Frame Rate=12 fps2026.05 | 0.82 | — | — | — | 12.23 | 3.67 | 1.56 | — | — | — | — | — | |
| Lin et al.Supervised training Schedule=H2S2024.06 | — | 0.022 | — | — | 14.9 | 7.3 | 3.9 | — | — | — | — | — | |
| SHuBERTSSL=true, PT data (hrs)=9842024.11 | — | 0.162 | 0.499 | — | — | — | — | — | — | — | — | — | |
| SignMusketeersSupervised training Schedule=H2S, Pre-training Epochs=5, PT % of Frames=1.2%2024.06 | — | 0.024 | — | — | 18.8 | 8.1 | 4.2 | — | — | — | — | — | |
| SignMusketeersSupervised training Schedule=YT, Pre-training Epochs=5, PT % of Frames=1.2%2024.06 | — | 0.052 | — | — | 26.3 | 13.8 | 8.2 | — | — | — | — | — | |
| SignMusketeersSupervised training Schedule=YT -> H2S, Pre-training Epochs=5, PT % of Frames=1.2%2024.06 | — | 0.143 | — | — | 41.5 | 27.2 | 19.3 | — | — | — | — | — | |
| SMPT data (hrs)=9842024.11 | — | 0.143 | — | — | — | — | — | — | — | — | — | — |