Audio-Visual Speech Separation on LRS2 (Efficiency Evaluation)
0.85Parameters (M)RTFS-Net
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| RTFS-NetPre-trained Video Encoder=w/o2025.09 | 0.85 | 33.56 | — | — | |
| Swift-NetPre-trained Video Encoder=w/o2025.09 | 1.47 | 18.24 | — | — | |
| IIANetPre-trained Video Encoder=w/o2025.09 | 3.82 | 18.56 | — | — | |
| AVLiT-8Pre-trained Video Encoder=w/o2025.09 | 5.54 | 17.52 | — | — | |
| AVLiT-8Pre-trained Video Encoder=w/2025.09 | 5.57 | 17.53 | — | — | |
| DolphinPre-trained Video Encoder=w/o2025.09 | 6.22 | 8.51 | — | — | |
| CTCNetPre-trained Video Encoder=w/o2025.09 | 6.87 | 75.8 | — | — | |
| DolphinPre-trained Video Encoder=w/2025.09 | 7 | 10.89 | — | — | |
| RTFS-NetPre-trained Video Encoder=w/2025.09 | 12.03 | 41.51 | — | — | |
| Swift-NetPre-trained Video Encoder=w/2025.09 | 12.66 | 26.18 | — | — | |
| AV-ConvTasNetPre-trained Video Encoder=w/o2025.09 | 13.7 | 6.94 | — | — | |
| IIANetPre-trained Video Encoder=w/2025.09 | 15.01 | 26.51 | — | — | |
| CTCNetPre-trained Video Encoder=w/2025.09 | 18.06 | 83.75 | — | — | |
| AV-ConvTasNetPre-trained Video Encoder=w/2025.09 | 25.01 | 14.9 | — | — | |
| AV-Mossformer2Pre-trained Video Encoder=w/o2025.09 | 55.87 | 111.79 | — | — | |
| AV-Mossformer2Pre-trained Video Encoder=w/2025.09 | 68.52 | 124.46 | — | — | |
| VisualvoicePre-trained Video Encoder=w/o2025.09 | 68.61 | 16.82 | — | — | |
| VisualvoicePre-trained Video Encoder=w/2025.09 | 77.75 | 18.2 | — | — | |
| AV-ConvTasNetInference Device=CPU, Memory Mode=RAM2025.09 | — | — | 342.15 | 95.47 | |
| AV-ConvTasNetInference Device=GPU, Memory Mode=GPU Inf.2025.09 | — | — | 10.54 | 129.51 | |
| AV-Mossformer2Inference Device=CPU, Memory Mode=RAM2025.09 | — | — | 3,083.86 | 262.45 | |
| AV-Mossformer2Inference Device=GPU, Memory Mode=GPU Inf.2025.09 | — | — | 62.3 | 398.76 | |
| AVLiT-8Inference Device=CPU, Memory Mode=RAM2025.09 | — | — | 547.77 | 21.23 | |
| AVLiT-8Inference Device=GPU, Memory Mode=GPU Inf.2025.09 | — | — | 23.31 | 52.09 | |
| CTCNetInference Device=CPU, Memory Mode=RAM2025.09 | — | — | 1,433.68 | 68.95 | |
| CTCNetInference Device=GPU, Memory Mode=GPU Inf.2025.09 | — | — | 37.52 | 138.42 | |
| DolphinInference Device=CPU, Memory Mode=RAM2025.09 | — | — | 2,117.96 | 26.84 | |
| DolphinInference Device=GPU, Memory Mode=GPU Inf.2025.09 | — | — | 33.24 | 251.12 | |
| IIANetInference Device=CPU, Memory Mode=RAM2025.09 | — | — | 3,213.82 | 57.31 | |
| IIANetInference Device=GPU, Memory Mode=GPU Inf.2025.09 | — | — | 142.3 | 148.14 | |
| RTFS-NetInference Device=CPU, Memory Mode=RAM2025.09 | — | — | 4,021.93 | 48.4 | |
| RTFS-NetInference Device=GPU, Memory Mode=GPU Inf.2025.09 | — | — | 56.59 | 249.49 | |
| Swift-NetInference Device=CPU, Memory Mode=RAM2025.09 | — | — | 2,589.76 | 48.33 | |
| Swift-NetInference Device=GPU, Memory Mode=GPU Inf.2025.09 | — | — | 39.45 | 244.53 | |
| VisualvoiceInference Device=CPU, Memory Mode=RAM2025.09 | — | — | 280.29 | 296.84 | |
| VisualvoiceInference Device=GPU, Memory Mode=GPU Inf.2025.09 | — | — | 7.19 | 338.96 |