Reading Recognition on EGTEA
89.6AccuracyVLM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VLMGaze modality=false, RGB modality=true, IMU modality=false, Fusion capability=true, Number of parameters=11B, Sensing cost (power)=high, RGB requirements=full RGB, Real-time=false, Inference time (ms)=567.410, Zero-shot capability=true2025.05 | 89.6 | 61.5 | |
| Flexible Multimodal TransformerGaze modality=true, RGB modality=true, IMU modality=true, Fusion capability=true, Number of parameters=130k, Sensing cost (power)=low, RGB requirements=foveated patch (5° FoV) (optional), Real-time=true, Inference time (ms)=0.545, Zero-shot capability=true2025.05 | 89.6 | 70.6 | |
| Action recognitionGaze modality=false, RGB modality=true, IMU modality=false, Fusion capability=false, Number of parameters=25M, Sensing cost (power)=high, RGB requirements=full RGB video, Real-time=false, Inference time (ms)=895.511 (incl. flow), Zero-shot capability=false2025.05 | 88.8 | 65.8 | |
| Alternative arch. with gaze onlyGaze modality=true, RGB modality=false, IMU modality=false, Fusion capability=false, Number of parameters=1k, Sensing cost (power)=low, RGB requirements=-, Real-time=true, Inference time (ms)=0.310, Zero-shot capability=false2025.05 | 85.8 | 62.8 |