Spoken Mathematics Reasoning on VoxEval Elementary 1.0 (test)
84.7AccuracySGS
Evaluation results
| Method | Links | |
|---|---|---|
| SGSModel=Qwen2.5, Hyperparams. (k, α)=4, 0.0252026.03 | 84.7 | |
| VanillaModel=Qwen2.5, Hyperparams. (k, α)=3, 0.0252026.03 | 83.9 | |
| TGSModel=Qwen2.5, Hyperparams. (k, α)=3, 0.052026.03 | 83.6 | |
| CoTModel=Qwen2.5, Hyperparams. (k, α)=-2026.03 | 83 | |
| TGSModel=Phi4-mm, Hyperparams. (k, α)=3, 0.0252026.03 | 71.4 | |
| CoTModel=Phi4-mm, Hyperparams. (k, α)=-2026.03 | 69.6 | |
| VanillaModel=Phi4-mm, Hyperparams. (k, α)=5, 0.0252026.03 | 69.1 | |
| VanillaModel=Voxtral, Hyperparams. (k, α)=5, 0.0252026.03 | 68.8 | |
| SGSModel=Phi4-mm, Hyperparams. (k, α)=3, 0.052026.03 | 68.8 | |
| CoTModel=Voxtral, Hyperparams. (k, α)=-2026.03 | 67.4 | |
| TGSModel=Voxtral, Hyperparams. (k, α)=4, 0.0252026.03 | 67.1 | |
| SGSModel=Voxtral, Hyperparams. (k, α)=3, 0.12026.03 | 65.8 | |
| NormalModel=Voxtral, Hyperparams. (k, α)=-2026.03 | 56.9 | |
| NormalModel=Qwen2.5, Hyperparams. (k, α)=-2026.03 | 52.1 | |
| TGSModel=AF3, Hyperparams. (k, α)=5, 0.12026.03 | 42.9 | |
| VanillaModel=AF3, Hyperparams. (k, α)=3, 0.12026.03 | 37.6 | |
| SGSModel=AF3, Hyperparams. (k, α)=1, 0.0252026.03 | 36 | |
| NormalModel=Phi4-mm, Hyperparams. (k, α)=-2026.03 | 33.1 | |
| CoTModel=AF3, Hyperparams. (k, α)=-2026.03 | 30.4 | |
| NormalModel=AF3, Hyperparams. (k, α)=-2026.03 | 26 |