Spoken Scientific Reasoning on ReveAL-CoT SpeechR 1.0 (test)
74AccuracyVanilla
Evaluation results
| Method | Links | |
|---|---|---|
| VanillaModel=Qwen2.5, Hyperparams. (k, α)=3, 0.0252026.03 | 74 | |
| SGSModel=Qwen2.5, Hyperparams. (k, α)=4, 0.0252026.03 | 71.5 | |
| TGSModel=Qwen2.5, Hyperparams. (k, α)=3, 0.052026.03 | 71.5 | |
| CoTModel=Qwen2.5, Hyperparams. (k, α)=-2026.03 | 71 | |
| SGSModel=Phi4-mm, Hyperparams. (k, α)=3, 0.052026.03 | 70.5 | |
| TGSModel=Phi4-mm, Hyperparams. (k, α)=3, 0.0252026.03 | 70.5 | |
| VanillaModel=Phi4-mm, Hyperparams. (k, α)=5, 0.0252026.03 | 69.5 | |
| CoTModel=Phi4-mm, Hyperparams. (k, α)=-2026.03 | 68.5 | |
| SGSModel=Voxtral, Hyperparams. (k, α)=3, 0.12026.03 | 57 | |
| VanillaModel=Voxtral, Hyperparams. (k, α)=5, 0.0252026.03 | 56 | |
| TGSModel=Voxtral, Hyperparams. (k, α)=4, 0.0252026.03 | 54.5 | |
| NormalModel=Voxtral, Hyperparams. (k, α)=-2026.03 | 53.5 | |
| NormalModel=Qwen2.5, Hyperparams. (k, α)=-2026.03 | 51.5 | |
| VanillaModel=AF3, Hyperparams. (k, α)=3, 0.12026.03 | 47.5 | |
| NormalModel=AF3, Hyperparams. (k, α)=-2026.03 | 46.5 | |
| TGSModel=AF3, Hyperparams. (k, α)=5, 0.12026.03 | 46.5 | |
| CoTModel=AF3, Hyperparams. (k, α)=-2026.03 | 45.5 | |
| CoTModel=Voxtral, Hyperparams. (k, α)=-2026.03 | 43 | |
| SGSModel=AF3, Hyperparams. (k, α)=1, 0.0252026.03 | 42.5 | |
| NormalModel=Phi4-mm, Hyperparams. (k, α)=-2026.03 | 38.5 |