Question Answering on ARC (test) with Throughput Metrics
88.59AccuracyFDM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FDMBackbone=LLaDA-1.5, Search Width (K)=42025.12 | 88.59 | 4.56 | |
| FDMBackbone=LLaDA-1.5, Search Width (K)=32025.12 | 88.45 | 5.59 | |
| FDMBackbone=LLaDA-1.5, Search Width (K)=22025.12 | 88.18 | 7.71 | |
| MarginBackbone=LLaDA-1.5, Step Number (T)=2562025.12 | 87.44 | 10.82 | |
| EntropyBackbone=LLaDA-1.5, Step Number (T)=2562025.12 | 87.38 | 10.38 | |
| ProbabilityBackbone=LLaDA-1.5, Step Number (T)=2562025.12 | 87.06 | 10.98 | |
| FDMBackbone=LLaDA-8B-Instruct, Search Width (K)=42025.12 | 86.68 | 4.58 | |
| FDMBackbone=LLaDA-8B-Instruct, Search Width (K)=32025.12 | 86.46 | 5.58 | |
| FDMBackbone=LLaDA-8B-Instruct, Search Width (K)=22025.12 | 86 | 7.72 | |
| FDMBackbone=LLaDA-MoE-7B-Instruct, Search Width (K)=42025.12 | 83.51 | 3.64 | |
| FDMBackbone=LLaDA-MoE-7B-Instruct, Search Width (K)=32025.12 | 83.45 | 3.73 | |
| FDMBackbone=LLaDA-MoE-7B-Instruct, Search Width (K)=22025.12 | 82.89 | 3.85 | |
| MarginBackbone=LLaDA-8B-Instruct, Step Number (T)=2562025.12 | 82.55 | 10.85 | |
| ProbabilityBackbone=LLaDA-8B-Instruct, Step Number (T)=2562025.12 | 80.96 | 10.96 | |
| EntropyBackbone=LLaDA-8B-Instruct, Step Number (T)=2562025.12 | 78.13 | 10.43 | |
| ProbabilityBackbone=LLaDA-MoE-7B-Instruct, Step Number (T)=2562025.12 | 76.19 | 4.01 | |
| MarginBackbone=LLaDA-MoE-7B-Instruct, Step Number (T)=2562025.12 | 75.85 | 4.01 | |
| EntropyBackbone=LLaDA-MoE-7B-Instruct, Step Number (T)=2562025.12 | 71.84 | 3.98 | |
| FDMBackbone=MMaDA-8B-MixCoT, Search Width (K)=42025.12 | 59.76 | 4.37 | |
| FDMBackbone=MMaDA-8B-MixCoT, Search Width (K)=32025.12 | 59.61 | 5.58 | |
| FDMBackbone=MMaDA-8B-MixCoT, Search Width (K)=22025.12 | 59.43 | 7.68 | |
| EntropyBackbone=MMaDA-8B-MixCoT, Step Number (T)=2562025.12 | 58.33 | 10.18 | |
| ProbabilityBackbone=MMaDA-8B-MixCoT, Step Number (T)=2562025.12 | 58.26 | 10.94 | |
| MarginBackbone=MMaDA-8B-MixCoT, Step Number (T)=2562025.12 | 56.37 | 10.62 |