Graduate-level Q&A on GPQA 0-shot
27.5GPQA Accuracy (0-shot)AR SFT
Evaluation results
| Method | Links | |
|---|---|---|
| AR SFTModel Scale=7B, One-token mode (τ=1.0)=true2026.04 | 27.5 | |
| MARS-7BModel Scale=7B, Block Size (B)=4, One-token mode (τ=1.0)=true2026.04 | 26.6 | |
| MARS-0.5BModel Scale=0.5B, Block Size (B)=4, SFT Loss Configuration=True, One-token mode (τ=1.0)=true2026.04 | 19.4 | |
| MARS-0.5B w/o SFT lossModel Scale=0.5B, Block Size (B)=4, SFT Loss Configuration=False, One-token mode (τ=1.0)=true2026.04 | 19 | |
| AR SFT (5 ep)Model Scale=0.5B, Training Epochs=5, One-token mode (τ=1.0)=true2026.04 | 17.9 | |
| Block DiffusionModel Scale=0.5B, Block Size (B)=4, One-token mode (τ=1.0)=true2026.04 | 17.9 | |
| AR SFT (10 ep)Model Scale=0.5B, Training Epochs=10, One-token mode (τ=1.0)=true2026.04 | 14.1 | |
| Base (no fine-tune)Model Scale=0.5B, One-token mode (τ=1.0)=true2026.04 | 13.6 | |
| Base (no fine-tune)Model Scale=7B, One-token mode (τ=1.0)=true2026.04 | 11.6 |