Science Question Answering on GPQA (Accuracy and Majority Voting)
55.8AccuracyUAB
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| UABBackbone Model=Cohere, Budget=N=42026.05 | 55.8 | — | — | — | |
| UniformBackbone Model=Cohere, Budget=N=42026.05 | 55 | — | — | — | |
| LLM-JudgeBackbone Model=Cohere, Budget=N=42026.05 | 53.9 | — | — | — | |
| N=1Backbone Model=Cohere, Budget=N=12026.05 | 52.2 | — | — | — | |
| UABBackbone Model=Gemma3-27B, Budget=N=42026.05 | 49.7 | — | — | — | |
| UniformBackbone Model=Gemma3-27B, Budget=N=42026.05 | 49 | — | — | — | |
| LLM-JudgeBackbone Model=Gemma3-27B, Budget=N=42026.05 | 48.8 | — | — | — | |
| N=1Backbone Model=Gemma3-27B, Budget=N=12026.05 | 40.9 | — | — | — | |
| RLTR2026.02 | 34.8 | 37.3 | 37.5 | 37.7 | |
| HSIR-DPOIteration=3, Backbone=Qwen2.5-7B, Training Dataset=MedQA2026.05 | 34.6 | — | — | — | |
| RLVR2026.02 | 33 | 35.2 | 36.8 | 37 | |
| BaseBackbone=Qwen2.5-7B, Temperature (τ)=0.102026.02 | 32.83 | — | — | — | |
| Base2026.02 | 32.4 | 35.3 | 35 | 35.2 | |
| Top-KBackbone=Qwen2.5-7B, Temperature (τ)=0.902026.02 | 32.32 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.90, β=0.01, λ=0.12026.02 | 32.32 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.10, β=0.01, λ=0.12026.02 | 31.82 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.10, β=0.02, λ=0.22026.02 | 31.82 | — | — | — | |
| HSIR-DPOIteration=2, Backbone=Qwen2.5-7B, Training Dataset=MedQA2026.05 | 31.7 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.50, β=0.02, λ=0.22026.02 | 30.3 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.10, β=0.05, λ=0.52026.02 | 30.3 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.25, β=0.05, λ=0.52026.02 | 30.3 | — | — | — | |
| Top-KBackbone=Qwen2.5-7B, Temperature (τ)=0.252026.02 | 29.8 | — | — | — | |
| Top-KBackbone=Qwen2.5-7B, Temperature (τ)=0.702026.02 | 29.8 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.25, β=0.01, λ=0.12026.02 | 29.8 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.50, β=0.01, λ=0.12026.02 | 29.8 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.50, β=0.05, λ=0.52026.02 | 29.8 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.70, β=0.01, λ=0.12026.02 | 29.29 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.25, β=0.02, λ=0.22026.02 | 29.29 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.90, β=0.02, λ=0.22026.02 | 29.29 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.70, β=0.05, λ=0.52026.02 | 29.29 | — | — | — | |
| HSIR-DPOIteration=1, Backbone=Qwen2.5-7B, Training Dataset=MedQA2026.05 | 28.79 | — | — | — | |
| Top-KBackbone=Qwen2.5-7B, Temperature (τ)=0.102026.02 | 28.28 | — | — | — | |
| BaseBackbone=Qwen2.5-7B, Temperature (τ)=0.252026.02 | 27.78 | — | — | — | |
| Top-KBackbone=Qwen2.5-7B, Temperature (τ)=0.502026.02 | 27.78 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.70, β=0.02, λ=0.22026.02 | 27.78 | — | — | — | |
| CoDDDecoding Strategy=Random, Diffusion Steps=2562026.02 | 26.34 | — | — | — | |
| CoDDDecoding Strategy=Margin, Diffusion Steps=2562026.02 | 25.89 | — | — | — | |
| BaseBackbone=Qwen2.5-7B, Temperature (τ)=0.702026.02 | 25.76 | — | — | — | |
| LMNetBackbone=Qwen2.5-0.5B2025.05 | 25.6 | — | — | — | |
| Self-ConsistencyBackbone=Qwen2.5-0.5B, Samples=162025.05 | 25.2 | — | — | — | |
| PromptBackbone=Qwen2.5-0.5B2025.05 | 24.8 | — | — | — | |
| CoDDDecoding Strategy=Low Confidence, Diffusion Steps=2562026.02 | 24.55 | — | — | — | |
| BaseBackbone=Qwen2.5-7B, Temperature (τ)=0.902026.02 | 24.24 | — | — | — | |
| BoKBackbone=Qwen2.5-7B, Temperature (τ)=0.90, β=0.05, λ=0.52026.02 | 24.24 | — | — | — | |
| UABBackbone Model=GPT-OSS-20B, Budget=N=42026.05 | 23.7 | — | — | — | |
| IRPOIteration=1, Backbone=Qwen2.5-7B, Training Dataset=MedQA2026.05 | 23.66 | — | — | — | |
| Self-RefineBackbone=Qwen2.5-0.5B, Passes=162025.05 | 23.5 | — | — | — | |
| LLM-JudgeBackbone Model=GPT-OSS-20B, Budget=N=42026.05 | 23.2 | — | — | — | |
| IRPOIteration=2, Backbone=Qwen2.5-7B, Training Dataset=MedQA2026.05 | 22.55 | — | — | — | |
| IRPOIteration=3, Backbone=Qwen2.5-7B, Training Dataset=MedQA2026.05 | 22.54 | — | — | — | |
| LLaDADecoding Strategy=Random, Diffusion Steps=2562026.02 | 22.1 | — | — | — | |
| LLaDADecoding Strategy=Margin, Diffusion Steps=2562026.02 | 22.1 | — | — | — | |
| UniformBackbone Model=GPT-OSS-20B, Budget=N=42026.05 | 22.1 | — | — | — | |
| SFTBackbone=Qwen2.5-0.5B2025.05 | 22 | — | — | — | |
| BaseBackbone=Qwen2.5-7B, Temperature (τ)=0.502026.02 | 21.72 | — | — | — | |
| CoDDDecoding Strategy=Random, Diffusion Steps=1282026.02 | 21.65 | — | — | — | |
| LLaDADecoding Strategy=Low Confidence, Diffusion Steps=2562026.02 | 21.43 | — | — | — | |
| LLaDADecoding Strategy=Random, Diffusion Steps=1282026.02 | 19.2 | — | — | — | |
| CoDDDecoding Strategy=Margin, Diffusion Steps=1282026.02 | 18.53 | — | — | — | |
| LLaDADecoding Strategy=Low Confidence, Diffusion Steps=1282026.02 | 17.86 | — | — | — | |
| CoDDDecoding Strategy=Low Confidence, Diffusion Steps=1282026.02 | 17.86 | — | — | — | |
| LLaDADecoding Strategy=Margin, Diffusion Steps=1282026.02 | 17.19 | — | — | — | |
| N=1Backbone Model=GPT-OSS-20B, Budget=N=12026.05 | 16.5 | — | — | — | |
| CoDDDecoding Strategy=Random, Diffusion Steps=642026.02 | 11.38 | — | — | — | |
| CoDDDecoding Strategy=Margin, Diffusion Steps=642026.02 | 10.94 | — | — | — | |
| LLaDADecoding Strategy=Random, Diffusion Steps=642026.02 | 10.71 | — | — | — | |
| CoDDDecoding Strategy=Low Confidence, Diffusion Steps=642026.02 | 9.15 | — | — | — | |
| LLaDADecoding Strategy=Margin, Diffusion Steps=642026.02 | 8.93 | — | — | — | |
| LLaDADecoding Strategy=Low Confidence, Diffusion Steps=642026.02 | 8.04 | — | — | — |