Question Answering on PopQA (FAR, AER, Reward)
52.3FAR (Overall)GPT-5 mini
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-5 miniScheme=Pure Eval2026.04 | 52.3 | — | — | 13,768 | 7,205 | 7,205 | 5.8 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=0, Reward (Abstain)=–, Scheme=A2026.04 | 54.2 | 49.1 | 6,528 | 12,403 | 6,089 | 7,739 | 21.3 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=0, Reward (Abstain)=0.6, Scheme=B2026.04 | 54.2 | 43.1 | 8,233.4 | 10,377 | 4,474 | 7,732 | 42.1 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=0, Reward (Abstain)=0.4, Scheme=B2026.04 | 54.7 | 44.5 | 7,380.6 | 10,788 | 4,799 | 7,798 | 38.5 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=-1, Reward (Abstain)=–, Scheme=A2026.04 | 54.8 | 48.2 | -1,381 | 11,982 | 5,771 | 7,824 | 26.2 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=0, Reward (Abstain)=0.2, Scheme=B2026.04 | 54.9 | 44.4 | 6,709.2 | 10,830 | 4,808 | 7,837 | 38.6 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=0, Reward (Abstain)=0.2, Scheme=B w/ norms2026.04 | 55 | 39.9 | 6,573.4 | 9,280 | 3,703 | 7,844 | 52.8 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=-1, Reward (Abstain)=0.4, Scheme=B w/ norms2026.04 | 55 | 34.2 | 5,039.6 | 7,888 | 2,700 | 7,847 | 65.6 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=0, Reward (Abstain)=0.8, Scheme=B2026.04 | 55.1 | 43.5 | 8,941.6 | 10,515 | 4,575 | 7,863 | 41.8 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=0, Reward (Abstain)=0.4, Scheme=B w/ norms2026.04 | 55.3 | 40.4 | 7,566 | 9,501 | 3,841 | 7,896 | 51.4 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=-1, Reward (Abstain)=0.4, Scheme=B2026.04 | 55.5 | 41 | 3,577.2 | 9,683 | 3,969 | 7,921 | 49.9 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=-1, Reward (Abstain)=0.6, Scheme=B2026.04 | 55.6 | 41.2 | 4,477.8 | 9,633 | 3,967 | 7,932 | 50 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=-1, Reward (Abstain)=0.2, Scheme=B2026.04 | 55.8 | 41 | 2,655 | 9,704 | 3,979 | 7,956 | 50 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=-1, Reward (Abstain)=0.8, Scheme=B2026.04 | 55.9 | 39.2 | 6,076.8 | 9,124 | 3,579 | 7,971 | 55.1 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=-1, Reward (Abstain)=0.6, Scheme=B w/ norms2026.04 | 56.1 | 36.5 | 5,824.6 | 8,282 | 3,021 | 8,001 | 62.2 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=-1, Reward (Abstain)=0.2, Scheme=B w/ norms2026.04 | 57.5 | 35.5 | 3,589.8 | 8,253 | 2,933 | 8,210 | 64.3 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=0, Reward (Abstain)=0.6, Scheme=B w/ norms2026.04 | 57.8 | 37.9 | 8,744.8 | 8,774 | 3,325 | 8,244 | 59.7 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=0, Reward (Abstain)=0.8, Scheme=B w/ norms2026.04 | 57.8 | 40.4 | 9,487.2 | 9,428 | 3,812 | 8,274 | 53.8 | |
| GPT-5 miniReward (Correct)=1, Penalty (Wrong)=-1, Reward (Abstain)=0.8, Scheme=B w/ norms2026.04 | 58.1 | 37 | 6,815.6 | 8,510 | 3,150 | 8,294 | 62 |