List your company
ResearchFormerly WizWand

Targeted error generation on TheoremQA Tier-1 (first-20 sweep)

54Targeted Error RateP1 (answer + EA)
52.662.0571.580.95May 27, 2026
Updated 5mo ago

Evaluation results

MethodLinks
P1 (answer + EA)Backbone=o3+4o, GA/EA Retries=unlimited, Verification=human-verified2026.0554
P0 (baseline)Backbone=GPT-5-mini, GA/EA Retries=unlimited, Verification=human-verified2026.0561
P6 (expanded examples)Backbone=o3+4o, GA/EA Retries=unlimited, Verification=human-verified2026.0562
P4 (P3 + textbook (EA))Backbone=GPT-5-mini, GA/EA Retries=unlimited, Verification=human-verified2026.0564
P2 (answer, no EA)Backbone=o3+4o, GA/EA Retries=unlimited, Verification=human-verified2026.0566
P2 (answer, no EA)Backbone=GPT-5-mini, GA/EA Retries=unlimited, Verification=human-verified2026.0568
P6 (expanded examples)Backbone=GPT-5-mini, GA/EA Retries=unlimited, Verification=human-verified2026.0568
P0 (baseline)Backbone=o3+4o, GA/EA Retries=unlimited, Verification=human-verified2026.0569
P5 (hybrid + textbook)Backbone=o3+4o, GA/EA Retries=unlimited, Verification=human-verified2026.0570
P1 (answer + EA)Backbone=GPT-5-mini, GA/EA Retries=unlimited, Verification=human-verified2026.0571
P3 (P1 + feedback)Backbone=GPT-5-mini, GA/EA Retries=unlimited, Verification=human-verified2026.0571
P4 (P3 + textbook (EA))Backbone=o3+4o, GA/EA Retries=unlimited, Verification=human-verified2026.0572
P5 (hybrid + textbook)Backbone=GPT-5-mini, GA/EA Retries=unlimited, Verification=human-verified2026.0572
P8 (fine-tuned EA)Backbone=GPT-5-mini, GA/EA Retries=unlimited, Verification=human-verified2026.0574
P3 (P1 + feedback)Backbone=o3+4o, GA/EA Retries=unlimited, Verification=human-verified2026.0575
P7 (P6 + textbook)Backbone=o3+4o, GA/EA Retries=unlimited, Verification=human-verified2026.0576
P7 (P6 + textbook)Backbone=GPT-5-mini, GA/EA Retries=unlimited, Verification=human-verified2026.0576
P0 (baseline)Backbone=GPT-5, GA/EA Retries=unlimited, Verification=human-verified2026.0577
P5 (hybrid + textbook)Backbone=GPT-5, GA/EA Retries=unlimited, Verification=human-verified2026.0577
P6 (expanded examples)Backbone=GPT-5, GA/EA Retries=unlimited, Verification=human-verified2026.0577
P7 (P6 + textbook)Backbone=GPT-5, GA/EA Retries=unlimited, Verification=human-verified2026.0577
P8 (fine-tuned EA)Backbone=o3+4o, GA/EA Retries=unlimited, Verification=human-verified2026.0579
P2 (answer, no EA)Backbone=GPT-5, GA/EA Retries=unlimited, Verification=human-verified2026.0581
P8 (fine-tuned EA)Backbone=GPT-5, GA/EA Retries=unlimited, Verification=human-verified2026.0583
P4 (P3 + textbook (EA))Backbone=GPT-5, GA/EA Retries=unlimited, Verification=human-verified2026.0586
P1 (answer + EA)Backbone=GPT-5, GA/EA Retries=unlimited, Verification=human-verified2026.0587
P3 (P1 + feedback)Backbone=GPT-5, GA/EA Retries=unlimited, Verification=human-verified2026.0589