# Targeted error generation on TheoremQA Tier-1 (first-20 sweep) Source: [Targeted error generation on TheoremQA Tier-1 (first-20 sweep)]() Collection index: [llms.txt]() Data updated: 2026-05-29T07:42:46.621Z Evaluation of model capability for targeted error generation specifically on the Tier-1 subset (first 20 samples) of the TheoremQA dataset, measured by the average targeted-error rate. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Targeted error generation | | Dataset (including any stated split) | TheoremQA Tier-1 (first-20 sweep) | | Metric | Targeted Error Rate | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Targeted Error Rate](): Lower is better; unit: PERCENT. ## Recorded results | Method | Targeted Error Rate | Result date | Report paper | | --- | --- | --- | --- | | P1 (answer + EA) | 54 | Not recorded | 6a191d9edbd7150506f82806 | | P0 (baseline) | 61 | Not recorded | 6a191d9edbd7150506f82806 | | P6 (expanded examples) | 62 | Not recorded | 6a191d9edbd7150506f82806 | | P4 (P3 + textbook (EA)) | 64 | Not recorded | 6a191d9edbd7150506f82806 | | P2 (answer, no EA) | 66 | Not recorded | 6a191d9edbd7150506f82806 | | P2 (answer, no EA) | 68 | Not recorded | 6a191d9edbd7150506f82806 | | P6 (expanded examples) | 68 | Not recorded | 6a191d9edbd7150506f82806 | | P0 (baseline) | 69 | Not recorded | 6a191d9edbd7150506f82806 | | P5 (hybrid + textbook) | 70 | Not recorded | 6a191d9edbd7150506f82806 | | P1 (answer + EA) | 71 | Not recorded | 6a191d9edbd7150506f82806 | | P3 (P1 + feedback) | 71 | Not recorded | 6a191d9edbd7150506f82806 | | P4 (P3 + textbook (EA)) | 72 | Not recorded | 6a191d9edbd7150506f82806 | | P5 (hybrid + textbook) | 72 | Not recorded | 6a191d9edbd7150506f82806 | | P8 (fine-tuned EA) | 74 | Not recorded | 6a191d9edbd7150506f82806 | | P3 (P1 + feedback) | 75 | Not recorded | 6a191d9edbd7150506f82806 | | P7 (P6 + textbook) | 76 | Not recorded | 6a191d9edbd7150506f82806 | | P7 (P6 + textbook) | 76 | Not recorded | 6a191d9edbd7150506f82806 | | P0 (baseline) | 77 | Not recorded | 6a191d9edbd7150506f82806 | | P5 (hybrid + textbook) | 77 | Not recorded | 6a191d9edbd7150506f82806 | | P6 (expanded examples) | 77 | Not recorded | 6a191d9edbd7150506f82806 | | P7 (P6 + textbook) | 77 | Not recorded | 6a191d9edbd7150506f82806 | | P8 (fine-tuned EA) | 79 | Not recorded | 6a191d9edbd7150506f82806 | | P2 (answer, no EA) | 81 | Not recorded | 6a191d9edbd7150506f82806 | | P8 (fine-tuned EA) | 83 | Not recorded | 6a191d9edbd7150506f82806 | | P4 (P3 + textbook (EA)) | 86 | Not recorded | 6a191d9edbd7150506f82806 | | P1 (answer + EA) | 87 | Not recorded | 6a191d9edbd7150506f82806 | | P3 (P1 + feedback) | 89 | Not recorded | 6a191d9edbd7150506f82806 | ### P1 (answer + EA) Targeted Error Rate: 54. Result ID: 6a1942aafbc373592c1351a0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"o3+4o","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P0 (baseline) Targeted Error Rate: 61. Result ID: 6a1942aafbc373592c13519f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5-mini","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P6 (expanded examples) Targeted Error Rate: 62. Result ID: 6a1942aafbc373592c1351af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"o3+4o","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P4 (P3 + textbook (EA)) Targeted Error Rate: 64. Result ID: 6a1942aafbc373592c1351ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5-mini","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P2 (answer, no EA) Targeted Error Rate: 66. Result ID: 6a1942aafbc373592c1351a3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"o3+4o","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P2 (answer, no EA) Targeted Error Rate: 68. Result ID: 6a1942aafbc373592c1351a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5-mini","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P6 (expanded examples) Targeted Error Rate: 68. Result ID: 6a1942aafbc373592c1351b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5-mini","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P0 (baseline) Targeted Error Rate: 69. Result ID: 6a1942aafbc373592c13519d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"o3+4o","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P5 (hybrid + textbook) Targeted Error Rate: 70. Result ID: 6a1942aafbc373592c1351ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"o3+4o","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P1 (answer + EA) Targeted Error Rate: 71. Result ID: 6a1942aafbc373592c1351a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5-mini","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P3 (P1 + feedback) Targeted Error Rate: 71. Result ID: 6a1942aafbc373592c1351a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5-mini","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P4 (P3 + textbook (EA)) Targeted Error Rate: 72. Result ID: 6a1942aafbc373592c1351a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"o3+4o","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P5 (hybrid + textbook) Targeted Error Rate: 72. Result ID: 6a1942aafbc373592c1351ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5-mini","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P8 (fine-tuned EA) Targeted Error Rate: 74. Result ID: 6a1942aafbc373592c1351b7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5-mini","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P3 (P1 + feedback) Targeted Error Rate: 75. Result ID: 6a1942aafbc373592c1351a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"o3+4o","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P7 (P6 + textbook) Targeted Error Rate: 76. Result ID: 6a1942aafbc373592c1351b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"o3+4o","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P7 (P6 + textbook) Targeted Error Rate: 76. Result ID: 6a1942aafbc373592c1351b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5-mini","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P0 (baseline) Targeted Error Rate: 77. Result ID: 6a1942aafbc373592c13519e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P5 (hybrid + textbook) Targeted Error Rate: 77. Result ID: 6a1942aafbc373592c1351ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P6 (expanded examples) Targeted Error Rate: 77. Result ID: 6a1942aafbc373592c1351b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P7 (P6 + textbook) Targeted Error Rate: 77. Result ID: 6a1942aafbc373592c1351b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P8 (fine-tuned EA) Targeted Error Rate: 79. Result ID: 6a1942aafbc373592c1351b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"o3+4o","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P2 (answer, no EA) Targeted Error Rate: 81. Result ID: 6a1942aafbc373592c1351a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P8 (fine-tuned EA) Targeted Error Rate: 83. Result ID: 6a1942aafbc373592c1351b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P4 (P3 + textbook (EA)) Targeted Error Rate: 86. Result ID: 6a1942aafbc373592c1351aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P1 (answer + EA) Targeted Error Rate: 87. Result ID: 6a1942aafbc373592c1351a1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. ### P3 (P1 + feedback) Targeted Error Rate: 89. Result ID: 6a1942aafbc373592c1351a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5","GA/EA Retries":"unlimited","Verification":"human-verified"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.