# Theorem Question Answering on ThmQA Source: [Theorem Question Answering on ThmQA]() Collection index: [llms.txt]() Data updated: 2026-05-27T07:51:21.337Z Evaluation of model performance on the ThmQA dataset for Theorem Question Answering tasks, measured by pass@1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Theorem Question Answering | | Dataset (including any stated split) | ThmQA | | Metric | Pass@1 | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 15 of 15 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: SCALAR. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | Teacher | 26.12 | Not recorded | 6a167aa1dbd7150506f80002 | | Bridge-Garden (Skew FKL) | 5.24 | Not recorded | 6a167aa1dbd7150506f80002 | | Skew FKL | 5.01 | Not recorded | 6a167aa1dbd7150506f80002 | | Bridge-Garden (Reverse KL) | 4.57 | Not recorded | 6a167aa1dbd7150506f80002 | | Bridge-Garden (α-β divergence) | 4.54 | Not recorded | 6a167aa1dbd7150506f80002 | | α-β divergence | 4.34 | Not recorded | 6a167aa1dbd7150506f80002 | | Forward KL | 4.25 | Not recorded | 6a167aa1dbd7150506f80002 | | Bridge-Garden (Forward KL) | 4.22 | Not recorded | 6a167aa1dbd7150506f80002 | | Bridge-Garden (Total Variation) | 4 | Not recorded | 6a167aa1dbd7150506f80002 | | Bridge-Garden (Skew RKL) | 3.41 | Not recorded | 6a167aa1dbd7150506f80002 | | Skew RKL | 3.38 | Not recorded | 6a167aa1dbd7150506f80002 | | Hard KD | 3.15 | Not recorded | 6a167aa1dbd7150506f80002 | | Total Variation | 1.29 | Not recorded | 6a167aa1dbd7150506f80002 | | Reverse KL | 1.26 | Not recorded | 6a167aa1dbd7150506f80002 | | Student (No Distill) | 0.95 | Not recorded | 6a167aa1dbd7150506f80002 | ### Teacher Pass@1: 26.12. Result ID: 6a16a277fbc373592c0f8a6d. [Reporting paper]() Recorded evaluation configuration: {"Model":"Gemma3-4B"}. This record does not assert independent reproduction. ### Bridge-Garden (Skew FKL) Pass@1: 5.24. Result ID: 6a16a277fbc373592c0f8a77. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distillation Family":"Distance & Skewed Family"}. This record does not assert independent reproduction. ### Skew FKL Pass@1: 5.01. Result ID: 6a16a277fbc373592c0f8a76. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distillation Family":"Distance & Skewed Family"}. This record does not assert independent reproduction. ### Bridge-Garden (Reverse KL) Pass@1: 4.57. Result ID: 6a16a277fbc373592c0f8a73. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distillation Family":"KL Divergence Family"}. This record does not assert independent reproduction. ### Bridge-Garden (α-β divergence) Pass@1: 4.54. Result ID: 6a16a277fbc373592c0f8a7b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distillation Family":"Unified Divergence Family"}. This record does not assert independent reproduction. ### α-β divergence Pass@1: 4.34. Result ID: 6a16a277fbc373592c0f8a7a. [Reporting paper]() Recorded evaluation configuration: {"Distillation Family":"Unified Divergence Family"}. This record does not assert independent reproduction. ### Forward KL Pass@1: 4.25. Result ID: 6a16a277fbc373592c0f8a70. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distillation Family":"KL Divergence Family"}. This record does not assert independent reproduction. ### Bridge-Garden (Forward KL) Pass@1: 4.22. Result ID: 6a16a277fbc373592c0f8a71. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distillation Family":"KL Divergence Family"}. This record does not assert independent reproduction. ### Bridge-Garden (Total Variation) Pass@1: 4. Result ID: 6a16a277fbc373592c0f8a75. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distillation Family":"Distance & Skewed Family"}. This record does not assert independent reproduction. ### Bridge-Garden (Skew RKL) Pass@1: 3.41. Result ID: 6a16a277fbc373592c0f8a79. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distillation Family":"Distance & Skewed Family"}. This record does not assert independent reproduction. ### Skew RKL Pass@1: 3.38. Result ID: 6a16a277fbc373592c0f8a78. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Distillation Family":"Distance & Skewed Family"}. This record does not assert independent reproduction. ### Hard KD Pass@1: 3.15. Result ID: 6a16a277fbc373592c0f8a6f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Teacher Model":"Gemma3-4B","Student Model":"Gemma3-1B"}. This record does not assert independent reproduction. ### Total Variation Pass@1: 1.29. Result ID: 6a16a277fbc373592c0f8a74. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Distillation Family":"Distance & Skewed Family"}. This record does not assert independent reproduction. ### Reverse KL Pass@1: 1.26. Result ID: 6a16a277fbc373592c0f8a72. [Reporting paper]() Recorded evaluation configuration: {"Distillation Family":"KL Divergence Family"}. This record does not assert independent reproduction. ### Student (No Distill) Pass@1: 0.95. Result ID: 6a16a277fbc373592c0f8a6e. [Reporting paper]() Recorded evaluation configuration: {"Model":"Gemma3-1B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.