# Distractor Generation on CLOTH Source: [Distractor Generation on CLOTH]() Collection index: [llms.txt]() Data updated: 2026-05-20T08:23:58.932Z Evaluation of Distractor Generation performance on the CLOTH dataset, reporting metrics based on difficulty levels (Hardest, Middle, Easiest). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Distractor Generation | | Dataset (including any stated split) | CLOTH | | Metric | Hardest Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Hardest Accuracy](): Higher is better; unit: PERCENT. - [Middle Accuracy](): Higher is better; unit: PERCENT. - [Easiest Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Hardest Accuracy | Result date | Report paper | | --- | --- | --- | --- | | DCDG + ASDE, DDDE | 73.25 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG + ASDE | 72.14 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG | 70.35 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG + DDDE | 67.97 | Not recorded | 690988873ef7ccd74e5e5cab | | GPT-4o (0-shot) | 56.77 | Not recorded | 690988873ef7ccd74e5e5cab | | GPT-4o (5-shot) | 53.81 | Not recorded | 690988873ef7ccd74e5e5cab | | GPT-4o (0-shot) | 29.6 | Not recorded | 690988873ef7ccd74e5e5cab | | GPT-4o (5-shot) | 24.35 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG + DDDE | 22.69 | Not recorded | 690988873ef7ccd74e5e5cab | | GPT-4o (5-shot) | 21.84 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG | 20.4 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG + ASDE, DDDE | 20.04 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG + ASDE | 19.64 | Not recorded | 690988873ef7ccd74e5e5cab | | GPT-4o (0-shot) | 13.64 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG + DDDE | 9.34 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG | 9.25 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG + ASDE | 8.22 | Not recorded | 690988873ef7ccd74e5e5cab | | DCDG + ASDE, DDDE | 6.71 | Not recorded | 690988873ef7ccd74e5e5cab | ### DCDG + ASDE, DDDE Hardest Accuracy: 73.25. Result ID: 6a0d6f97fbc373592c056bdb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Hard"}. This record does not assert independent reproduction. ### DCDG + ASDE Hardest Accuracy: 72.14. Result ID: 6a0d6f97fbc373592c056bd5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Hard"}. This record does not assert independent reproduction. ### DCDG Hardest Accuracy: 70.35. Result ID: 6a0d6f97fbc373592c056bd2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Hard"}. This record does not assert independent reproduction. ### DCDG + DDDE Hardest Accuracy: 67.97. Result ID: 6a0d6f97fbc373592c056bd8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Hard"}. This record does not assert independent reproduction. ### GPT-4o (0-shot) Hardest Accuracy: 56.77. Result ID: 6a0d6f97fbc373592c056bcc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Hard"}. This record does not assert independent reproduction. ### GPT-4o (5-shot) Hardest Accuracy: 53.81. Result ID: 6a0d6f97fbc373592c056bcf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Hard"}. This record does not assert independent reproduction. ### GPT-4o (0-shot) Hardest Accuracy: 29.6. Result ID: 6a0d6f97fbc373592c056bcb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Easy"}. This record does not assert independent reproduction. ### GPT-4o (5-shot) Hardest Accuracy: 24.35. Result ID: 6a0d6f97fbc373592c056bcd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Original"}. This record does not assert independent reproduction. ### DCDG + DDDE Hardest Accuracy: 22.69. Result ID: 6a0d6f97fbc373592c056bd6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Original"}. This record does not assert independent reproduction. ### GPT-4o (5-shot) Hardest Accuracy: 21.84. Result ID: 6a0d6f97fbc373592c056bce. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Easy"}. This record does not assert independent reproduction. ### DCDG Hardest Accuracy: 20.4. Result ID: 6a0d6f97fbc373592c056bd0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Original"}. This record does not assert independent reproduction. ### DCDG + ASDE, DDDE Hardest Accuracy: 20.04. Result ID: 6a0d6f97fbc373592c056bd9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Original"}. This record does not assert independent reproduction. ### DCDG + ASDE Hardest Accuracy: 19.64. Result ID: 6a0d6f97fbc373592c056bd3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Original"}. This record does not assert independent reproduction. ### GPT-4o (0-shot) Hardest Accuracy: 13.64. Result ID: 6a0d6f97fbc373592c056bca. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Original"}. This record does not assert independent reproduction. ### DCDG + DDDE Hardest Accuracy: 9.34. Result ID: 6a0d6f97fbc373592c056bd7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Easy"}. This record does not assert independent reproduction. ### DCDG Hardest Accuracy: 9.25. Result ID: 6a0d6f97fbc373592c056bd1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Easy"}. This record does not assert independent reproduction. ### DCDG + ASDE Hardest Accuracy: 8.22. Result ID: 6a0d6f97fbc373592c056bd4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Easy"}. This record does not assert independent reproduction. ### DCDG + ASDE, DDDE Hardest Accuracy: 6.71. Result ID: 6a0d6f97fbc373592c056bda. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Easy"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.