Distractor Generation on ARC Challenge
21.08P@1GPT-3(few-shot-random)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-3(few-shot-random)Approach=Prompting2026.04 | 21.08 | 7.07 | 14.96 | 24.82 | 27.75 | 0.26 | 0.17 | |
| GPT-3(few-shot-k-NN)Approach=Prompting2026.04 | 20.48 | 6.87 | 15.05 | 24.06 | 26.8 | 0.17 | 0.09 | |
| GPT-3(few-shot-COT-random)Approach=Prompting2026.04 | 20.31 | 6.81 | 15.29 | 24.23 | 27.23 | 0.34 | 0.09 | |
| GPT-3(few-shot-COT-k-NN)Approach=Prompting2026.04 | 19.88 | 6.67 | 14.6 | 23.25 | 25.73 | 0.17 | 0.09 | |
| GPT-3(zero-shot)Approach=Prompting2026.04 | 17.41 | 5.85 | 11.96 | 20.31 | 22.53 | 0.09 | 0.09 | |
| TinyLlamaApproach=Instruction Tuning2026.04 | 12.37 | 4.15 | 8.6 | 15.15 | 17.55 | 1.37 | 1.11 | |
| T5(contrast)Approach=Text2Text2026.04 | 9.13 | 3.09 | 5.47 | 11.26 | 13.06 | 9.64 | 3.07 | |
| GPT-3-CSG-DS(clustering)Approach=CSG-DS2026.04 | 8.7 | 2.92 | 4 | 10.75 | 12.68 | 0 | 0 | |
| T5-CSG-DS(beam)Approach=CSG-DS2026.04 | 7.85 | 2.67 | 6.5 | 11.46 | 14.71 | 31.83 | 0.26 | |
| T5(base)Approach=Text2Text2026.04 | 7.17 | 2.45 | 3.68 | 8.79 | 10.14 | 19.2 | 2.9 | |
| T5-CSG-DS(clustering)Approach=CSG-DS2026.04 | 5.46 | 1.86 | 2.23 | 7.35 | 9.18 | 12.97 | 0 | |
| T5(multi-task)Approach=Text2Text2026.04 | 5.38 | 1.85 | 3.63 | 7.21 | 8.81 | 18.43 | 2.39 |