Distractor Generation on ARC Easy
24.24P@1GPT-3(few-shot-k-NN)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-3(few-shot-k-NN)Approach=Prompting2026.04 | 24.24 | 8.1 | 18.1 | 29.08 | 32.82 | 0 | 0.17 | |
| GPT-3(few-shot-COT-random)Approach=Prompting2026.04 | 23.7 | 7.92 | 17.77 | 28.67 | 32.44 | 0 | 0 | |
| GPT-3(few-shot-COT-k-NN)Approach=Prompting2026.04 | 23.7 | 7.92 | 17.95 | 28.36 | 31.74 | 0 | 0 | |
| GPT-3(few-shot-random)Approach=Prompting2026.04 | 23.19 | 7.75 | 17.74 | 28.37 | 32.41 | 0 | 0.04 | |
| GPT-3(zero-shot)Approach=Prompting2026.04 | 20.75 | 6.94 | 15.28 | 25.55 | 29.32 | 0 | 0 | |
| TinyLlamaApproach=Instruction Tuning2026.04 | 15.03 | 5.03 | 12.73 | 19.76 | 23.66 | 1.98 | 1.05 | |
| T5(contrast)Approach=Text2Text2026.04 | 13.85 | 4.64 | 10.39 | 17.53 | 20.45 | 6.69 | 3.41 | |
| T5-CSG-DS(beam)Approach=CSG-DS2026.04 | 12.75 | 4.28 | 9.56 | 16.88 | 20.45 | 33.63 | 0.63 | |
| T5(base)Approach=Text2Text2026.04 | 11.24 | 3.77 | 7.38 | 14.05 | 16.3 | 13.97 | 7.7 | |
| T5(multi-task)Approach=Text2Text2026.04 | 10.06 | 3.38 | 6.97 | 12.73 | 14.9 | 13.51 | 5.77 | |
| GPT-3-CSG-DS(clustering)Approach=CSG-DS2026.04 | 9.81 | 3.28 | 5.55 | 13.06 | 16.08 | 0.08 | 0 | |
| T5-CSG-DS(clustering)Approach=CSG-DS2026.04 | 7.95 | 2.68 | 4.5 | 10.67 | 13.16 | 16.16 | 0 |