Counterfactual Text Generation on DREsS
59.3Similarity ScoreGumbel Machine
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gumbel MachineModel=Qwen3-4B-Instruct2026.05 | 59.3 | 31.5 | |
| Identify-and-ReplaceModel=Claude Sonnet 4.62026.05 | 47.7 | 40.5 | |
| Gumbel MachineModel=LLaMA-3.1-8B-Instruct2026.05 | 47.7 | 40.4 | |
| Gumbel MachineModel=LLaMA-3.1-8B-Instruct2026.05 | 47.7 | 40.4 | |
| In-Context LearningModel=Claude Sonnet 4.62026.05 | 41.8 | 48.4 | |
| Identify-and-ReplaceModel=Claude Haiku 4.52026.05 | 38.4 | 44.1 | |
| Minimal-EditModel=Claude Sonnet 4.62026.05 | 37.5 | 43.9 | |
| In-Context LearningModel=LLaMA-3.1-8B-Instruct2026.05 | 33 | 26.4 | |
| In-Context LearningModel=Claude Haiku 4.52026.05 | 31.4 | 12.5 | |
| Minimal-EditModel=Claude Haiku 4.52026.05 | 31.2 | 9.3 | |
| Identify-and-ReplaceModel=LLaMA-3.1-8B-Instruct2026.05 | 30.8 | 32.5 | |
| Minimal-EditModel=LLaMA-3.1-8B-Instruct2026.05 | 30.5 | 18.8 |