Counterfactual Text Generation on CLASSE
57.5Sim ScoreGumbel Machine
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Gumbel MachineModel=Qwen3-4B-Instruct2026.05 | 57.5 | 56.6 | |
| Identify-and-ReplaceModel=Claude Sonnet 4.62026.05 | 55.4 | 43.4 | |
| Identify-and-ReplaceModel=LLaMA-3.1-8B-Instruct2026.05 | 49.8 | 10.3 | |
| Identify-and-ReplaceModel=Claude Haiku 4.52026.05 | 45.1 | 38.8 | |
| Gumbel MachineModel=LLaMA-3.1-8B-Instruct2026.05 | 39.1 | 66.8 | |
| Gumbel MachineModel=LLaMA-3.1-8B-Instruct2026.05 | 39.1 | 66.8 | |
| Minimal-EditModel=Claude Sonnet 4.62026.05 | 37.6 | 34.4 | |
| In-Context LearningModel=Claude Sonnet 4.62026.05 | 35.9 | 60.4 | |
| Minimal-EditModel=LLaMA-3.1-8B-Instruct2026.05 | 31.4 | 18.3 | |
| In-Context LearningModel=LLaMA-3.1-8B-Instruct2026.05 | 30.8 | 32.7 | |
| In-Context LearningModel=Claude Haiku 4.52026.05 | 28.1 | 31.2 | |
| Minimal-EditModel=Claude Haiku 4.52026.05 | 28 | 14.4 |