Counterfactual Generation on SIB200
86.7SLFRGemma3-12B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Gemma3-12BLanguage=ar, Protocol=DG-CF2026.05 | 86.7 | 78.9 | 25.48 | 52.1 | 0.467 | 0.533 | |
| MACROModel=Gemma3-12B, Language=avg2026.05 | 85 | — | 26.3 | 57 | — | — | |
| Gemma3-12BLanguage=zh, Protocol=DG-CF2026.05 | 84.2 | 78.9 | 31.61 | 55.1 | 0.554 | 0.446 | |
| Gemma3-12BLanguage=en, Protocol=DG-CF2026.05 | 83 | 76.3 | 53.9 | 53 | 0.471 | 0.529 | |
| SFTModel=Gemma3-12B, Language=avg2026.05 | 83 | — | 23.6 | 58 | — | — | |
| Gemma3-12BLanguage=zh, Protocol=TB-CF2026.05 | 82.8 | 57.6 | 16.96 | 7.3 | 22.704 | -21.704 | |
| Gemma3-12BLanguage=sw, Protocol=TB-CF2026.05 | 82.8 | 69.7 | 12.45 | 39.9 | 7.464 | -6.464 | |
| Gemma3-12BLanguage=ar, Protocol=TB-CF2026.05 | 81.8 | 55.6 | 25.19 | 6.3 | 8.081 | -7.081 | |
| Gemma3-12BLanguage=sw, Protocol=DG-CF2026.05 | 81.4 | 73.3 | 14.86 | 65.3 | 0.446 | 0.554 | |
| DG-CFModel=Gemma3-12B, Language=avg2026.05 | 81 | — | 27.6 | 57 | — | — | |
| MACROModel=Gemma3-4B, Language=zh2026.05 | 80 | — | 30.1 | 66 | — | — | |
| Gemma3-12BLanguage=vi, Protocol=DG-CF2026.05 | 79.4 | 70.9 | 18.54 | 55.7 | 0.508 | 0.492 | |
| Gemma3-12BLanguage=de, Protocol=TB-CF2026.05 | 78.8 | 66.7 | 13.32 | 23.9 | 7.754 | -6.754 | |
| Gemma3-12BLanguage=ru, Protocol=DG-CF2026.05 | 77.9 | 69.1 | 21.03 | 58.5 | 0.494 | 0.506 | |
| Qwen3-4BLanguage=zh, Protocol=TB-CF2026.05 | 76.8 | 37.4 | 44.39 | 7.8 | 1.211 | -0.211 | |
| Qwen3-4BLanguage=ar, Protocol=TB-CF2026.05 | 76.8 | 37.4 | 40.49 | 8.4 | 0.941 | 0.059 | |
| Gemma3-12BLanguage=vi, Protocol=TB-CF2026.05 | 76.8 | 65.7 | 12.51 | 25.5 | 8.144 | -7.144 | |
| Qwen3-4BLanguage=zh, Protocol=DG-CF2026.05 | 76.7 | 72.4 | 43.75 | 59.8 | 0.438 | 0.562 | |
| Gemma3-12BLanguage=de, Protocol=DG-CF2026.05 | 76.7 | 70.3 | 28.13 | 57.4 | 0.472 | 0.528 | |
| SFTModel=Gemma3-4B, Language=zh2026.05 | 76 | — | 28.5 | 65 | — | — | |
| Qwen3-8BLanguage=ar, Protocol=TB-CF2026.05 | 75.8 | 40.4 | 34.94 | 8 | 0.951 | 0.049 | |
| Gemma3-12BLanguage=ru, Protocol=TB-CF2026.05 | 75.8 | 62.6 | 15.23 | 24 | 7.769 | -6.769 | |
| SFTModel=Gemma3-4B, Language=ar2026.05 | 74 | — | 24.6 | 60 | — | — | |
| MACROModel=Gemma3-4B, Language=sw2026.05 | 74 | — | 17.6 | 67 | — | — | |
| Gemma3-4BLanguage=de, Protocol=TB-CF2026.05 | 73.7 | 41.4 | 11.95 | 26.7 | 7.64 | -6.64 | |
| SFTModel=Gemma3-4B, Language=sw2026.05 | 72 | — | 18.2 | 69 | — | — | |
| SFTModel=Gemma3-4B, Language=ru2026.05 | 72 | — | 19.3 | 70 | — | — | |
| Qwen3-8BLanguage=zh, Protocol=TB-CF2026.05 | 71.7 | 40.4 | 34.98 | 8.3 | 1.235 | -0.235 | |
| SFTModel=Gemma3-4B, Language=avg2026.05 | 71 | — | 23.2 | 65 | — | — | |
| SFTModel=Qwen3-8B, Language=avg2026.05 | 71 | — | 24.4 | 66 | — | — | |
| Qwen3-4BLanguage=de, Protocol=DG-CF2026.05 | 70.9 | 61.2 | 45.15 | 65.8 | 0.306 | 0.694 | |
| SFTModel=Gemma3-4B, Language=en2026.05 | 70 | — | 31.4 | 55 | — | — | |
| MACROModel=Qwen3-8B, Language=avg2026.05 | 70 | — | 27.9 | 70 | — | — | |
| Gemma3-4BLanguage=zh, Protocol=TB-CF2026.05 | 68.7 | 31.3 | 15.68 | 7.4 | 22.77 | -21.77 | |
| Gemma3-4BLanguage=sw, Protocol=TB-CF2026.05 | 68.7 | 42.4 | 13.59 | 42.7 | 7.62 | -6.62 | |
| Qwen3-8BLanguage=vi, Protocol=DG-CF2026.05 | 68.1 | 61.4 | 22.66 | 69.2 | 0.308 | 0.692 | |
| DG-CFModel=Gemma3-4B, Language=sw2026.05 | 68 | — | 18.7 | 68 | — | — | |
| MACROModel=Gemma3-4B, Language=avg2026.05 | 68 | — | 24 | 69 | — | — | |
| Qwen3-4BLanguage=vi, Protocol=DG-CF2026.05 | 67.8 | 57.8 | 26.15 | 64.9 | 0.32 | 0.68 | |
| Gemma3-4BLanguage=vi, Protocol=TB-CF2026.05 | 67.7 | 43.4 | 10.8 | 31.7 | 7.959 | -6.959 | |
| Gemma3-4BLanguage=ru, Protocol=TB-CF2026.05 | 67.7 | 40.4 | 12.04 | 28.8 | 8.039 | -7.039 | |
| Gemma3-4BLanguage=sw, Protocol=DG-CF2026.05 | 67.6 | 54.9 | 18.71 | 68.1 | 0.446 | 0.554 | |
| SFTModel=Qwen3-4B, Language=avg2026.05 | 67 | — | 26.1 | 68 | — | — | |
| Gemma3-4BLanguage=zh, Protocol=DG-CF2026.05 | 66.2 | 62.9 | 35.09 | 69.2 | 0.467 | 0.533 | |
| DG-CFModel=Gemma3-4B, Language=zh2026.05 | 66 | — | 35.1 | 69 | — | — | |
| MACROModel=Gemma3-4B, Language=ar2026.05 | 66 | — | 26.3 | 65 | — | — | |
| Gemma3-4BLanguage=ar, Protocol=TB-CF2026.05 | 65.7 | 29.3 | 19.29 | 7.3 | 8.045 | -7.045 | |
| Qwen3-8BLanguage=de, Protocol=DG-CF2026.05 | 65.2 | 56.8 | 32.09 | 68.6 | 0.263 | 0.737 | |
| SFTModel=Gemma3-4B, Language=de2026.05 | 65 | — | 24 | 71 | — | — | |
| SFTModel=Gemma3-4B, Language=vi2026.05 | 65 | — | 16.5 | 67 | — | — | |
| MACROModel=Gemma3-4B, Language=vi2026.05 | 65 | — | 18.7 | 71 | — | — | |
| Qwen3-4BLanguage=ar, Protocol=DG-CF2026.05 | 64.6 | 59.1 | 40.77 | 70.4 | 0.251 | 0.749 | |
| MACROModel=Gemma3-4B, Language=en2026.05 | 64 | — | 31.7 | 65 | — | — | |
| MACROModel=Gemma3-4B, Language=de2026.05 | 64 | — | 24.4 | 78 | — | — | |
| MACROModel=Gemma3-4B, Language=ru2026.05 | 64 | — | 19.1 | 74 | — | — | |
| Qwen3-8BLanguage=ru, Protocol=DG-CF2026.05 | 63.5 | 57.2 | 22.08 | 69.6 | 0.295 | 0.705 | |
| DG-CFModel=Gemma3-4B, Language=en2026.05 | 63 | — | 48.8 | 61 | — | — | |
| MACROModel=Qwen3-4B, Language=avg2026.05 | 63 | — | 27.6 | 75 | — | — | |
| Gemma3-4BLanguage=en, Protocol=DG-CF2026.05 | 62.7 | 55.9 | 48.76 | 61.3 | 0.529 | 0.471 | |
| DG-CFModel=Gemma3-4B, Language=avg2026.05 | 62 | — | 28.3 | 69 | — | — | |
| Qwen3-8BLanguage=zh, Protocol=DG-CF2026.05 | 61.5 | 53.4 | 38.59 | 68.1 | 0.32 | 0.68 | |
| DG-CFModel=Gemma3-4B, Language=ar2026.05 | 61 | — | 27.7 | 66 | — | — | |
| DG-CFModel=Gemma3-4B, Language=ru2026.05 | 61 | — | 20.3 | 74 | — | — | |
| DG-CFModel=Qwen3-4B, Language=avg2026.05 | 61 | — | 38.8 | 72 | — | — | |
| DG-CFModel=Qwen3-8B, Language=avg2026.05 | 61 | — | 32.4 | 72 | — | — | |
| Gemma3-4BLanguage=ar, Protocol=DG-CF2026.05 | 60.9 | 54.8 | 27.74 | 66.3 | 0.401 | 0.599 | |
| Gemma3-4BLanguage=ru, Protocol=DG-CF2026.05 | 60.5 | 51.4 | 20.26 | 73.7 | 0.459 | 0.541 | |
| Qwen3-4BLanguage=de, Protocol=TB-CF2026.05 | 59.6 | 45.5 | 41.4 | 73.3 | 0.562 | 0.438 | |
| Qwen3-8BLanguage=de, Protocol=TB-CF2026.05 | 59.6 | 49.5 | 30.57 | 66.1 | 0.558 | 0.442 | |
| Qwen3-8BLanguage=ru, Protocol=TB-CF2026.05 | 59.6 | 45.5 | 22.43 | 66.2 | 0.609 | 0.391 | |
| Qwen3-8BLanguage=en, Protocol=DG-CF2026.05 | 59.4 | 48.9 | 53.24 | 73.1 | 0.231 | 0.769 | |
| Gemma3-4BLanguage=de, Protocol=DG-CF2026.05 | 59.2 | 49.9 | 28.54 | 75.4 | 0.389 | 0.611 | |
| Gemma3-4BLanguage=vi, Protocol=DG-CF2026.05 | 59 | 50.8 | 18.88 | 71.1 | 0.456 | 0.544 | |
| DG-CFModel=Gemma3-4B, Language=de2026.05 | 59 | — | 28.5 | 75 | — | — | |
| DG-CFModel=Gemma3-4B, Language=vi2026.05 | 59 | — | 18.9 | 71 | — | — | |
| Qwen3-8BLanguage=vi, Protocol=TB-CF2026.05 | 58.6 | 51.5 | 19.4 | 65.6 | 0.583 | 0.417 | |
| Qwen3-8BLanguage=ar, Protocol=DG-CF2026.05 | 57.6 | 50.5 | 34.88 | 73.5 | 0.169 | 0.831 | |
| Qwen3-4BLanguage=sw, Protocol=TB-CF2026.05 | 56.6 | 14.1 | 18.69 | 49.1 | 1.889 | -0.889 | |
| Qwen3-8BLanguage=sw, Protocol=TB-CF2026.05 | 55.6 | 24.2 | 22.4 | 52.1 | 1.512 | -0.512 | |
| Qwen3-8BLanguage=sw, Protocol=DG-CF2026.05 | 53.3 | 24.4 | 22.93 | 84.4 | 0.324 | 0.676 | |
| Qwen3-4BLanguage=en, Protocol=DG-CF2026.05 | 53.1 | 48.2 | 58.52 | 80.1 | 0.224 | 0.776 | |
| Qwen3-4BLanguage=sw, Protocol=DG-CF2026.05 | 51.2 | 13.7 | 42.85 | 71.4 | 0.397 | 0.603 | |
| Qwen3-4BLanguage=ru, Protocol=TB-CF2026.05 | 50.5 | 44.4 | 32.23 | 73.1 | 0.612 | 0.388 | |
| Qwen3-4BLanguage=ru, Protocol=DG-CF2026.05 | 49.6 | 42.3 | 23.73 | 79.3 | 0.199 | 0.801 | |
| Qwen3-4BLanguage=vi, Protocol=TB-CF2026.05 | 47.5 | 39.4 | 20.38 | 73.3 | 0.579 | 0.421 |