Causal Variable Identification on Code-Preference
80.2F1 (X)GPT-o4
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-o4Setting=Decompositional Framework, Model Variant=GPT-o4-mini-high2025.05 | 80.2 | 69 | 66.3 | 77 | |
| Llama4-MSetting=Decompositional Framework, Model Variant=Llama-4-Maverick-17B-128E2025.05 | 75.2 | 63.4 | 65.7 | 80.4 | |
| Llama4-SSetting=Decompositional Framework, Model Variant=Llama-4-Scout-17B2025.05 | 73.6 | 61.9 | 64.5 | 79.3 | |
| Qwen3Setting=Decompositional Framework, Model Variant=Qwen3-VL-235B-A22B Thinking2025.05 | 72.9 | 60.5 | 50.4 | 78.6 | |
| Gemini2.5Setting=Decompositional Framework, Model Variant=Gemini2.5-Pro2025.05 | 68.4 | 66.5 | 48.2 | 76.1 | |
| DeepSeekSetting=Decompositional Framework, Model Variant=DeepSeek-VL2025.05 | 61.2 | 59.3 | 52.5 | 59.8 | |
| GPT-5Setting=Decompositional Framework2025.05 | 49.6 | 68.4 | 65.9 | 75.3 |