Causal Discovery on CausalMan Small
48.3F1 ScoreCAST (HITL)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CAST (HITL)LLM=Qwen 3 14B2026.07 | 48.3 | 60.8 | 40.2 | 102.4 | 1,856.65 | |
| CASTLLM Backbone=Qw.3 14B, HITL=true, RAG=false2026.07 | 48.3 | 60.8 | — | 102.4 | — | |
| CAST (RAG)LLM=Qwen 3 14B2026.07 | 42.6 | 65.6 | 33.5 | 111.4 | 944.92 | |
| CASTLLM Backbone=Qw.3 14B, HITL=false, RAG=true2026.07 | 42.6 | 65.6 | — | 111.4 | — | |
| CAST (HITL+RAG)LLM=Qwen 3 14B2026.07 | 42.4 | 72.4 | 30.7 | 103 | 1,064.03 | |
| CASTLLM Backbone=Qw.3 14B, HITL=true, RAG=true2026.07 | 42.4 | 72.4 | — | 103 | — | |
| CAST (HITL)LLM=o3-mini2026.07 | 30.4 | 48.5 | 22.4 | 112.2 | 254.66 | |
| CausalStewardLLM Backbone=o3-mini, HITL=true, RAG=false2026.07 | 30.4 | 48.5 | — | 112.2 | — | |
| CAST (HITL+RAG)LLM=o3-mini2026.07 | 30.3 | 45.4 | 22.8 | 118.4 | 316.65 | |
| CausalStewardLLM Backbone=o3-mini, HITL=true, RAG=true2026.07 | 30.3 | 45.4 | — | 118.4 | — | |
| LLM-BFSLLM=o3-mini2026.07 | 28.3 | 100 | — | 315.2 | — | |
| LLM-BFSLLM Backbone=o3-mini2026.07 | 28.3 | 100 | — | 315.2 | — | |
| CAST (RAG)LLM=o3-mini2026.07 | 23.5 | 37.3 | 17.8 | 121.4 | 455.31 | |
| CausalStewardLLM Backbone=o3-mini, HITL=false, RAG=true2026.07 | 23.5 | 37.3 | — | 121.4 | — | |
| PairwiseLLM=Qwen3-14B2026.07 | 15.3 | 9.3 | — | 583.5 | — | |
| PairwiseLLM Backbone=Qwen3-14B2026.07 | 15.3 | 9.3 | — | 583.5 | — | |
| CAST (HITL+RAG)LLM=gpt 4o-mini2026.07 | 14.2 | 21.7 | 10.5 | 140.4 | 1,630.19 | |
| CASTLLM Backbone=o4-mini, HITL=true, RAG=true2026.07 | 14.2 | 21.7 | — | 140.4 | — | |
| CAST (HITL)LLM=gpt 4o-mini2026.07 | 13.9 | 32.6 | 9.6 | 126.8 | 1,456.33 | |
| CASTLLM Backbone=o4-mini, HITL=true, RAG=false2026.07 | 13.9 | 32.6 | — | 126.8 | — | |
| PairwiseLLM=o3-mini2026.07 | 12.6 | 7.4 | 42.1 | 710 | — | |
| CAST (RAG)LLM=gpt 4o-mini2026.07 | 11.3 | 16.9 | 8.9 | 145 | 913.15 | |
| CASTLLM Backbone=o4-mini, HITL=false, RAG=true2026.07 | 11.3 | 16.9 | — | 145 | — | |
| CausalCopilot2026.07 | 7.8 | 26 | 4.8 | 117.4 | — | |
| CausalCopilot2026.07 | 7.8 | 26 | — | 117.4 | — | |
| XGES2026.07 | 5.6 | 3.7 | 11.4 | 448.8 | 25,173.8 | |
| XGES2026.07 | 5.6 | 3.7 | — | 448.8 | — | |
| BOSS2026.07 | 5.4 | 2.9 | — | 864.2 | — | |
| BOSS2026.07 | 5.4 | 2.9 | — | 864.2 | — | |
| FCI2026.07 | 2.8 | 2.1 | 4.3 | 221.4 | 9.37 | |
| FCI2026.07 | 2.8 | 2.1 | — | 221.4 | — | |
| GranDAG2026.07 | 0.2 | 1.8 | 0.2 | 113.4 | 333.43 | |
| GranDAG2026.07 | 0.2 | 1.8 | — | 113.4 | — |