Causal Tool Verification on Causal-ToolBench Moderate Confounding n = 1,050
0False Execution RateOracleSCM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OracleSCMDescription=Uses the SCM-given θ2026.05 | 0 | 49.7 | 100 | 2.9 | |
| PolicyGateType=Policy/heuristic gate2026.05 | 0 | 49.7 | 100 | 2.9 | |
| CIVeXSeeds=72026.05 | 0 | 29.6 | 79.9 | 2.29 | |
| ContextOnlyNoCausalType=Causal-baseline spectrum2026.05 | 0 | 23.3 | 73.6 | 2.21 | |
| CausalNoExperimentType=LCB-only2026.05 | 0 | 20.8 | 71 | 2.04 | |
| AlwaysAbstainType=Refuse-all2026.05 | 0 | 0 | 50.3 | 1.43 | |
| ObservationalAssociationType=Sign-only2026.05 | 0 | 0 | 50.3 | 1.43 | |
| CausalPromptingCoT-OpusModel=claude-opus-4-7, Prompting strategy=Chain-of-thought, max_tokens=512, Exemplars=2-shot2026.05 | 0.2 | 29.6 | 79.7 | 2.28 | |
| CausalPromptingCoT-SonnetModel=claude-sonnet-4-6, Prompting strategy=Chain-of-thought, max_tokens=512, Exemplars=2-shot2026.05 | 2.3 | 28.2 | 76.2 | 2.08 | |
| CausalPromptingTersePrompting strategy=Original single-prompt, max_tokens=202026.05 | 5.7 | 26.6 | 71.1 | 1.87 | |
| NameOnlyClassifierType=Policy/heuristic gate2026.05 | 7.6 | 9 | 51.7 | 1.4 | |
| FamilyMajorityClassifierType=Policy/heuristic gate2026.05 | 50.3 | 49.7 | 49.7 | -0.64 | |
| SchemaGateType=Policy/heuristic gate2026.05 | 50.3 | 49.7 | 49.7 | -0.64 | |
| SemanticOntologyGateType=Policy/heuristic gate2026.05 | 50.3 | 49.7 | 49.7 | -0.64 |