Causal Tool Use on Causal-ToolBench Adversarial confounding
0False Execution RateOracleSCM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OracleSCMNumber of seeds=72026.05 | 0 | 54.5 | 100 | 2.76 | |
| CIVeXNumber of seeds=72026.05 | 0 | 39.4 | 84.9 | 2.23 | |
| CausalPromptingCoT-OpusBackbone=Opus, Number of seeds=52026.05 | 0 | 20 | 65.2 | 1.65 | |
| AlwaysAbstainNumber of seeds=72026.05 | 0 | 0 | 45.5 | 0.99 | |
| CausalNoExperimentNumber of seeds=72026.05 | 0 | 0 | 45.5 | 0.99 | |
| ContextOnlyNoCausalNumber of seeds=72026.05 | 0 | 0 | 45.5 | 0.99 | |
| ObservationalAssociationNumber of seeds=72026.05 | 0 | 0 | 45.5 | 0.99 | |
| CausalPromptingCoT-SonnetBackbone=Sonnet, Number of seeds=52026.05 | 1 | 28 | 72.2 | 1.82 | |
| NameOnlyClassifierNumber of seeds=72026.05 | 8.8 | 7.9 | 44.5 | 0.78 | |
| CausalPromptingTerseNumber of seeds=72026.05 | 10.2 | 39.4 | 74.6 | 1.57 | |
| PolicyGateNumber of seeds=72026.05 | 28.3 | 6.9 | 24 | -0.27 | |
| FamilyMajorityClassifierNumber of seeds=72026.05 | 45.5 | 54.5 | 54.5 | -0.03 | |
| SchemaGateNumber of seeds=72026.05 | 45.5 | 54.5 | 54.5 | -0.03 | |
| SemanticOntologyGateNumber of seeds=72026.05 | 45.5 | 54.5 | 54.5 | -0.03 |