Mechanistic Explanation Generation on VC-Traces representative subset aligned with Tahoe-X1 (test)
100ValidityClaude-Sonnet-4
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Claude-Sonnet-4Model Type=Closed2026.04 | 100 | 86.7 | 65.7 | 50.4 | |
| VCR-AgentModel Type=Closed2026.04 | 100 | 94.5 | 72.5 | 52.8 | |
| Qwen3-30BModel Type=Open2026.04 | 86 | 96.5 | 52.8 | 27.2 | |
| Llama3.3-70BModel Type=Open2026.04 | 84.1 | 88.7 | 32.2 | 9 | |
| DeepSeek-R1-8BModel Type=Open2026.04 | 0.3 | 100 | 0 | 0 |