Explainable AI on CRAFTER-XAI-Bench (test)
0.54FaithfulnessFAX
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FAXUse structured workflow?=⃝, Use verification stage?=⃝, Query Category=Why2026.05 | 0.54 | 0.92 | 0.99 | 0.98 | |
| FAXUse structured workflow?=⃝, Use verification stage?=⃝, Query Category=What if2026.05 | 0.48 | 0.89 | 0.99 | 0.97 | |
| FAXUse structured workflow?=⃝, Use verification stage?=⃝, Query Category=Plan2026.05 | 0.48 | 0.86 | 0.99 | 0.98 | |
| FAXUse structured workflow?=⃝, Use verification stage?=⃝, Query Category=Average2026.05 | 0.46 | 0.9 | 0.98 | 0.97 | |
| FAXUse structured workflow?=⃝, Use verification stage?=⃝, Query Category=Counterfactual2026.05 | 0.35 | 0.93 | 0.94 | 0.95 | |
| Unstructured Agentic XAIUse structured workflow?=×, Use verification stage?=△, Query Category=What if2026.05 | 0.34 | 0.9 | 0.99 | 0.98 | |
| Explainer DashboardUse structured workflow?=N/A, Use verification stage?=N/A, Query Category=Why2026.05 | 0.31 | 0.32 | 0.45 | 0.26 | |
| Structured Agentic XAI w/o verificationUse structured workflow?=⃝, Use verification stage?=×, Query Category=What if2026.05 | 0.28 | 0.9 | 1 | 0.98 | |
| Explainer DashboardUse structured workflow?=N/A, Use verification stage?=N/A, Query Category=Average2026.05 | 0.2 | 0.29 | 0.4 | 0.26 | |
| Explainer DashboardUse structured workflow?=N/A, Use verification stage?=N/A, Query Category=What if2026.05 | 0.19 | 0.25 | 0.36 | 0.26 | |
| Unstructured Agentic XAIUse structured workflow?=×, Use verification stage?=△, Query Category=Average2026.05 | 0.18 | 0.89 | 0.98 | 0.99 | |
| Naive LLMUse structured workflow?=×, Use verification stage?=×, Query Category=What if2026.05 | 0.17 | 0.91 | 0.98 | 0.99 | |
| Naive LLMUse structured workflow?=×, Use verification stage?=×, Query Category=Plan2026.05 | 0.17 | 0.82 | 0.99 | 0.99 | |
| Unstructured Agentic XAIUse structured workflow?=×, Use verification stage?=△, Query Category=Plan2026.05 | 0.17 | 0.86 | 0.97 | 0.99 | |
| Structured Agentic XAI w/o verificationUse structured workflow?=⃝, Use verification stage?=×, Query Category=Average2026.05 | 0.17 | 0.9 | 0.99 | 0.99 | |
| Structured Agentic XAI w/o verificationUse structured workflow?=⃝, Use verification stage?=×, Query Category=Plan2026.05 | 0.15 | 0.86 | 0.99 | 0.99 | |
| Explainer DashboardUse structured workflow?=N/A, Use verification stage?=N/A, Query Category=Counterfactual2026.05 | 0.14 | 0.27 | 0.31 | 0.26 | |
| Explainer DashboardUse structured workflow?=N/A, Use verification stage?=N/A, Query Category=Plan2026.05 | 0.14 | 0.34 | 0.48 | 0.26 | |
| Naive LLMUse structured workflow?=×, Use verification stage?=×, Query Category=Average2026.05 | 0.14 | 0.85 | 0.98 | 0.99 | |
| Naive LLMUse structured workflow?=×, Use verification stage?=×, Query Category=Why2026.05 | 0.13 | 0.91 | 1 | 0.99 | |
| Structured Agentic XAI w/o verificationUse structured workflow?=⃝, Use verification stage?=×, Query Category=Why2026.05 | 0.13 | 0.91 | 1 | 0.99 | |
| Unstructured Agentic XAIUse structured workflow?=×, Use verification stage?=△, Query Category=Counterfactual2026.05 | 0.12 | 0.91 | 0.98 | 0.99 | |
| Naive LLMUse structured workflow?=×, Use verification stage?=×, Query Category=Counterfactual2026.05 | 0.11 | 0.77 | 0.95 | 0.99 | |
| Structured Agentic XAI w/o verificationUse structured workflow?=⃝, Use verification stage?=×, Query Category=Counterfactual2026.05 | 0.11 | 0.92 | 0.99 | 0.99 | |
| Unstructured Agentic XAIUse structured workflow?=×, Use verification stage?=△, Query Category=Why2026.05 | 0.08 | 0.9 | 1 | 0.99 |