Embodied Decision-Grounded QA on Microsoft PdM
90Structural OKFull IndustryAssetEQA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Full IndustryAssetEQAModel=Claude Sonnet 42026.04 | 90 | 89 | 95 | 91 | 78 | 74 | |
| Full IndustryAssetEQAModel=GPT-4o-mini2026.04 | 88 | 89 | 94 | 88 | 72 | 67 | |
| + Provenance-EnforcedModel=GPT-4o-mini2026.04 | 82 | 83 | 89 | 45 | 63 | 59 | |
| + Provenance-EnforcedModel=Claude Sonnet 42026.04 | 80 | 84 | 87 | 44 | 61 | 60 | |
| LLM + Episodic + KGModel=Claude Sonnet 42026.04 | 55 | 62 | 73 | 44 | 58 | 54 | |
| LLM + EpisodicModel=Claude Sonnet 42026.04 | 54 | 65 | 73 | 44 | 27 | 28 | |
| LLM + EpisodicModel=GPT-4o-mini2026.04 | 52 | 62 | 71 | 45 | 23 | 25 | |
| LLM + Episodic + KGModel=GPT-4o-mini2026.04 | 52 | 62 | 73 | 45 | 56 | 51 | |
| LLM-only QAModel=GPT-4o-mini2026.04 | 42 | 47 | 62 | 45 | 8 | 12 | |
| LLM-only QAModel=Claude Sonnet 42026.04 | 39 | 44 | 59 | 44 | 10 | 12 |