P&ID Question Answering on CypherRAG Conceptual Graph
86AccuracyGPT-5-mini
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-5-miniModel Provider=OpenAI2026.03 | 86 | 0.002 | — | |
| GPT-5Model Provider=OpenAI2026.03 | 79 | 0.012 | — | |
| Claude-Sonnet-4Model Provider=Anthropic2026.03 | 69 | 0.005 | — | |
| Claude-Opus-4-1Model Provider=Anthropic2026.03 | 68 | 0.024 | — | |
| Claude-3-7-SonnetModel Provider=Anthropic2026.03 | 61 | 0.005 | — | |
| GPT-4oModel Provider=OpenAI2026.03 | 60 | 0.002 | — | |
| Claude-3-5-HaikuModel Provider=Anthropic2026.03 | 54 | 0.001 | — | |
| GPT-4o-miniModel Provider=OpenAI2026.03 | 54 | 0.0001 | — | |
| Qwen3:14BModel Provider=Ollama, Hardware Configuration=Mac mini M4 with 24GB RAM2026.03 | 52 | — | 273 | |
| GPT-OSS:20BModel Provider=Ollama, Hardware Configuration=Mac mini M4 with 24GB RAM2026.03 | 45 | — | 73 | |
| Qwen3:8BModel Provider=Ollama, Hardware Configuration=Mac mini M4 with 24GB RAM2026.03 | 33 | — | 86 |