P&ID Question Answering on PathRAG Conceptual Graph
83AccuracyGPT-5-mini
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-5-miniModel Provider=OpenAI2026.03 | 83 | 0.002 | — | |
| GPT-5Model Provider=OpenAI2026.03 | 80 | 0.015 | — | |
| Claude-3-5-HaikuModel Provider=Anthropic2026.03 | 72 | 0.002 | — | |
| Claude-3-7-SonnetModel Provider=Anthropic2026.03 | 72 | 0.009 | — | |
| Claude-Sonnet-4Model Provider=Anthropic2026.03 | 72 | 0.009 | — | |
| Claude-Opus-4-1Model Provider=Anthropic2026.03 | 72 | 0.045 | — | |
| Qwen3:8BModel Provider=Ollama, Hardware Configuration=Mac mini M4 with 24GB RAM2026.03 | 70 | — | 145 | |
| GPT-OSS:20BModel Provider=Ollama, Hardware Configuration=Mac mini M4 with 24GB RAM2026.03 | 70 | — | 81 | |
| Qwen3:4BModel Provider=Ollama, Hardware Configuration=Mac mini M4 with 24GB RAM2026.03 | 68 | — | 141 | |
| Qwen3:14BModel Provider=Ollama, Hardware Configuration=Mac mini M4 with 24GB RAM2026.03 | 68 | — | 139 | |
| GPT-4o-miniModel Provider=OpenAI2026.03 | 64 | 0.0002 | — | |
| GPT-4oModel Provider=OpenAI2026.03 | 63 | 0.003 | — | |
| LLaMA3.1:8BModel Provider=Ollama, Hardware Configuration=Mac mini M4 with 24GB RAM2026.03 | 62 | — | 100 |