Graph Reasoning on WL-Graph (NLGraph-based, test)
99Undirected CycleGraphForge
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GraphForgeReasoning Strategy=GraphTool-Instruction (Function Calling)2024.12 | 99 | 97.8 | 98.8 | 98.6 | 98.2 | 99.6 | 97.2 | 98.2 | 98 | 98 | 98.4 | |
| Llama3 with GraphTool-InstructionReasoning Strategy=GraphTool-Instruction2024.12 | 97.8 | 87.6 | 94.6 | 93.4 | 93.2 | 96 | 97.8 | 93 | 92.2 | 93.6 | 94 | |
| GraphWizReasoning Strategy=CoT/Text-Instruction2024.12 | 79.8 | — | 81 | 1 | 7.6 | 76.4 | 40.2 | 79.6 | 2.2 | 6 | 41.5 | |
| Claude-3-opusReasoning Strategy=CoT/Text-Instruction2024.12 | 75.8 | 21 | 79.8 | 9.8 | 21.2 | 73.2 | 22.2 | 83.2 | 12.2 | 19 | 41.7 | |
| NLGraphReasoning Strategy=CoT/Text-Instruction2024.12 | 72.2 | — | 79.2 | 10.8 | 28 | 70.2 | 34.2 | 84.2 | 11 | 26.2 | 46.2 | |
| GPT-4-turboReasoning Strategy=CoT/Text-Instruction2024.12 | 68.2 | 20.6 | 77.2 | 7.8 | 25.8 | 64.2 | 34.4 | 84 | 9 | 22.6 | 41.4 | |
| GPT-3.5-turboReasoning Strategy=CoT/Text-Instruction2024.12 | 59.5 | 10.4 | 72.2 | 0.6 | 24.4 | 68 | 24.6 | 67.6 | 8.8 | 21.2 | 35.7 | |
| Claude-3-sonnetReasoning Strategy=CoT/Text-Instruction2024.12 | 52.6 | 13.6 | 74.2 | 7.8 | 11.4 | 53 | 17 | 77 | 7 | 13.8 | 32.7 | |
| Claude-3-haikuReasoning Strategy=CoT/Text-Instruction2024.12 | 48.8 | 16.6 | 76.6 | 3.6 | 8.8 | 50.2 | 16.6 | 73.2 | 4.2 | 9.6 | 30.8 |