Knowledge Graph Completion (KGC) on CHI
0.48GPT AccuracyUrbanKGent-13B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| UrbanKGent-13BType=Agent2024.02 | 0.48 | 4.42 | 0.49 | |
| UrbanKGent-8BType=Agent2024.02 | 0.44 | 4.05 | 0.46 | |
| UrbanKGent-7BType=Agent2024.02 | 0.43 | 4.58 | 0.45 | |
| GPT-4Type=UrbanKGent Inference2024.02 | 0.41 | 4.35 | 0.43 | |
| GPT-3.5Type=UrbanKGent Inference2024.02 | 0.39 | 3.87 | 0.41 | |
| GPT-3.5Type=In-context Learning2024.02 | 0.36 | 4.23 | 0.36 | |
| GPT-4Type=Zero-shot Reasoning2024.02 | 0.32 | 4.03 | 0.35 | |
| GPT-3.5Type=Zero-shot Reasoning2024.02 | 0.31 | 3.16 | 0.29 | |
| Llama-2-7BType=Vanilla Fine-tuning2024.02 | 0.3 | 3.65 | 0.31 | |
| Llama-3-8BType=Vanilla Fine-tuning2024.02 | 0.29 | 4.15 | 0.34 | |
| Llama-2-13BType=Vanilla Fine-tuning2024.02 | 0.29 | 3.87 | 0.35 | |
| Llama-3-70BType=UrbanKGent Inference2024.02 | 0.29 | 4.81 | 0.35 | |
| Llama-2-70BType=UrbanKGent Inference2024.02 | 0.28 | 4.01 | 0.34 | |
| Llama-2-13BType=UrbanKGent Inference2024.02 | 0.26 | 3.28 | 0.32 | |
| Llama-2-7BType=UrbanKGent Inference2024.02 | 0.24 | 3.54 | 0.29 | |
| Llama-3-8BType=UrbanKGent Inference2024.02 | 0.24 | 3.97 | 0.31 | |
| Vicuna-7BType=UrbanKGent Inference2024.02 | 0.22 | 3.95 | 0.25 | |
| Llama-2-13BType=In-context Learning2024.02 | 0.21 | 2.53 | 0.26 | |
| Alpaca-7BType=UrbanKGent Inference2024.02 | 0.21 | 4.12 | 0.29 | |
| Mistral-7BType=UrbanKGent Inference2024.02 | 0.21 | 4.08 | 0.26 | |
| KG-BERTType=End-to-end Models2024.02 | 0.19 | 4.12 | 0.24 | |
| Llama-2-7BType=In-context Learning2024.02 | 0.18 | 3.27 | 0.2 | |
| Llama-3-70BType=Zero-shot Reasoning2024.02 | 0.17 | 4.26 | 0.23 | |
| Llama-3-8BType=In-context Learning2024.02 | 0.17 | 4.14 | 0.21 | |
| Alpaca-7BType=Zero-shot Reasoning2024.02 | 0.16 | 4.15 | 0.16 | |
| Llama-2-13BType=Zero-shot Reasoning2024.02 | 0.16 | 2.47 | 0.24 | |
| Llama-2-70BType=Zero-shot Reasoning2024.02 | 0.16 | 2.47 | 0.23 | |
| KG-T5Type=End-to-end Models2024.02 | 0.15 | 3.98 | 0.24 | |
| Mistral-7BType=Zero-shot Reasoning2024.02 | 0.15 | 3.76 | 0.19 | |
| Llama-2-7BType=Zero-shot Reasoning2024.02 | 0.15 | 2.83 | 0.22 | |
| Llama-3-8BType=Zero-shot Reasoning2024.02 | 0.15 | 4.08 | 0.22 | |
| Vicuna-7BType=Zero-shot Reasoning2024.02 | 0.14 | 4.03 | 0.18 |