Knowledge Graph Completion on NYC
56GPT AccuracyUrbanKGent-13B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| UrbanKGent-13BType=Agent2024.02 | 56 | 4.13 | 47 | |
| UrbanKGent-8BType=Agent2024.02 | 51 | 4.15 | 45 | |
| UrbanKGent-7BType=Agent2024.02 | 49 | 3.97 | 44 | |
| GPT-4Type=UrbanKGent Inference2024.02 | 48 | 4.02 | 42 | |
| GPT-3.5Type=UrbanKGent Inference2024.02 | 46 | 3.88 | 34 | |
| GPT-3.5Type=In-context Learning2024.02 | 42 | 4.08 | 31 | |
| Llama-2-13BType=Vanilla Fine-tuning2024.02 | 41 | 3.92 | 29 | |
| GPT-4Type=Zero-shot Reasoning2024.02 | 39 | 3.82 | 29 | |
| Llama-2-7BType=Vanilla Fine-tuning2024.02 | 38 | 3.65 | 27 | |
| GPT-3.5Type=Zero-shot Reasoning2024.02 | 36 | 3.47 | 23 | |
| Llama-3-70BType=UrbanKGent Inference2024.02 | 36 | 4.81 | 28 | |
| Llama-3-8BType=Vanilla Fine-tuning2024.02 | 35 | 4.18 | 26 | |
| Llama-2-70BType=UrbanKGent Inference2024.02 | 35 | 4.27 | 29 | |
| Llama-2-13BType=UrbanKGent Inference2024.02 | 32 | 3.56 | 27 | |
| Llama-2-13BType=In-context Learning2024.02 | 31 | 3.28 | 24 | |
| Llama-3-8BType=UrbanKGent Inference2024.02 | 31 | 4.08 | 26 | |
| Llama-3-70BType=Zero-shot Reasoning2024.02 | 29 | 4.31 | 24 | |
| Llama-2-13BType=Zero-shot Reasoning2024.02 | 28 | 3.91 | 22 | |
| Llama-2-70BType=Zero-shot Reasoning2024.02 | 28 | 3.75 | 24 | |
| Alpaca-7BType=UrbanKGent Inference2024.02 | 27 | 3.83 | 22 | |
| Llama-2-7BType=UrbanKGent Inference2024.02 | 26 | 4.12 | 24 | |
| Mistral-7BType=UrbanKGent Inference2024.02 | 25 | 4.08 | 23 | |
| KG-BERTType=End-to-end Models2024.02 | 24 | 3.15 | 23 | |
| Vicuna-7BType=UrbanKGent Inference2024.02 | 24 | 3.95 | 23 | |
| KG-T5Type=End-to-end Models2024.02 | 21 | 4.02 | 21 | |
| Mistral-7BType=Zero-shot Reasoning2024.02 | 21 | 4.11 | 18 | |
| Llama-2-7BType=In-context Learning2024.02 | 21 | 3.96 | 18 | |
| Vicuna-7BType=Zero-shot Reasoning2024.02 | 19 | 4.06 | 16 | |
| Llama-2-7BType=Zero-shot Reasoning2024.02 | 18 | 3.75 | 18 | |
| Llama-3-8BType=In-context Learning2024.02 | 18 | 3.53 | 22 | |
| Alpaca-7BType=Zero-shot Reasoning2024.02 | 17 | 3.87 | 17 | |
| Llama-3-8BType=Zero-shot Reasoning2024.02 | 15 | 4.02 | 21 |