Relational Triplet Extraction on NYC
52GPT AccuracyUrbanKGent-13B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| UrbanKGent-13BType=Agent2024.02 | 52 | 4.38 | 54 | |
| UrbanKGent-8BType=Agent2024.02 | 47 | 3.97 | 49 | |
| UrbanKGent-7BType=Agent2024.02 | 46 | 4.12 | 48 | |
| GPT-4Type=UrbanKGent Inference2024.02 | 45 | 4.08 | 47 | |
| GPT-3.5Type=UrbanKGent Inference2024.02 | 43 | 4.12 | 43 | |
| GPT-3.5Type=In-context Learning2024.02 | 41 | 4.65 | 42 | |
| GPT-4Type=Zero-shot Reasoning2024.02 | 38 | 4.03 | 41 | |
| Llama-2-13BType=Vanilla Fine-tuning2024.02 | 35 | 4.26 | 39 | |
| Llama-3-70BType=UrbanKGent Inference2024.02 | 35 | 4.26 | 34 | |
| Llama-2-70BType=UrbanKGent Inference2024.02 | 33 | 4.28 | 33 | |
| Llama-2-7BType=Vanilla Fine-tuning2024.02 | 32 | 4.37 | 32 | |
| Llama-3-8BType=Vanilla Fine-tuning2024.02 | 31 | 4.18 | 35 | |
| Llama-2-13BType=UrbanKGent Inference2024.02 | 31 | 3.87 | 35 | |
| GPT-3.5Type=Zero-shot Reasoning2024.02 | 29 | 4.11 | 31 | |
| Llama-3-8BType=UrbanKGent Inference2024.02 | 29 | 4.15 | 33 | |
| Llama-2-7BType=UrbanKGent Inference2024.02 | 27 | 3.05 | 28 | |
| Llama-2-13BType=In-context Learning2024.02 | 26 | 3.52 | 23 | |
| Alpaca-7BType=UrbanKGent Inference2024.02 | 26 | 3.85 | 26 | |
| Mistral-7BType=UrbanKGent Inference2024.02 | 26 | 4.15 | 28 | |
| Llama-2-70BType=Zero-shot Reasoning2024.02 | 25 | 3.07 | 22 | |
| Llama-3-70BType=Zero-shot Reasoning2024.02 | 24 | 4.18 | 23 | |
| Vicuna-7BType=UrbanKGent Inference2024.02 | 24 | 3.07 | 29 | |
| Llama-2-13BType=Zero-shot Reasoning2024.02 | 21 | 2.07 | 19 | |
| Llama-2-7BType=In-context Learning2024.02 | 18 | 2.15 | 19 | |
| Llama-3-8BType=In-context Learning2024.02 | 17 | 4.06 | 21 | |
| Llama-3-8BType=Zero-shot Reasoning2024.02 | 15 | 4.02 | 20 | |
| Mistral-7BType=Zero-shot Reasoning2024.02 | 14 | 4.12 | 17 | |
| Llama-2-7BType=Zero-shot Reasoning2024.02 | 14 | 1.98 | 16 | |
| RelationPromptType=End-to-end Models2024.02 | 12 | 3.38 | 12 | |
| Vicuna-7BType=Zero-shot Reasoning2024.02 | 12 | 2.84 | 14 | |
| Alpaca-7BType=Zero-shot Reasoning2024.02 | 11 | 3.75 | 15 | |
| PRGCType=End-to-end Models2024.02 | 8 | 4.01 | 6 |