Relational Triplet Extraction on CHI
53GPT AccuracyUrbanKGent-13B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| UrbanKGent-13BType=Agent2024.02 | 53 | 4.15 | 59 | |
| UrbanKGent-7BType=Agent2024.02 | 49 | 4.28 | 54 | |
| UrbanKGent-8BType=Agent2024.02 | 49 | 3.97 | 55 | |
| GPT-4Type=UrbanKGent Inference2024.02 | 46 | 4.17 | 52 | |
| GPT-3.5Type=UrbanKGent Inference2024.02 | 40 | 4.21 | 46 | |
| GPT-4Type=Zero-shot Reasoning2024.02 | 39 | 4.08 | 43 | |
| GPT-3.5Type=In-context Learning2024.02 | 36 | 4.24 | 39 | |
| GPT-3.5Type=Zero-shot Reasoning2024.02 | 31 | 3.79 | 31 | |
| Llama-3-8BType=Vanilla Fine-tuning2024.02 | 31 | 4.18 | 32 | |
| Llama-2-13BType=Vanilla Fine-tuning2024.02 | 31 | 4.14 | 37 | |
| Llama-2-7BType=Vanilla Fine-tuning2024.02 | 29 | 3.8 | 33 | |
| Llama-2-70BType=UrbanKGent Inference2024.02 | 29 | 3.8 | 32 | |
| Llama-3-70BType=UrbanKGent Inference2024.02 | 29 | 4.12 | 31 | |
| Llama-3-8BType=In-context Learning2024.02 | 28 | 4.31 | 24 | |
| Llama-2-13BType=In-context Learning2024.02 | 28 | 2.65 | 25 | |
| Llama-2-13BType=UrbanKGent Inference2024.02 | 28 | 3.24 | 31 | |
| Llama-2-70BType=Zero-shot Reasoning2024.02 | 27 | 3.55 | 24 | |
| Vicuna-7BType=UrbanKGent Inference2024.02 | 27 | 4.12 | 23 | |
| Alpaca-7BType=UrbanKGent Inference2024.02 | 27 | 3.83 | 27 | |
| Llama-2-7BType=UrbanKGent Inference2024.02 | 27 | 2.87 | 26 | |
| Llama-2-7BType=Zero-shot Reasoning2024.02 | 26 | 1.96 | 21 | |
| Llama-3-70BType=Zero-shot Reasoning2024.02 | 26 | 3.98 | 25 | |
| Llama-3-8BType=UrbanKGent Inference2024.02 | 26 | 3.28 | 30 | |
| Llama-2-7BType=In-context Learning2024.02 | 25 | 2.44 | 23 | |
| Mistral-7BType=UrbanKGent Inference2024.02 | 25 | 3.61 | 25 | |
| Llama-3-8BType=Zero-shot Reasoning2024.02 | 24 | 3.75 | 22 | |
| Alpaca-7BType=Zero-shot Reasoning2024.02 | 23 | 3.96 | 20 | |
| Vicuna-7BType=Zero-shot Reasoning2024.02 | 22 | 4.12 | 21 | |
| Llama-2-13BType=Zero-shot Reasoning2024.02 | 22 | 2.19 | 22 | |
| RelationPromptType=End-to-end Models2024.02 | 21 | 3.53 | 18 | |
| Mistral-7BType=Zero-shot Reasoning2024.02 | 21 | 3.75 | 19 | |
| PRGCType=End-to-end Models2024.02 | 13 | 4.15 | 15 |