Event Extraction on ACE (test)
41.39Practical EMRLQG
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RLQGMethod Category=Reinforcement Learning Question Generation (Proposed)2024.05 | 41.39 | 48.58 | 67.94 | 21.71 | 24.19 | 31.8 | |
| SFT (Dynamic)Method Category=Supervised Fine-tuning, Prompt Template Type=Dynamic2024.05 | 39.31 | 46.78 | 66.24 | 20.35 | 23.05 | 30.53 | |
| GPT-4 (5shot)Method Category=In-context learning, Backbone Model=GPT-4, Number of Shots=52024.05 | 39.24 | 47.59 | 65.92 | 16.32 | 19.37 | 27.46 | |
| Dynamic-QMethod Category=Template, Prompt Template Type=Dynamic2024.05 | 38.7 | 45.79 | 65.55 | 20.45 | 23.12 | 30.79 | |
| SFT (Guideline)Method Category=Supervised Fine-tuning, Prompt Template Type=Guideline2024.05 | 38.62 | 44.69 | 64.66 | 17.33 | 19.61 | 27.77 | |
| Guideline-QMethod Category=Template, Prompt Template Type=Guideline2024.05 | 38.51 | 45.28 | 65.54 | 17.61 | 19.96 | 28.14 | |
| SFT (Back-Trans)Method Category=Supervised Fine-tuning, Prompt Template Type=Back-Translation2024.05 | 38.24 | 43.56 | 64.11 | 17.47 | 18.9 | 27.32 | |
| SFT (Standard)Method Category=Supervised Fine-tuning, Prompt Template Type=Standard2024.05 | 37.63 | 42.95 | 62.36 | 15.31 | 17.13 | 25.72 | |
| Standard-QMethod Category=Template, Prompt Template Type=Standard2024.05 | 37.42 | 43.87 | 63.7 | 15.6 | 17.36 | 25.92 | |
| Back-Trans-QMethod Category=Template, Prompt Template Type=Back-Translation2024.05 | 36.13 | 41.39 | 62.41 | 15.14 | 16.67 | 25.28 | |
| Simple-QMethod Category=Template, Prompt Template Type=Simple2024.05 | 35.41 | 40.23 | 60.93 | 14.38 | 16.17 | 24.55 | |
| GPT-4 (0shot)Method Category=In-context learning, Backbone Model=GPT-4, Number of Shots=02024.05 | 28.97 | 35.83 | 57.9 | 11.14 | 13.54 | 23.35 | |
| LLaMA-2-13b-Chat (5shot)Method Category=In-context learning, Backbone Model=LLaMA-2-13b-Chat, Number of Shots=52024.05 | 27.97 | 33.04 | 53.69 | 13.01 | 14.93 | 23.54 | |
| LLaMA-2-13b-Chat (0shot)Method Category=In-context learning, Backbone Model=LLaMA-2-13b-Chat, Number of Shots=02024.05 | 1.21 | 3.5 | 35.88 | 0.43 | 1.25 | 21.78 |