Article-level Frame Prediction on Economic News Articles (5-Fold Cross val)
51.5Type Macro F1Base Classifier
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Base Classifierbackbone=RoBERTa, training=fine-tuned2024.02 | 51.5 | 69.7 | 49.3 | |
| Base Classifier + DAPTbackbone=RoBERTa, DAPT=true, training=fine-tuned2024.02 | 47.4 | 63.6 | 47.5 | |
| Relational (best)backbone=RoBERTa, probabilistic_soft_logic=true, rule_set=r1+r2+r52024.02 | 43.8 | 71.7 | 52.2 | |
| Mistralmodel=Mistral-7b-Instruct-v0.2, prompting=zero-shot2024.02 | 38.6 | 23.6 | 34.6 | |
| Mistralmodel=Mistral-7b-Instruct-v0.2, prompting=two-shot2024.02 | 36.7 | 18.6 | 38.5 | |
| Majority Label2024.02 | 23.7 | 26.9 | 25.2 | |
| Random2024.02 | 13.6 | 14.8 | 19 |