Quantity-level Frame Prediction on Economic News Articles 5-Fold Cross (val)
74.8Type Macro F1Relational (best)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Relational (best)backbone=RoBERTa, probabilistic_soft_logic=true, rule_set=r1+r2+r52024.02 | 74.8 | 84.9 | 81.3 | |
| Base Classifier + DAPTbackbone=RoBERTa, DAPT=true, training=fine-tuned2024.02 | 73.1 | 82.6 | 81.2 | |
| Base Classifierbackbone=RoBERTa, training=fine-tuned2024.02 | 68.5 | 82.4 | 79.6 | |
| Mistralmodel=Mistral-7b-Instruct-v0.2, prompting=zero-shot2024.02 | 56.6 | 26.2 | 38.2 | |
| Mistralmodel=Mistral-7b-Instruct-v0.2, prompting=two-shot2024.02 | 35.8 | 47 | 43 | |
| Majority Label2024.02 | 19.3 | 8 | 38.2 | |
| Random2024.02 | 11.1 | 6.4 | 32.4 |