Fine-Grained Narrative Classification on INDI-PROP Farmer's Protest 1.0 (test)
0.757Micro PrecisionFANTA
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| FANTABackbone=Mistral2025.12 | 0.757 | 0.691 | 0.723 | 0.522 | 0.381 | 0.427 | 0.746 | 0.691 | 0.696 | |
| FANTABackbone=GPT-4o-mini2025.12 | 0.732 | 0.805 | 0.767 | 0.702 | 0.78 | 0.709 | 0.775 | 0.805 | 0.772 | |
| FANTABackbone=GPT-4o-mini, mode=Concise2025.12 | 0.73 | 0.81 | 0.77 | 0.65 | 0.83 | 0.7 | 0.78 | 0.81 | 0.78 | |
| RoBERTaBackbone=RoBERTa2025.12 | 0.714 | 0.671 | 0.692 | 0.41 | 0.343 | 0.345 | 0.726 | 0.671 | 0.684 | |
| BERTBackbone=BERT2025.12 | 0.712 | 0.664 | 0.688 | 0.183 | 0.226 | 0.201 | 0.604 | 0.664 | 0.629 | |
| DeBERTaBackbone=DeBERTa2025.12 | 0.681 | 0.658 | 0.669 | 0.211 | 0.236 | 0.221 | 0.621 | 0.658 | 0.637 | |
| DistillBERTBackbone=DistillBERT2025.12 | 0.621 | 0.584 | 0.602 | 0.163 | 0.212 | 0.179 | 0.562 | 0.584 | 0.561 | |
| FANTABackbone=Gemma2025.12 | 0.37 | 0.563 | 0.446 | 0.336 | 0.792 | 0.393 | 0.676 | 0.563 | 0.482 | |
| GPT-4o-minimode=Zero-Shot2025.12 | 0.3 | 0.45 | 0.367 | 0.119 | 0.279 | 0.13 | 0.47 | 0.46 | 0.42 | |
| GPT-4o-minimode=Few-Shot2025.12 | 0.1 | 0.24 | 0.15 | 0.048 | 0.298 | 0.077 | 0.05 | 0.235 | 0.082 |