Fine-Grained Narrative Classification on INDI-PROP CAA 1.0 (test)
0.744Micro PrecisionRoBERTa
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| RoBERTaBackbone=RoBERTa2025.12 | 0.744 | 0.379 | 0.502 | 0.518 | 0.234 | 0.299 | 0.708 | 0.379 | 0.457 | |
| DeBERTaBackbone=DeBERTa2025.12 | 0.68 | 0.444 | 0.538 | 0.35 | 0.251 | 0.269 | 0.566 | 0.444 | 0.471 | |
| FANTABackbone=GPT-4o-mini, mode=Concise2025.12 | 0.67 | 0.83 | 0.74 | 0.59 | 0.79 | 0.66 | 0.7 | 0.83 | 0.74 | |
| DistillBERTBackbone=DistillBERT2025.12 | 0.664 | 0.497 | 0.568 | 0.542 | 0.366 | 0.405 | 0.678 | 0.497 | 0.555 | |
| FANTABackbone=GPT-4o-mini2025.12 | 0.654 | 0.81 | 0.724 | 0.564 | 0.768 | 0.632 | 0.708 | 0.81 | 0.732 | |
| FANTABackbone=Mistral2025.12 | 0.65 | 0.52 | 0.58 | 0.58 | 0.41 | 0.43 | 0.7 | 0.52 | 0.55 | |
| BERTBackbone=BERT2025.12 | 0.646 | 0.399 | 0.493 | 0.502 | 0.289 | 0.337 | 0.618 | 0.399 | 0.448 | |
| FANTABackbone=Gemma2025.12 | 0.4 | 0.42 | 0.41 | 0.45 | 0.38 | 0.32 | 0.59 | 0.42 | 0.37 | |
| GPT-4o-minimode=Zero-Shot2025.12 | 0.34 | 0.2 | 0.25 | 0.19 | 0.12 | 0.09 | 0.3 | 0.2 | 0.15 | |
| GPT-4o-minimode=Few-Shot2025.12 | 0.06 | 0.1 | 0.08 | 0.02 | 0.33 | 0.04 | 0.007 | 0.1 | 0.014 |