SCPI extraction on SCPI extraction corpus
84Annotator 1 SCPI Scoregpt-oss-120b
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| gpt-oss-120bVector=None, Label=disability2026.06 | 84 | 8 | 8 | 82 | 14 | 4 | 86 | |
| gpt-oss-120bVector=None, Label=average2026.06 | 71 | 19 | 10 | 69 | 23 | 9 | 82 | |
| gpt-oss-120bVector=None, Label=crime2026.06 | 66 | 26 | 8 | 68 | 24 | 8 | 84 | |
| gpt-oss-120bVector=None, Label=medical history2026.06 | 62 | 24 | 14 | 56 | 30 | 14 | 76 | |
| ModernBERTVector=None, Label=crime, class weight=None2026.06 | 56 | 36 | 8 | 52 | 42 | 6 | 82 | |
| ModernBERTVector=None, Label=disability, class weight=None2026.06 | 48 | 50 | 2 | 52 | 48 | 0 | 86 | |
| ModernBERTVector=None, Label=average, class weight=None2026.06 | 47 | 47 | 6 | 45 | 51 | 3 | 85 | |
| ModernBERTVector=None, Label=medical history, class weight=None2026.06 | 38 | 54 | 8 | 32 | 64 | 4 | 86 | |
| SVCVector=Doc2Vec, Label=crime, class weight=None2026.06 | 36 | 52 | 12 | 28 | 60 | 12 | 66 | |
| LGBMVector=TF-IDF, Label=crime, class weight=balanced2026.06 | 36 | 56 | 8 | 36 | 52 | 12 | 74 | |
| SVCVector=Doc2Vec, Label=medical history, class weight=None2026.06 | 32 | 52 | 16 | 24 | 64 | 12 | 84 | |
| SVCVector=Doc2Vec, Label=average, class weight=None2026.06 | 31 | 53 | 16 | 31 | 55 | 14 | 73 | |
| SVCVector=Doc2Vec, Label=disability, class weight=None2026.06 | 24 | 56 | 20 | 40 | 42 | 18 | 68 | |
| LGBMVector=TF-IDF, Label=average, class weight=balanced2026.06 | 23 | 65 | 13 | 22 | 69 | 9 | 79 | |
| LGBMVector=TF-IDF, Label=medical history, class weight=balanced2026.06 | 16 | 76 | 8 | 14 | 78 | 8 | 86 | |
| LGBMVector=TF-IDF, Label=disability, class weight=balanced2026.06 | 16 | 62 | 22 | 16 | 78 | 6 | 76 |