Binary Novelty Classification on Novelty Classification Dataset (test)
74.4AccuracyFine-tuned SciBERT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Fine-tuned SciBERTBase Model=scibert-scivocab-uncased, Input Features=Title, Abstract, Categories, SPECTER2, Proximity embedding, Training=Multi-modal fine-tuning2025.12 | 74.4 | 18.7 | 31.3 | 23.4 | |
| SFT Qwen3-4BBase Model=Qwen3-4B, Optimization=Supervised Fine-Tuning (SFT), Loss=Cross-entropy2025.12 | 62.7 | 19.4 | 63.2 | 29.7 | |
| DPO Qwen3-4BBase Model=Qwen3-4B, Optimization=Direct Preference Optimization (DPO), Initial Checkpoint=SFT checkpoint2025.12 | 61.2 | 20.5 | 73.5 | 32.1 | |
| GPT-5.1Model Type=Large-Scale Model, Training=Zero-shot / No specific training2025.12 | 24.2 | 12 | 98.6 | 21.5 |