Formulation QA on Formulation QA (Standard)
58.7Accuracyw summary
Evaluation Results
| Method | Links | |
|---|---|---|
| w summaryToken=30*5, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 58.7 | |
| vanilla RAGToken=512*3, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 56.7 | |
| SciDCToken=30*5, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 56.7 | |
| w GPT-5Token=50*3, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 52.7 | |
| SciDCToken=30*5, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 47.5 | |
| w wikiToken=800*5, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 47.3 | |
| vanilla RAGToken=512*3, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 46.1 | |
| w summaryToken=30*5, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 46 | |
| w GPT-5Token=50*3, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 45.5 | |
| OriginalToken=0, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 43.3 | |
| w wikiToken=800*5, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 41.7 | |
| OriginalToken=0, Backbone=LLaMA-3.2-3B-Instruct2026.04 | 41.4 |