Biomedical Question Answering on PubMedQA (Accuracy, TTFT, Delay)
68.32AccuracyNemotron-CC_ASI
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Nemotron-CC_ASIPre-training Curation Strategy=AI Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 68.32 | — | — | |
| Nemotron-CCPre-training Curation Strategy=AI Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 67.68 | — | — | |
| Nemotron-CC_ASI+Pre-training Curation Strategy=AI Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 67.68 | — | — | |
| Fineweb-EduPre-training Curation Strategy=Human Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 67.04 | — | — | |
| DCLMPre-training Curation Strategy=Human Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 66.56 | — | — | |
| StreamingThinkerParadigm=Streaming, Reasoning Depth=D3, Model=Qwen3-4B2025.10 | 65.3 | 21.74 | 6.76 | |
| InterleavedParadigm=Interleaved, Reasoning Depth=D3, Model=Qwen3-4B2025.10 | 64.6 | 21.74 | 17.45 | |
| Ultra-FinewebPre-training Curation Strategy=Human Discovered, Model Parameters=3B, Training Token Budget=500B tokens2026.03 | 64.44 | — | — | |
| StreamingThinkerParadigm=Streaming, Reasoning Depth=D2, Model=Qwen3-4B2025.10 | 64.1 | 21.74 | 4.92 | |
| InterleavedParadigm=Interleaved, Reasoning Depth=D2, Model=Qwen3-4B2025.10 | 63.3 | 21.74 | 15.71 | |
| BatchParadigm=Batch, Model=Qwen3-4B2025.10 | 60.9 | 357.468 | 15.29 | |
| StreamingThinkerParadigm=Streaming, Reasoning Depth=D1, Model=Qwen3-4B2025.10 | 57.9 | 21.74 | 0.74 | |
| InterleavedParadigm=Interleaved, Reasoning Depth=D1, Model=Qwen3-4B2025.10 | 57.1 | 21.74 | 11.09 |