Complex Reasoning on SciFact (test)
76.17Macro-F1LLM annotation
Evaluation results
| Method | Links | |
|---|---|---|
| LLM annotationBackbone=gpt-4o-mini2026.06 | 76.17 | |
| LLM-EvalDownstream Model=RoBERTa-large2026.06 | 76.17 | |
| LLM-EvalDownstream Model=Qwen3-1.7B2026.06 | 76.17 | |
| LLM-EvalDownstream Model=Llama-3.1-8B2026.06 | 76.17 | |
| LLM-AnnotationDownstream Model=RoBERTa-large2026.06 | 59.19 | |
| EvoPoolBackbone=gpt-4o-mini2026.06 | 58.9 | |
| LLM-AnnotationBackbone=RoBERTa-large2026.06 | 51.34 | |
| EvoPoolBackbone=Llama-3.1-8B2026.06 | 50.88 | |
| LLM-AnnotationDownstream Model=Llama-3.1-8B2026.06 | 47.88 | |
| LLM-AnnotationBackbone=Qwen2-1.5B2026.06 | 46.97 | |
| LLM-AnnotationBackbone=Llama-3.1-8B2026.06 | 46.47 | |
| EvoPoolDownstream Model=Llama-3.1-8B2026.06 | 44.42 | |
| LLM-AnnotationDownstream Model=Qwen3-1.7B2026.06 | 43.55 | |
| Self-TrainDownstream Model=Qwen3-1.7B2026.06 | 42.05 | |
| GoldenDownstream Model=Llama-3.1-8B2026.06 | 40.96 | |
| Self-TrainDownstream Model=Llama-3.1-8B2026.06 | 40.6 | |
| EvoPoolBackbone=Qwen2-1.5B2026.06 | 38.71 | |
| EvoPoolDownstream Model=Qwen3-1.7B2026.06 | 37.97 | |
| DataSculptBackbone=Llama-3.1-8B2026.06 | 36.09 | |
| Self-TrainDownstream Model=RoBERTa-large2026.06 | 35.53 | |
| AlchemistBackbone=Llama-3.1-8B2026.06 | 32.48 | |
| DataSculptDownstream Model=RoBERTa-large2026.06 | 31.59 | |
| DataSculptDownstream Model=Qwen3-1.7B2026.06 | 31.11 | |
| EvoPoolBackbone=RoBERTa-large2026.06 | 29.81 | |
| GoldenDownstream Model=Qwen3-1.7B2026.06 | 29.29 | |
| AlchemistDownstream Model=Llama-3.1-8B2026.06 | 29.03 | |
| DataSculptBackbone=Qwen2-1.5B2026.06 | 29.01 | |
| EvoPoolDownstream Model=RoBERTa-large2026.06 | 28.9 | |
| GoldenDownstream Model=RoBERTa-large2026.06 | 28.64 | |
| AlchemistBackbone=gpt-4o-mini2026.06 | 28.55 | |
| DataSculptDownstream Model=Llama-3.1-8B2026.06 | 27.59 | |
| AlchemistDownstream Model=Qwen3-1.7B2026.06 | 27.43 | |
| AlchemistBackbone=Qwen2-1.5B2026.06 | 26.13 | |
| DataSculptBackbone=RoBERTa-large2026.06 | 21.12 | |
| AlchemistBackbone=RoBERTa-large2026.06 | 20.51 | |
| AlchemistDownstream Model=RoBERTa-large2026.06 | 20.51 | |
| DataSculptBackbone=gpt-4o-mini2026.06 | 20 |