Logical reasoning multi-choice QA on LogiQA v2 (test)
55.5Macro F1 ScoreT5-large
Evaluation results
| Method | Links | ||
|---|---|---|---|
| T5-largeWarmup=true2025.02 | 55.5 | 55.62 | |
| T5-largeWarmup=false2025.02 | 54.46 | 54.54 | |
| T5-baseWarmup=true2025.02 | 50 | 50.06 | |
| T5-baseWarmup=false2025.02 | 49.16 | 49.19 | |
| Llama-3.2-1BWarmup=true2025.02 | 32.63 | 33.55 | |
| Llama-3.2-1BWarmup=false2025.02 | 30.12 | 31.7 |