Question Difficulty Alignment on Human Evaluation Trial 70 question pairs (test)
50V1 ScoreMulti-IRT
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Multi-IRT2026.02 | 50 | 50 | 48.6 | 54.3 | 48.6 | 57.1 | 47.1 | 47.1 | 44.3 | 48.6 | 52.9 | |
| Simple Rank2026.02 | 41.4 | 54.3 | 50 | 60 | 51.4 | 54.3 | 54.3 | 50 | 50 | 52.9 | 62.9 | |
| 1PL-IRT2026.02 | 37.1 | 44.3 | 40 | 52.9 | 51.4 | 44.3 | 45.7 | 45.7 | 48.6 | 41.4 | 50 | |
| RankLLM2026.02 | 37.1 | 70 | 62.9 | 71.4 | 67.1 | 61.4 | 74.3 | 64.3 | 57.1 | 62.9 | 90 | |
| 2PL-IRT2026.02 | 35.7 | 45.7 | 41.4 | 54.3 | 52.9 | 42.9 | 44.3 | 47.1 | 47.1 | 42.9 | 51.4 |