Attribute Relevance Selection on Attribute Relevance Selection Human Majority Judgments (test)
87AccuracyQwen3-4B + GRPO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen3-4B + GRPOmode=Training-based, optimization=GRPO2026.06 | 87 | 61.1 | 67.4 | 64.1 | |
| RoBERTamode=Training-based, role=binary relevance predictor2026.06 | 85.9 | 60.8 | 60.5 | 60.6 | |
| Claude-Sonnet-4.6mode=Zero-shot2026.06 | 71.9 | 37.2 | 83.8 | 51.5 | |
| Qwen3.5-27Bmode=Zero-shot2026.06 | 68.9 | 34.5 | 83.2 | 48.8 | |
| Gemma-4-31Bmode=Zero-shot2026.06 | 68.5 | 35.5 | 93.4 | 51.4 | |
| GPT-5.4mode=Zero-shot2026.06 | 67.6 | 33.4 | 81.9 | 47.4 | |
| Qwen3-4Bmode=Zero-shot2026.06 | 53 | 26.7 | 94.2 | 41.7 | |
| Llama-3.3-70Bmode=Zero-shot2026.06 | 52.9 | 27.2 | 97.9 | 42.6 |