User preference prediction on Steam Pref (test)
73.6AccuracyEP+SEM_CRIT
Evaluation Results
| Method | Links | |
|---|---|---|
| EP+SEM_CRITModel=gpt-4o-mini-2024-07-182025.10 | 73.6 | |
| EP_CRITModel=gpt-4o-mini-2024-07-182025.10 | 71.2 | |
| EP_CRITModel=gpt-5-2025-08-072025.10 | 69.2 | |
| EP_CRITModel=gpt-5-2025-08-072025.10 | 69.2 | |
| EP+SEM_CRITModel=gpt-5-2025-08-072025.10 | 68.8 | |
| EP+SEM_CRITModel=gpt-5-2025-08-072025.10 | 68.8 | |
| EP_CRITModel=qwen3-235b-a22b-instruct-25072025.10 | 68.4 | |
| EP+SEM_CRITModel=qwen3-235b-a22b-instruct-25072025.10 | 68.4 | |
| EP_LABELModel=llama-4-scout2025.10 | 66.8 | |
| EP_LABELModel=gpt-5-2025-08-072025.10 | 64.4 | |
| SEM_CRITModel=gpt-5-2025-08-072025.10 | 64.4 | |
| EP_CRITModel=qwen3-vl-235b-a22b-thinking2025.10 | 64.4 | |
| EP_LABELModel=gpt-5-2025-08-072025.10 | 64.4 | |
| SEM_CRITModel=gpt-5-2025-08-072025.10 | 64.4 | |
| EP_CRITModel=qwen3-vl-235b-a22b-thinking2025.10 | 64.4 | |
| SEM_CRITModel=qwen3-235b-a22b-instruct-25072025.10 | 64 | |
| EP+SEM_CRITModel=llama-4-scout2025.10 | 63.6 | |
| EP_CRITModel=gpt-oss-20b2025.10 | 63.2 | |
| SEM_CRITModel=llama-4-scout2025.10 | 63.2 | |
| EP_CRITModel=gpt-oss-20b2025.10 | 63.2 | |
| EP+SEM_CRITModel=gpt-oss-20b2025.10 | 62.8 | |
| EP+SEM_CRITModel=gpt-oss-20b2025.10 | 62.8 | |
| EP_CRITModel=llama-4-scout2025.10 | 62.4 | |
| EP+SEM_CRITModel=qwen3-vl-235b-a22b-thinking2025.10 | 62 | |
| EP+SEM_CRITModel=qwen3-vl-235b-a22b-thinking2025.10 | 62 | |
| EP_LABELModel=gpt-4o-mini-2024-07-182025.10 | 61.2 | |
| SEM_CRITModel=gpt-4o-mini-2024-07-182025.10 | 60.4 | |
| zero_shotModel=qwen3-vl-235b-a22b-thinking2025.10 | 58.8 | |
| zero_shotModel=qwen3-vl-235b-a22b-thinking2025.10 | 58.8 | |
| zero_shotModel=gpt-4o-mini-2024-07-182025.10 | 58.4 | |
| EP_LABELModel=gpt-oss-20b2025.10 | 57.2 | |
| SEM_CRITModel=gpt-oss-20b2025.10 | 57.2 | |
| SEM_CRITModel=qwen3-vl-235b-a22b-thinking2025.10 | 57.2 | |
| EP_LABELModel=gpt-oss-20b2025.10 | 57.2 | |
| SEM_CRITModel=gpt-oss-20b2025.10 | 57.2 | |
| SEM_CRITModel=qwen3-vl-235b-a22b-thinking2025.10 | 57.2 | |
| EP_LABELModel=qwen3-vl-235b-a22b-thinking2025.10 | 56.6 | |
| EP_LABELModel=qwen3-vl-235b-a22b-thinking2025.10 | 56.6 | |
| zero_shotModel=llama-4-scout2025.10 | 56.4 | |
| EP_LABELModel=qwen3-235b-a22b-instruct-25072025.10 | 55.6 | |
| zero_shotModel=qwen3-235b-a22b-instruct-25072025.10 | 54.8 | |
| zero_shotModel=gpt-5-2025-08-072025.10 | 53.6 | |
| zero_shotModel=gpt-5-2025-08-072025.10 | 53.6 | |
| zero_shotModel=gpt-oss-20b2025.10 | 52.4 | |
| zero_shotModel=gpt-oss-20b2025.10 | 52.4 |