Utility Judgment on TREC DL
63.18F1 ScoreITEM-As w. ExpA
Evaluation Results
| Method | Links | |
|---|---|---|
| ITEM-As w. ExpAapproach=Listwise, LLM=ChatGPT, m-value=32024.06 | 63.18 | |
| ITEM-As w. ExpAapproach=Listwise, LLM=ChatGPT, m-value=12024.06 | 62.52 | |
| ITEM-ARs w. ExpAapproach=Listwise, LLM=ChatGPT, m-value=12024.06 | 62 | |
| ITEM-ARs w. ExpAapproach=Listwise, LLM=ChatGPT, m-value=32024.06 | 61.37 | |
| ITEM-As w. ImpAapproach=Listwise, LLM=ChatGPT, m-value=32024.06 | 60.56 | |
| 5-samplingapproach=Listwise, LLM=ChatGPT2024.06 | 60.49 | |
| ITEM-As w. ImpAapproach=Listwise, LLM=ChatGPT, m-value=12024.06 | 58.11 | |
| UJ-ExpAapproach=Listwise, LLM=ChatGPT2024.06 | 57.49 | |
| ITEM-ARs w. ExpAapproach=Listwise, LLM=Mistral, m-value=32024.06 | 56.27 | |
| UJ-ImpAapproach=Listwise, LLM=ChatGPT2024.06 | 56.18 | |
| ITEM-As w. ExpAapproach=Listwise, LLM=Llama 3, m-value=32024.06 | 56.03 | |
| ITEM-As w. ExpAapproach=Pointwise, LLM=ChatGPT, m-value=32024.06 | 55.74 | |
| Vanillaapproach=Listwise, LLM=ChatGPT2024.06 | 55.19 | |
| ITEM-As w. ImpAapproach=Listwise, LLM=Llama 3, m-value=32024.06 | 55.14 | |
| ITEM-As w. ImpAapproach=Pointwise, LLM=ChatGPT, m-value=32024.06 | 54.9 | |
| ITEM-As w. ExpAapproach=Listwise, LLM=Mistral, m-value=32024.06 | 54.86 | |
| ITEM-As w. ImpAapproach=Pointwise, LLM=ChatGPT, m-value=12024.06 | 54.61 | |
| UJ-ExpAapproach=Listwise, LLM=Mistral2024.06 | 54.1 | |
| ITEM-As w. ExpAapproach=Listwise, LLM=Mistral, m-value=12024.06 | 53.78 | |
| ITEM-As w. ImpAapproach=Pointwise, LLM=Llama 3, m-value=32024.06 | 53.76 | |
| UJ-ExpAapproach=Pointwise, LLM=Llama 32024.06 | 53.73 | |
| ITEM-As w. ImpAapproach=Listwise, LLM=Llama 3, m-value=12024.06 | 53.73 | |
| ITEM-As w. ImpAapproach=Pointwise, LLM=Llama 3, m-value=12024.06 | 53.68 | |
| ITEM-As w. ExpAapproach=Listwise, LLM=Llama 3, m-value=12024.06 | 53.66 | |
| ITEM-As w. ExpAapproach=Pointwise, LLM=ChatGPT, m-value=12024.06 | 53.61 | |
| UJ-ExpAapproach=Listwise, LLM=Llama 32024.06 | 52.83 | |
| 5-samplingapproach=Listwise, LLM=Llama 32024.06 | 52.68 | |
| ITEM-As w. ExpAapproach=Pointwise, LLM=Llama 3, m-value=32024.06 | 52.46 | |
| ITEM-ARs w. ExpAapproach=Listwise, LLM=Mistral, m-value=12024.06 | 52.34 | |
| 5-samplingapproach=Listwise, LLM=Mistral2024.06 | 52.31 | |
| ITEM-ARs w. ExpAapproach=Listwise, LLM=Llama 3, m-value=32024.06 | 52.1 | |
| ITEM-As w. ExpAapproach=Pointwise, LLM=Llama 3, m-value=12024.06 | 52.09 | |
| ITEM-As w. ImpAapproach=Listwise, LLM=Mistral, m-value=32024.06 | 52.05 | |
| ITEM-As w. ExpAapproach=Pointwise, LLM=Mistral, m-value=32024.06 | 51.74 | |
| UJ-ImpAapproach=Pointwise, LLM=Llama 32024.06 | 50.2 | |
| Vanillaapproach=Pointwise, LLM=ChatGPT2024.06 | 49.84 | |
| ITEM-As w. ExpAapproach=Pointwise, LLM=Mistral, m-value=12024.06 | 49.44 | |
| Vanillaapproach=Listwise, LLM=Llama 32024.06 | 49.39 | |
| ITEM-As w. ImpAapproach=Listwise, LLM=Mistral, m-value=12024.06 | 49.39 | |
| ITEM-ARs w. ExpAapproach=Listwise, LLM=Llama 3, m-value=12024.06 | 48.97 | |
| UJ-ImpAapproach=Pointwise, LLM=ChatGPT2024.06 | 48.83 | |
| UJ-ImpAapproach=Pointwise, LLM=Mistral2024.06 | 48.31 | |
| UJ-ImpAapproach=Listwise, LLM=Mistral2024.06 | 48.29 | |
| UJ-ImpAapproach=Listwise, LLM=Llama 32024.06 | 48.22 | |
| UJ-ExpAapproach=Pointwise, LLM=ChatGPT2024.06 | 48.09 | |
| Vanillaapproach=Pointwise, LLM=Llama 32024.06 | 47.64 | |
| ITEM-As w. ImpAapproach=Pointwise, LLM=Mistral, m-value=32024.06 | 46.59 | |
| ITEM-As w. ImpAapproach=Pointwise, LLM=Mistral, m-value=12024.06 | 46.01 | |
| Vanillaapproach=Listwise, LLM=Mistral2024.06 | 45.67 | |
| Vanillaapproach=Pointwise, LLM=Mistral2024.06 | 45.11 | |
| UJ-ExpAapproach=Pointwise, LLM=Mistral2024.06 | 43.53 |