# Utility Judgment on GTI-NQ Source: [Utility Judgment on GTI-NQ]() Collection index: [llms.txt]() Data updated: 2026-04-14T07:00:28.668Z Evaluation of model performance on the GTI-NQ dataset using the F1 score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Utility Judgment | | Dataset (including any stated split) | GTI-NQ | | Metric | F1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 34 of 34 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | F1 Score | Result date | Report paper | | --- | --- | --- | --- | | ITEM-ARs-ExpA | 76.34 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-ARs-ExpA | 74.43 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-ExpA | 73.55 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-ExpA | 72.44 | Not recorded | 68fa7aba1e209131cea84d8e | | k-sampling | 71.17 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-Imp | 69.68 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-Imp | 68.92 | Not recorded | 68fa7aba1e209131cea84d8e | | UJ-ExapA | 66.13 | Not recorded | 68fa7aba1e209131cea84d8e | | Vanilla | 59.37 | Not recorded | 68fa7aba1e209131cea84d8e | | UJ-ImpA | 57.4 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-ExpA | 55.45 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-ExpA | 54.89 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-ARs-ExpA | 51.22 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-ARs-ExpA | 50.77 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-ExpA | 49.73 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-ExpA | 49.26 | Not recorded | 68fa7aba1e209131cea84d8e | | k-sampling | 49.2 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-ExpA | 48.9 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-Imp | 48.03 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-ExpA | 47.52 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-Imp | 47.47 | Not recorded | 68fa7aba1e209131cea84d8e | | UJ-ExapA | 47.07 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-Imp | 43.58 | Not recorded | 68fa7aba1e209131cea84d8e | | Vanilla | 43.38 | Not recorded | 68fa7aba1e209131cea84d8e | | UJ-ImpA | 43.31 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-Imp | 43.17 | Not recorded | 68fa7aba1e209131cea84d8e | | UJ-ExapA | 39.32 | Not recorded | 68fa7aba1e209131cea84d8e | | UJ-ImpA | 38.72 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-Imp | 38.34 | Not recorded | 68fa7aba1e209131cea84d8e | | ITEM-As-Imp | 37.98 | Not recorded | 68fa7aba1e209131cea84d8e | | UJ-ImpA | 37.29 | Not recorded | 68fa7aba1e209131cea84d8e | | UJ-ExapA | 37.17 | Not recorded | 68fa7aba1e209131cea84d8e | | Vanilla | 35.31 | Not recorded | 68fa7aba1e209131cea84d8e | | Vanilla | 28.55 | Not recorded | 68fa7aba1e209131cea84d8e | ### ITEM-ARs-ExpA F1 Score: 76.34. Result ID: 69dde4c647f34ce924c4c833. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Listwise","m-value":"3"}. This record does not assert independent reproduction. ### ITEM-ARs-ExpA F1 Score: 74.43. Result ID: 69dde4c647f34ce924c4c829. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Listwise","m-value":"1"}. This record does not assert independent reproduction. ### ITEM-As-ExpA F1 Score: 73.55. Result ID: 69dde4c647f34ce924c4c82c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Listwise","m-value":"3"}. This record does not assert independent reproduction. ### ITEM-As-ExpA F1 Score: 72.44. Result ID: 69dde4c647f34ce924c4c822. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Listwise","m-value":"1"}. This record does not assert independent reproduction. ### k-sampling F1 Score: 71.17. Result ID: 69dde4c647f34ce924c4c81f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Listwise"}. This record does not assert independent reproduction. ### ITEM-As-Imp F1 Score: 69.68. Result ID: 69dde4c647f34ce924c4c830. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Listwise","m-value":"3"}. This record does not assert independent reproduction. ### ITEM-As-Imp F1 Score: 68.92. Result ID: 69dde4c647f34ce924c4c826. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Listwise","m-value":"1"}. This record does not assert independent reproduction. ### UJ-ExapA F1 Score: 66.13. Result ID: 69dde4c647f34ce924c4c818. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Listwise"}. This record does not assert independent reproduction. ### Vanilla F1 Score: 59.37. Result ID: 69dde4c647f34ce924c4c814. [Reporting paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Listwise"}. This record does not assert independent reproduction. ### UJ-ImpA F1 Score: 57.4. Result ID: 69dde4c647f34ce924c4c81c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Listwise"}. This record does not assert independent reproduction. ### ITEM-As-ExpA F1 Score: 55.45. Result ID: 69dde4c647f34ce924c4c82d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Pointwise","m-value":"3"}. This record does not assert independent reproduction. ### ITEM-As-ExpA F1 Score: 54.89. Result ID: 69dde4c647f34ce924c4c823. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Pointwise","m-value":"1"}. This record does not assert independent reproduction. ### ITEM-ARs-ExpA F1 Score: 51.22. Result ID: 69dde4c647f34ce924c4c832. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Listwise","m-value":"3"}. This record does not assert independent reproduction. ### ITEM-ARs-ExpA F1 Score: 50.77. Result ID: 69dde4c647f34ce924c4c828. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Listwise","m-value":"1"}. This record does not assert independent reproduction. ### ITEM-As-ExpA F1 Score: 49.73. Result ID: 69dde4c647f34ce924c4c82a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Listwise","m-value":"3"}. This record does not assert independent reproduction. ### ITEM-As-ExpA F1 Score: 49.26. Result ID: 69dde4c647f34ce924c4c820. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Listwise","m-value":"1"}. This record does not assert independent reproduction. ### k-sampling F1 Score: 49.2. Result ID: 69dde4c647f34ce924c4c81e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Listwise"}. This record does not assert independent reproduction. ### ITEM-As-ExpA F1 Score: 48.9. Result ID: 69dde4c647f34ce924c4c82b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Pointwise","m-value":"3"}. This record does not assert independent reproduction. ### ITEM-As-Imp F1 Score: 48.03. Result ID: 69dde4c647f34ce924c4c82e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Listwise","m-value":"3"}. This record does not assert independent reproduction. ### ITEM-As-ExpA F1 Score: 47.52. Result ID: 69dde4c647f34ce924c4c821. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Pointwise","m-value":"1"}. This record does not assert independent reproduction. ### ITEM-As-Imp F1 Score: 47.47. Result ID: 69dde4c647f34ce924c4c824. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Listwise","m-value":"1"}. This record does not assert independent reproduction. ### UJ-ExapA F1 Score: 47.07. Result ID: 69dde4c647f34ce924c4c816. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Listwise"}. This record does not assert independent reproduction. ### ITEM-As-Imp F1 Score: 43.58. Result ID: 69dde4c647f34ce924c4c831. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Pointwise","m-value":"3"}. This record does not assert independent reproduction. ### Vanilla F1 Score: 43.38. Result ID: 69dde4c647f34ce924c4c812. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Listwise"}. This record does not assert independent reproduction. ### UJ-ImpA F1 Score: 43.31. Result ID: 69dde4c647f34ce924c4c81a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Listwise"}. This record does not assert independent reproduction. ### ITEM-As-Imp F1 Score: 43.17. Result ID: 69dde4c647f34ce924c4c827. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Pointwise","m-value":"1"}. This record does not assert independent reproduction. ### UJ-ExapA F1 Score: 39.32. Result ID: 69dde4c647f34ce924c4c817. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Pointwise"}. This record does not assert independent reproduction. ### UJ-ImpA F1 Score: 38.72. Result ID: 69dde4c647f34ce924c4c81b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Pointwise"}. This record does not assert independent reproduction. ### ITEM-As-Imp F1 Score: 38.34. Result ID: 69dde4c647f34ce924c4c82f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Pointwise","m-value":"3"}. This record does not assert independent reproduction. ### ITEM-As-Imp F1 Score: 37.98. Result ID: 69dde4c647f34ce924c4c825. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Pointwise","m-value":"1"}. This record does not assert independent reproduction. ### UJ-ImpA F1 Score: 37.29. Result ID: 69dde4c647f34ce924c4c81d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Pointwise"}. This record does not assert independent reproduction. ### UJ-ExapA F1 Score: 37.17. Result ID: 69dde4c647f34ce924c4c819. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Pointwise"}. This record does not assert independent reproduction. ### Vanilla F1 Score: 35.31. Result ID: 69dde4c647f34ce924c4c815. [Reporting paper]() Recorded evaluation configuration: {"LLM":"ChatGPT","Approach":"Pointwise"}. This record does not assert independent reproduction. ### Vanilla F1 Score: 28.55. Result ID: 69dde4c647f34ce924c4c813. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Llama3-8B","Approach":"Pointwise"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.