# Context-aware Instruction Following on Context-aware Instructions (test) Source: [Context-aware Instruction Following on Context-aware Instructions (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:54:39.231Z Evaluation of model performance on context-aware instruction following tasks using the test split of the Context-aware Instructions dataset, measured by Overlap (with/without context) and Perplexity. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Context-aware Instruction Following | | Dataset (including any stated split) | Context-aware Instructions (test) | | Metric | Overlap (with context) | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Overlap (with context)](): Higher is better; unit: SCALAR. - [Perplexity](): Lower is better; unit: SCALAR. - [Overlap (without context)](): Higher is better; unit: SCALAR. ## Recorded results | Method | Overlap (with context) | Result date | Report paper | | --- | --- | --- | --- | | GPT-4-turbo | 8.85 | Not recorded | 68fa738b1e209131cea5d1d0 | | Qwen-Chat(v1.5) | 8.7 | Not recorded | 68fa738b1e209131cea5d1d0 | | StableLM-Zephyr + GPT-4-turbo (Sketch-based CoGenesis) | 8.48 | Not recorded | 68fa738b1e209131cea5d1d0 | | Qwen-Chat(v1) | 8.38 | Not recorded | 68fa738b1e209131cea5d1d0 | | GPT-3.5-turbo | 8.3 | Not recorded | 68fa738b1e209131cea5d1d0 | | StableLM-Zephyr | 8.3 | Not recorded | 68fa738b1e209131cea5d1d0 | | Qwen-Chat(v1.5) + Qwen-Chat(v1.5) (Logit-based CoGenesis) | 8.28 | Not recorded | 68fa738b1e209131cea5d1d0 | | Qwen-Chat(v1.5) + GPT-4-turbo (Sketch-based CoGenesis) | 8.18 | Not recorded | 68fa738b1e209131cea5d1d0 | | Mixtral-8x7b | 8.12 | Not recorded | 68fa738b1e209131cea5d1d0 | | Qwen-Chat(v1) + Qwen-Chat(v1) (Logit-based CoGenesis) | 8.12 | Not recorded | 68fa738b1e209131cea5d1d0 | | H2O-danube-chat + GPT-4-turbo (Sketch-based CoGenesis) | 7.84 | Not recorded | 68fa738b1e209131cea5d1d0 | | Qwen-Chat(v1) + GPT-4-turbo (Sketch-based CoGenesis) | 7.8 | Not recorded | 68fa738b1e209131cea5d1d0 | | Llama-2-Chat | 7.78 | Not recorded | 68fa738b1e209131cea5d1d0 | | TinyLlama-Chat + Llama-2-Chat (Logit-based CoGenesis) | 7.76 | Not recorded | 68fa738b1e209131cea5d1d0 | | TinyLlama-Chat + GPT-4-turbo (Sketch-based CoGenesis) | 7.66 | Not recorded | 68fa738b1e209131cea5d1d0 | | StableLM-Zephyr | 6.88 | Not recorded | 68fa738b1e209131cea5d1d0 | | H2O-danube-chat | 6.56 | Not recorded | 68fa738b1e209131cea5d1d0 | | GPT-4-turbo | 6.1 | Not recorded | 68fa738b1e209131cea5d1d0 | ### GPT-4-turbo Overlap (with context): 8.85. Result ID: 6992ef833a6c314849127819. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mode":"zero-shot LLM with context (upper bound)","Params":"N/A"}. This record does not assert independent reproduction. ### Qwen-Chat(v1.5) Overlap (with context): 8.7. Result ID: 6992ef833a6c31484912781d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"zero-shot LLM with context (upper bound)","Params":"72B"}. This record does not assert independent reproduction. ### StableLM-Zephyr + GPT-4-turbo (Sketch-based CoGenesis) Overlap (with context): 8.48. Result ID: 6992ef833a6c314849127823. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"mixed-scale collaboration","Params":"mixed"}. This record does not assert independent reproduction. ### Qwen-Chat(v1) Overlap (with context): 8.38. Result ID: 6992ef833a6c31484912781c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"zero-shot LLM with context (upper bound)","Params":"72B"}. This record does not assert independent reproduction. ### GPT-3.5-turbo Overlap (with context): 8.3. Result ID: 6992ef833a6c31484912781a. [Reporting paper]() Recorded evaluation configuration: {"Mode":"zero-shot LLM with context (upper bound)","Params":"N/A"}. This record does not assert independent reproduction. ### StableLM-Zephyr Overlap (with context): 8.3. Result ID: 6992ef833a6c314849127822. [Reporting paper]() Recorded evaluation configuration: {"Mode":"finetuned SLM (+ LLM) with context","Params":"1.6B"}. This record does not assert independent reproduction. ### Qwen-Chat(v1.5) + Qwen-Chat(v1.5) (Logit-based CoGenesis) Overlap (with context): 8.28. Result ID: 6992ef833a6c31484912782a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"mixed-scale collaboration","Params":"mixed"}. This record does not assert independent reproduction. ### Qwen-Chat(v1.5) + GPT-4-turbo (Sketch-based CoGenesis) Overlap (with context): 8.18. Result ID: 6992ef833a6c314849127829. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"mixed-scale collaboration","Params":"mixed"}. This record does not assert independent reproduction. ### Mixtral-8x7b Overlap (with context): 8.12. Result ID: 6992ef833a6c31484912781e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mode":"zero-shot LLM with context (upper bound)","Params":"47B"}. This record does not assert independent reproduction. ### Qwen-Chat(v1) + Qwen-Chat(v1) (Logit-based CoGenesis) Overlap (with context): 8.12. Result ID: 6992ef833a6c314849127828. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"mixed-scale collaboration","Params":"mixed"}. This record does not assert independent reproduction. ### H2O-danube-chat + GPT-4-turbo (Sketch-based CoGenesis) Overlap (with context): 7.84. Result ID: 6992ef833a6c314849127824. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"mixed-scale collaboration","Params":"mixed"}. This record does not assert independent reproduction. ### Qwen-Chat(v1) + GPT-4-turbo (Sketch-based CoGenesis) Overlap (with context): 7.8. Result ID: 6992ef833a6c314849127827. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"mixed-scale collaboration","Params":"mixed"}. This record does not assert independent reproduction. ### Llama-2-Chat Overlap (with context): 7.78. Result ID: 6992ef833a6c31484912781b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mode":"zero-shot LLM with context (upper bound)","Params":"70B"}. This record does not assert independent reproduction. ### TinyLlama-Chat + Llama-2-Chat (Logit-based CoGenesis) Overlap (with context): 7.76. Result ID: 6992ef833a6c314849127826. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"mixed-scale collaboration","Params":"mixed"}. This record does not assert independent reproduction. ### TinyLlama-Chat + GPT-4-turbo (Sketch-based CoGenesis) Overlap (with context): 7.66. Result ID: 6992ef833a6c314849127825. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"mixed-scale collaboration","Params":"mixed"}. This record does not assert independent reproduction. ### StableLM-Zephyr Overlap (with context): 6.88. Result ID: 6992ef833a6c314849127820. [Reporting paper]() Recorded evaluation configuration: {"Mode":"zero-shot SLM with context","Params":"1.6B"}. This record does not assert independent reproduction. ### H2O-danube-chat Overlap (with context): 6.56. Result ID: 6992ef833a6c314849127821. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"zero-shot SLM with context","Params":"1.8B"}. This record does not assert independent reproduction. ### GPT-4-turbo Overlap (with context): 6.1. Result ID: 6992ef833a6c31484912781f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mode":"zero-shot LLM w/o context (lower bound)","Params":"N/A"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.