Context-aware Instruction Following on Context-aware Instructions (test)
8.85Overlap (with context)GPT-4-turbo
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4-turboMode=zero-shot LLM with context (upper bound), Params=N/A2024.03 | 8.85 | 8.9 | 8.54 | |
| Qwen-Chat(v1.5)Mode=zero-shot LLM with context (upper bound), Params=72B2024.03 | 8.7 | 8.67 | 8.26 | |
| StableLM-Zephyr + GPT-4-turbo (Sketch-based CoGenesis)Mode=mixed-scale collaboration, Params=mixed2024.03 | 8.48 | 8.56 | 7.98 | |
| Qwen-Chat(v1)Mode=zero-shot LLM with context (upper bound), Params=72B2024.03 | 8.38 | 8.38 | 8.14 | |
| GPT-3.5-turboMode=zero-shot LLM with context (upper bound), Params=N/A2024.03 | 8.3 | 8.33 | 7.58 | |
| StableLM-ZephyrMode=finetuned SLM (+ LLM) with context, Params=1.6B2024.03 | 8.3 | 8.56 | 7.73 | |
| Qwen-Chat(v1.5) + Qwen-Chat(v1.5) (Logit-based CoGenesis)Mode=mixed-scale collaboration, Params=mixed2024.03 | 8.28 | 8.22 | 7.8 | |
| Qwen-Chat(v1.5) + GPT-4-turbo (Sketch-based CoGenesis)Mode=mixed-scale collaboration, Params=mixed2024.03 | 8.18 | 7.98 | 7.62 | |
| Mixtral-8x7bMode=zero-shot LLM with context (upper bound), Params=47B2024.03 | 8.12 | 8.22 | 7.96 | |
| Qwen-Chat(v1) + Qwen-Chat(v1) (Logit-based CoGenesis)Mode=mixed-scale collaboration, Params=mixed2024.03 | 8.12 | 8.2 | 7.86 | |
| H2O-danube-chat + GPT-4-turbo (Sketch-based CoGenesis)Mode=mixed-scale collaboration, Params=mixed2024.03 | 7.84 | 7.78 | 7.14 | |
| Qwen-Chat(v1) + GPT-4-turbo (Sketch-based CoGenesis)Mode=mixed-scale collaboration, Params=mixed2024.03 | 7.8 | 7.82 | 7.64 | |
| Llama-2-ChatMode=zero-shot LLM with context (upper bound), Params=70B2024.03 | 7.78 | 7.98 | 7.96 | |
| TinyLlama-Chat + Llama-2-Chat (Logit-based CoGenesis)Mode=mixed-scale collaboration, Params=mixed2024.03 | 7.76 | 7.74 | 7.06 | |
| TinyLlama-Chat + GPT-4-turbo (Sketch-based CoGenesis)Mode=mixed-scale collaboration, Params=mixed2024.03 | 7.66 | 7.14 | 6.82 | |
| StableLM-ZephyrMode=zero-shot SLM with context, Params=1.6B2024.03 | 6.88 | 6.82 | 6.68 | |
| H2O-danube-chatMode=zero-shot SLM with context, Params=1.8B2024.03 | 6.56 | 6.94 | 6.6 | |
| GPT-4-turboMode=zero-shot LLM w/o context (lower bound), Params=N/A2024.03 | 6.1 | 6.46 | 8.75 |