Query Suggestion on EQS-Benchmark (offline)
86.1Strict AccuracyCold-EQS
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Cold-EQSModel Category=Ours, Backbone=Qwen3-4B2026.03 | 86.1 | 90.6 | 94.7 | 98.3 | 97 | |
| SFT-GRPO-Qwen3-4BModel Category=Ours, Training Strategy=SFT-GRPO, Backbone=Qwen3-4B2026.03 | 76.9 | 85.4 | 92.6 | 96 | 94.9 | |
| Qwen-flashModel Category=Closed-Source Large Language Models2026.03 | 75.4 | 81.3 | 85.4 | 98.6 | 96.4 | |
| GPT-4.1-miniModel Category=Closed-Source Large Language Models2026.03 | 70.5 | 79.2 | 84.9 | 99 | 94.3 | |
| Gemini-3-flashModel Category=Closed-Source Large Language Models2026.03 | 56 | 62.9 | 64.2 | 77.4 | 75.8 | |
| GPT-5-miniModel Category=Closed-Source Large Language Models2026.03 | 55.9 | 61.8 | 63.3 | 88.8 | 87.9 | |
| GRPO-Qwen3-4BModel Category=Ours, Training Strategy=GRPO, Backbone=Qwen3-4B2026.03 | 51.9 | 71 | 86.7 | 96.8 | 84.5 | |
| SFT-Qwen3-4BModel Category=Ours, Training Strategy=SFT, Backbone=Qwen3-4B2026.03 | 50.5 | 68.8 | 82.3 | 95.7 | 86.4 | |
| Qwen3-8BModel Category=Open-Source Large Language Models, Parameter Scale=8B2026.03 | 42.4 | 59.9 | 69.8 | 95.2 | 86.6 | |
| Gemini-2.5-flashModel Category=Closed-Source Large Language Models2026.03 | 38.9 | 55 | 63.2 | 85 | 77 | |
| Qwen3-4BModel Category=Open-Source Large Language Models, Parameter Scale=4B2026.03 | 36.7 | 53.2 | 62.1 | 95.1 | 86.6 | |
| Gpt-4o-miniModel Category=Closed-Source Large Language Models2026.03 | 34.9 | 53 | 60.6 | 99.3 | 92 | |
| DeepSeek-R1-Qwen3-8BModel Category=Open-Source Large Language Models, Backbone=Qwen3-8B2026.03 | 32.7 | 47.1 | 56 | 80.7 | 71.6 | |
| DeepSeek-R1-Llama-8BModel Category=Open-Source Large Language Models, Backbone=Llama-8B2026.03 | 27.9 | 44.4 | 54.7 | 97.8 | 88 | |
| Qwen3-1.7BModel Category=Open-Source Large Language Models, Parameter Scale=1.7B2026.03 | 22.4 | 47.6 | 62.8 | 96.4 | 82.1 | |
| Qwen3-0.6BModel Category=Open-Source Large Language Models, Parameter Scale=0.6B2026.03 | 11.8 | 33.6 | 55.1 | 95.5 | 73.1 |