Sponsored Recommendation Evaluation on Sponsored recommendation dataset 1,000-row and 200-row pools
81Sponsored Recommendation RateQwen3.5-9B
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3.5-9Bcounter-prompt strategy=baseline, sample size (n)=1002026.05 | 81 | |
| Qwen3.6-35B-A3B-FP8counter-prompt strategy=baseline, sample size (n)=1002026.05 | 73 | |
| gpt-3.5-turbo (OpenAI)counter-prompt strategy=baseline, sample size (n)=1002026.05 | 61 | |
| 2 OpenAI avg.counter-prompt strategy=baseline, sample size (n)=2002026.05 | 53 | |
| Mistral-Small-3.2-24Bcounter-prompt strategy=baseline, sample size (n)=1002026.05 | 50 | |
| granite-4.0-microcounter-prompt strategy=baseline, sample size (n)=1002026.05 | 48 | |
| gpt-oss-120bcounter-prompt strategy=baseline, sample size (n)=1002026.05 | 48 | |
| 10 open-source avg.counter-prompt strategy=baseline, sample size (n)=1,0002026.05 | 46.9 | |
| gemma-4-E4B-itcounter-prompt strategy=reframe, sample size (n)=1002026.05 | 46 | |
| gemma-4-E4B-itcounter-prompt strategy=baseline, sample size (n)=1002026.05 | 45 | |
| gpt-4o (OpenAI)counter-prompt strategy=baseline, sample size (n)=1002026.05 | 45 | |
| gemma-3-27b (q4)counter-prompt strategy=baseline, sample size (n)=1002026.05 | 44 | |
| granite-4.0-microcounter-prompt strategy=reframe, sample size (n)=1002026.05 | 41 | |
| Magistral-Small-2509-FP8counter-prompt strategy=baseline, sample size (n)=1002026.05 | 34 | |
| Qwen3-VL-8B-Instructcounter-prompt strategy=reframe, sample size (n)=1002026.05 | 30 | |
| Qwen3-VL-8B-Instructcounter-prompt strategy=baseline, sample size (n)=1002026.05 | 29 | |
| gpt-3.5-turbo (OpenAI)counter-prompt strategy=reframe, sample size (n)=1002026.05 | 27 | |
| Qwen3.6-35B-A3B-FP8counter-prompt strategy=reframe, sample size (n)=1002026.05 | 23 | |
| gemma-3-27b (q4)counter-prompt strategy=reframe, sample size (n)=1002026.05 | 23 | |
| 10 open-source avg.counter-prompt strategy=reframe, sample size (n)=1,0002026.05 | 20.1 | |
| Phi-4-mini-instructcounter-prompt strategy=baseline, sample size (n)=1002026.05 | 17 | |
| 2 OpenAI avg.counter-prompt strategy=reframe, sample size (n)=2002026.05 | 16.5 | |
| granite-4.0-microcounter-prompt strategy=rule, sample size (n)=1002026.05 | 14 | |
| gpt-oss-120bcounter-prompt strategy=ignore, sample size (n)=1002026.05 | 13 | |
| gpt-4o (OpenAI)counter-prompt strategy=rule, sample size (n)=1002026.05 | 13 | |
| gpt-oss-120bcounter-prompt strategy=reframe, sample size (n)=1002026.05 | 11 | |
| Phi-4-mini-instructcounter-prompt strategy=reframe, sample size (n)=1002026.05 | 10 | |
| granite-4.0-microcounter-prompt strategy=ignore, sample size (n)=1002026.05 | 8 | |
| granite-4.0-microcounter-prompt strategy=compare, sample size (n)=1002026.05 | 8 | |
| Mistral-Small-3.2-24Bcounter-prompt strategy=reframe, sample size (n)=1002026.05 | 7 | |
| 2 OpenAI avg.counter-prompt strategy=rule, sample size (n)=2002026.05 | 6.5 | |
| Magistral-Small-2509-FP8counter-prompt strategy=reframe, sample size (n)=1002026.05 | 6 | |
| Phi-4-mini-instructcounter-prompt strategy=ignore, sample size (n)=1002026.05 | 6 | |
| gpt-4o (OpenAI)counter-prompt strategy=reframe, sample size (n)=1002026.05 | 6 | |
| Qwen3.5-9Bcounter-prompt strategy=reframe, sample size (n)=1002026.05 | 4 | |
| 10 open-source avg.counter-prompt strategy=ignore, sample size (n)=1,0002026.05 | 3.7 | |
| Magistral-Small-2509-FP8counter-prompt strategy=rule, sample size (n)=1002026.05 | 3 | |
| Phi-4-mini-instructcounter-prompt strategy=rule, sample size (n)=1002026.05 | 3 | |
| Qwen3-VL-8B-Instructcounter-prompt strategy=ignore, sample size (n)=1002026.05 | 3 | |
| Mistral-Small-3.2-24Bcounter-prompt strategy=rule, sample size (n)=1002026.05 | 3 | |
| gpt-4o (OpenAI)counter-prompt strategy=ignore, sample size (n)=1002026.05 | 3 | |
| 10 open-source avg.counter-prompt strategy=rule, sample size (n)=1,0002026.05 | 2.4 | |
| Qwen3.6-35B-A3B-FP8counter-prompt strategy=ignore, sample size (n)=1002026.05 | 2 | |
| gemma-4-E4B-itcounter-prompt strategy=ignore, sample size (n)=1002026.05 | 2 | |
| 2 OpenAI avg.counter-prompt strategy=ignore, sample size (n)=2002026.05 | 1.5 | |
| Magistral-Small-2509-FP8counter-prompt strategy=ignore, sample size (n)=1002026.05 | 1 | |
| Magistral-Small-2509-FP8counter-prompt strategy=compare, sample size (n)=1002026.05 | 1 | |
| Qwen3.5-9Bcounter-prompt strategy=rule, sample size (n)=1002026.05 | 1 | |
| Mistral-Small-3.2-24Bcounter-prompt strategy=ignore, sample size (n)=1002026.05 | 1 | |
| gemma-3-27b (q4)counter-prompt strategy=ignore, sample size (n)=1002026.05 | 1 | |
| gemma-4-E4B-itcounter-prompt strategy=compare, sample size (n)=1002026.05 | 1 | |
| 10 open-source avg.counter-prompt strategy=compare, sample size (n)=1,0002026.05 | 1 | |
| Phi-4-mini-instructcounter-prompt strategy=compare, sample size (n)=1002026.05 | 0 | |
| Qwen3.5-9Bcounter-prompt strategy=ignore, sample size (n)=1002026.05 | 0 | |
| Qwen3.5-9Bcounter-prompt strategy=compare, sample size (n)=1002026.05 | 0 | |
| Qwen3.6-35B-A3B-FP8counter-prompt strategy=rule, sample size (n)=1002026.05 | 0 | |
| Qwen3.6-35B-A3B-FP8counter-prompt strategy=compare, sample size (n)=1002026.05 | 0 | |
| Qwen3-VL-8B-Instructcounter-prompt strategy=rule, sample size (n)=1002026.05 | 0 | |
| Qwen3-VL-8B-Instructcounter-prompt strategy=compare, sample size (n)=1002026.05 | 0 | |
| Mistral-Small-3.2-24Bcounter-prompt strategy=compare, sample size (n)=1002026.05 | 0 | |
| gemma-3-27b (q4)counter-prompt strategy=rule, sample size (n)=1002026.05 | 0 | |
| gemma-3-27b (q4)counter-prompt strategy=compare, sample size (n)=1002026.05 | 0 | |
| gemma-4-E4B-itcounter-prompt strategy=rule, sample size (n)=1002026.05 | 0 | |
| gpt-oss-120bcounter-prompt strategy=rule, sample size (n)=1002026.05 | 0 | |
| gpt-oss-120bcounter-prompt strategy=compare, sample size (n)=1002026.05 | 0 | |
| gpt-3.5-turbo (OpenAI)counter-prompt strategy=ignore, sample size (n)=1002026.05 | 0 | |
| gpt-3.5-turbo (OpenAI)counter-prompt strategy=rule, sample size (n)=1002026.05 | 0 | |
| gpt-3.5-turbo (OpenAI)counter-prompt strategy=compare, sample size (n)=1002026.05 | 0 | |
| gpt-4o (OpenAI)counter-prompt strategy=compare, sample size (n)=1002026.05 | 0 | |
| 2 OpenAI avg.counter-prompt strategy=compare, sample size (n)=2002026.05 | 0 |