# E-commerce Shopping Assistance on SmartShopBench 1.0 (test 120 queries) Source: [E-commerce Shopping Assistance on SmartShopBench 1.0 (test 120 queries)]() Collection index: [llms.txt]() Data updated: 2026-03-09T08:03:07.545Z Evaluation of E-commerce Shopping Assistance performance using the test split (120 queries) of the SmartShopBench 1.0 dataset, measured by metrics including PC, TR, DF, and L1/L2 Avg@4. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | E-commerce Shopping Assistance | | Dataset (including any stated split) | SmartShopBench 1.0 (test 120 queries) | | Metric | Precision (PC) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Precision (PC)](): Higher is better; unit: PERCENT. - [Task Success Rate (TR)](): Higher is better; unit: PERCENT. - [Diversity Factor (DF)](): Higher is better; unit: PERCENT. - [L1 Average Error @4](): Lower is better; unit: PERCENT. - [Pass Rate @4](): Higher is better; unit: PERCENT. - [L2 Average Error @4](): Lower is better; unit: SCALAR. - [L2 Std Dev @4](): Lower is better; unit: SCALAR. ## Recorded results | Method | Precision (PC) | Result date | Report paper | | --- | --- | --- | --- | | ChatShopBuddy-SFT-RL | 93.35 | Not recorded | 69ae53df2d603e6de3e4289b | | GPT-5.2 | 88.15 | Not recorded | 69ae53df2d603e6de3e4289b | | ChatShopBuddy-SFT | 86.88 | Not recorded | 69ae53df2d603e6de3e4289b | | DeepSeek-V3.2-reasoner | 86.05 | Not recorded | 69ae53df2d603e6de3e4289b | | Gemini-3-Pro-Preview | 80.4 | Not recorded | 69ae53df2d603e6de3e4289b | | GPT-5.2(think off) | 80.03 | Not recorded | 69ae53df2d603e6de3e4289b | | Doubao-seed-1.8 | 76.47 | Not recorded | 69ae53df2d603e6de3e4289b | | Gemini-3-Flash-Preview(minimal) | 74.8 | Not recorded | 69ae53df2d603e6de3e4289b | | Doubao-Seed-1.8(think off) | 73.53 | Not recorded | 69ae53df2d603e6de3e4289b | | Kimi-K2.5 | 70.42 | Not recorded | 69ae53df2d603e6de3e4289b | | DeepSeek-V3.2-chat | 69.8 | Not recorded | 69ae53df2d603e6de3e4289b | | GLM-4.7 | 64.8 | Not recorded | 69ae53df2d603e6de3e4289b | | GLM-4.7(think off) | 61.67 | Not recorded | 69ae53df2d603e6de3e4289b | | Qwen3-30B-A3B-Thinking-2507 | 47.3 | Not recorded | 69ae53df2d603e6de3e4289b | | Qwen3-235B-A22B-Instruct-2507 | 42.3 | Not recorded | 69ae53df2d603e6de3e4289b | | Qwen3-235B-A22B-Thinking-2507 | 36.88 | Not recorded | 69ae53df2d603e6de3e4289b | | Kimi-K2.5(think off) | 31.45 | Not recorded | 69ae53df2d603e6de3e4289b | | Qwen3-30B-A3B-Instruct-2507 | 14.57 | Not recorded | 69ae53df2d603e6de3e4289b | ### ChatShopBuddy-SFT-RL Precision (PC): 93.35. Result ID: 69ae7ea76dbb7075b95674d9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Thinking Model Category":"Thinking Model","# Tokens":"633"}. This record does not assert independent reproduction. ### GPT-5.2 Precision (PC): 88.15. Result ID: 69ae7ea76dbb7075b95674d5. [Reporting paper]() Recorded evaluation configuration: {"Thinking Model Category":"Thinking Model","# Tokens":"N/A"}. This record does not assert independent reproduction. ### ChatShopBuddy-SFT Precision (PC): 86.88. Result ID: 69ae7ea76dbb7075b95674d8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Thinking Model Category":"Thinking Model","# Tokens":"1,200"}. This record does not assert independent reproduction. ### DeepSeek-V3.2-reasoner Precision (PC): 86.05. Result ID: 69ae7ea76dbb7075b95674d0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Thinking Model Category":"Thinking Model","# Tokens":"1,412"}. This record does not assert independent reproduction. ### Gemini-3-Pro-Preview Precision (PC): 80.4. Result ID: 69ae7ea76dbb7075b95674d6. [Reporting paper]() Recorded evaluation configuration: {"Thinking Model Category":"Thinking Model","# Tokens":"N/A"}. This record does not assert independent reproduction. ### GPT-5.2(think off) Precision (PC): 80.03. Result ID: 69ae7ea76dbb7075b95674cd. [Reporting paper]() Recorded evaluation configuration: {"Thinking Model Category":"Non-Thinking Model","# Tokens":"-"}. This record does not assert independent reproduction. ### Doubao-seed-1.8 Precision (PC): 76.47. Result ID: 69ae7ea76dbb7075b95674d7. [Reporting paper]() Recorded evaluation configuration: {"Thinking Model Category":"Thinking Model","# Tokens":"1,056"}. This record does not assert independent reproduction. ### Gemini-3-Flash-Preview(minimal) Precision (PC): 74.8. Result ID: 69ae7ea76dbb7075b95674ce. [Reporting paper]() Recorded evaluation configuration: {"Thinking Model Category":"Non-Thinking Model","# Tokens":"-"}. This record does not assert independent reproduction. ### Doubao-Seed-1.8(think off) Precision (PC): 73.53. Result ID: 69ae7ea76dbb7075b95674cf. [Reporting paper]() Recorded evaluation configuration: {"Thinking Model Category":"Non-Thinking Model","# Tokens":"-"}. This record does not assert independent reproduction. ### Kimi-K2.5 Precision (PC): 70.42. Result ID: 69ae7ea76dbb7075b95674d2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Thinking Model Category":"Thinking Model","# Tokens":"515"}. This record does not assert independent reproduction. ### DeepSeek-V3.2-chat Precision (PC): 69.8. Result ID: 69ae7ea76dbb7075b95674c8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Thinking Model Category":"Non-Thinking Model","# Tokens":"-"}. This record does not assert independent reproduction. ### GLM-4.7 Precision (PC): 64.8. Result ID: 69ae7ea76dbb7075b95674d1. [Reporting paper]() Recorded evaluation configuration: {"Thinking Model Category":"Thinking Model","# Tokens":"764"}. This record does not assert independent reproduction. ### GLM-4.7(think off) Precision (PC): 61.67. Result ID: 69ae7ea76dbb7075b95674c9. [Reporting paper]() Recorded evaluation configuration: {"Thinking Model Category":"Non-Thinking Model","# Tokens":"-"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Thinking-2507 Precision (PC): 47.3. Result ID: 69ae7ea76dbb7075b95674d4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Thinking Model Category":"Thinking Model","# Tokens":"2,803"}. This record does not assert independent reproduction. ### Qwen3-235B-A22B-Instruct-2507 Precision (PC): 42.3. Result ID: 69ae7ea76dbb7075b95674cb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Thinking Model Category":"Non-Thinking Model","# Tokens":"-"}. This record does not assert independent reproduction. ### Qwen3-235B-A22B-Thinking-2507 Precision (PC): 36.88. Result ID: 69ae7ea76dbb7075b95674d3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Thinking Model Category":"Thinking Model","# Tokens":"1,526"}. This record does not assert independent reproduction. ### Kimi-K2.5(think off) Precision (PC): 31.45. Result ID: 69ae7ea76dbb7075b95674ca. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Thinking Model Category":"Non-Thinking Model","# Tokens":"-"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B-Instruct-2507 Precision (PC): 14.57. Result ID: 69ae7ea76dbb7075b95674cc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Thinking Model Category":"Non-Thinking Model","# Tokens":"-"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.