# Learning to Trust: Dynamic Utilization of Retrieval-Augmented Generation for E-commerce Search Relevance

Source: [Learning to Trust: Dynamic Utilization of Retrieval-Augmented Generation for E-commerce Search Relevance](<https://www.sota2.com/research/paper/68fa7c201e209131cea8c8ce>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-17T05:00:21.724Z

Authors: Tingqiao Xu, Shaowei Yao, Chenhe Dong, Yiming Jin, Zerui Huang, Dan Ou, Haihong Tang, Bo Zheng

Published: 2025-10-13T08:08:59.000Z

## Abstract

Accurately estimating query-item relevance is vital for e-commerce ranking and conversion. While Large Language Models (LLMs) excel at reasoning, they often lack specialized knowledge required for long-tail or fast-evolving queries, necessitating Retrieval-Augmented Generation (RAG). However, production environments face three critical challenges: (1) external context is inherently noisy and inconsistent; (2) extreme latency budgets prohibit multi-stage processing or refinement; and (3) the model must simultaneously assess relevance and context-trust within a unified inference pass. We propose DyKnow-RAG, a reinforcement learning framework that teaches LLMs to learn to trust through dynamic utilization of external knowledge. Built on Group Relative Policy Optimization (GRPO), DyKnow-RAG utilizes a dual-group rollout strategy (parametric-only vs. with-context) and a posterior-driven inter-group advantage scaling mechanism. This enables the model to optimize context utilization without human process labels or extra inference overhead. Our pipeline further integrates structured Chain-of-Thought (CoT) and an uncertainty-prioritized RL pool to stabilize training.Offline evaluations show significant Macro-F1 and Accuracy gains, particularly on noise-sensitive query slices. Importantly, DyKnow-RAG has been deployed in Taobao's production system, serving hundreds of millions of active users and billions of daily search requests. Controlled A/B tests demonstrate consistent lifts in key business metrics, including GSB and Item Goodrate, while maintaining a p99 latency under 400ms. This work provides a scalable and deployable paradigm for operationalizing noisy RAG under extreme efficiency constraints of large-scale industrial search.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [E-commerce Search Relevance Classification on Gold Benchmark Offline (offline)](<https://www.sota2.com/research/sota/e-commerce-search-relevance-classification-on-gold-benchmark-offline-offline.md>): E-commerce search relevance classification; Gold Benchmark Offline; C1 F1 Score: 49.48.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2510.11122>)

[PDF](<https://arxiv.org/pdf/2510.11122.pdf>)
