Ad-Injected Response Generation on CA-Prod
88.25Click-Through Rate (CTR)Ad-Chat
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ad-ChatBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Configuration=Original implementation2025.09 | 88.25 | 42.6 | 36.38 | 25.03 | 22.66 | 88.12 | 50.51 | — | — | — | — | — | |
| GIR-PBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Embedding model=text-embedding-3-small, Retrieval strategy=prompt-based, Final rewriting=included2025.09 | 87.24 | 59.19 | 35.4 | 47.39 | 36.88 | 84.34 | 58.41 | — | — | — | — | — | |
| GIR-RBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Embedding model=text-embedding-3-small, Retrieval strategy=response-based, Final rewriting=included2025.09 | 87.16 | 59.56 | 34.45 | 46.97 | 36.92 | 84.97 | 58.34 | — | — | — | — | — | |
| GI-RBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Embedding model=text-embedding-3-small, Retrieval strategy=response-based, Final rewriting=skipped2025.09 | 85.08 | 53.76 | 26.55 | 44.85 | 36.69 | 80.09 | 54.5 | — | — | — | — | — | |
| GI-RJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 84.04 | 71.05 | 10.31 | 38.39 | 30.08 | 28.79 | 43.78 | — | — | — | — | — | |
| GIR-RJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 83.61 | 69.84 | 19.72 | 39.66 | 30.11 | 27.24 | 45.03 | — | — | — | — | — | |
| GIR-PJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 83.53 | 69.97 | 19.81 | 40.28 | 30.58 | 27.84 | 45.34 | — | — | — | — | — | |
| Ad-ChatJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 80.08 | 42.62 | 7.87 | 10.31 | 5 | 24.03 | 28.32 | — | — | — | — | — | |
| Ad-ChatJudgement LLM=qwen-max2025.09 | 70.93 | 46.8 | 32.2 | 27.71 | 28.22 | 76.05 | 46.99 | — | — | — | — | — | |
| GIR-RJudgement LLM=qwen-max2025.09 | 69.15 | 58.9 | 33.16 | 45.47 | 36.92 | 59.79 | 50.57 | — | — | — | — | — | |
| GIR-PJudgement LLM=qwen-max2025.09 | 69.07 | 59.8 | 33.5 | 46.32 | 37.59 | 58.13 | 50.73 | — | — | — | — | — | |
| GIR-RJudgement LLM=kimi-k22025.09 | 64.04 | 39.01 | 21.25 | 34.93 | 13.28 | 11.98 | 30.75 | — | — | — | — | — | |
| GIR-PJudgement LLM=kimi-k22025.09 | 63.63 | 41.17 | 22.52 | 35.28 | 13.1 | 11.5 | 31.2 | — | — | — | — | — | |
| GI-RJudgement LLM=kimi-k22025.09 | 61.89 | 32.43 | 9.62 | 26.8 | 4.2 | 3.78 | 23.12 | — | — | — | — | — | |
| Ad-ChatJudgement LLM=kimi-k22025.09 | 61.73 | 29.48 | 13.44 | 14.96 | 0.9 | 14.58 | 22.51 | — | — | — | — | — | |
| GI-RJudgement LLM=qwen-max2025.09 | 61.4 | 53.21 | 27.99 | 42.46 | 35.47 | 48.08 | 44.77 | — | — | — | — | — | |
| Ad-ChatConfiguration=Default original implementation2025.09 | 42.02 | — | — | — | — | — | 62.13 | 85.92 | 43.52 | 35.99 | 65.34 | 100 | |
| GIR-PBase LLM=doubao-1.5-lite-32k, Embedding=text-embedding-3-small, Retrieval Basis=User Prompt, Rewriting Stage=Included2025.09 | 39.45 | — | — | — | — | — | 64.89 | 77.87 | 61.99 | 42.92 | 67.12 | 100 | |
| GI-RBase LLM=doubao-1.5-lite-32k, Embedding=text-embedding-3-small, Retrieval Basis=Original Response, Rewriting Stage=Excluded2025.09 | 35.23 | — | — | — | — | — | 66.07 | 86.12 | 63.49 | 42.1 | 69.46 | 100 | |
| GIR-RBase LLM=doubao-1.5-lite-32k, Embedding=text-embedding-3-small, Retrieval Basis=Original Response, Rewriting Stage=Included2025.09 | 35.23 | — | — | — | — | — | 64.8 | 81.07 | 62.58 | 43.09 | 66.83 | 100 |