Ad-Injected Response Generation on LM-Market
85.14AccuracyGIR-R
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GIR-RJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 85.14 | 48.84 | 75.56 | 67.29 | 48.35 | 70.84 | 66 | |
| GIR-PJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 84.5 | 47.99 | 74.62 | 67.11 | 43.44 | 69.23 | 64.48 | |
| GI-RJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 83.3 | 40.38 | 78.96 | 65.68 | 43.12 | 67.85 | 63.21 | |
| GIR-RBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Embedding model=text-embedding-3-small, Retrieval strategy=response-based, Final rewriting=included2025.09 | 80.32 | 62.49 | 71.88 | 70.47 | 80.14 | 77.1 | 73.73 | |
| GI-RBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Embedding model=text-embedding-3-small, Retrieval strategy=response-based, Final rewriting=skipped2025.09 | 79.55 | 47.46 | 76.49 | 69.3 | 75.11 | 74.84 | 70.46 | |
| GIR-PBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Embedding model=text-embedding-3-small, Retrieval strategy=prompt-based, Final rewriting=included2025.09 | 78.96 | 60.72 | 71.63 | 69.48 | 79.66 | 75.52 | 72.66 | |
| GIR-RJudgement LLM=qwen-max2025.09 | 75.86 | 53.71 | 65.13 | 52.67 | 60.63 | 60.11 | 61.35 | |
| GIR-PJudgement LLM=qwen-max2025.09 | 74.87 | 51.97 | 64.16 | 52.03 | 56.06 | 57.69 | 59.46 | |
| GI-RJudgement LLM=qwen-max2025.09 | 74.21 | 42.69 | 72.78 | 50.66 | 52.4 | 49.7 | 57.07 | |
| GIR-RJudgement LLM=kimi-k22025.09 | 71.51 | 41.36 | 68.89 | 48.92 | 42.51 | 56.83 | 55 | |
| GIR-PJudgement LLM=kimi-k22025.09 | 71.33 | 40.81 | 67.22 | 47.4 | 36.95 | 55.95 | 53.28 | |
| GI-RJudgement LLM=kimi-k22025.09 | 69.32 | 30.52 | 72.06 | 39.95 | 32.15 | 55.47 | 49.91 | |
| Ad-ChatBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Configuration=Original implementation2025.09 | 61.97 | 52.54 | 57.38 | 55.47 | 77.46 | 78.01 | 63.81 | |
| Ad-ChatJudgement LLM=qwen-max2025.09 | 61.58 | 46.88 | 53.28 | 44.18 | 63.37 | 66.68 | 55.99 | |
| Ad-ChatJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 61.4 | 31.56 | 44.19 | 36.09 | 39.64 | 73.1 | 47.66 | |
| Ad-ChatJudgement LLM=kimi-k22025.09 | 45.77 | 34.66 | 47.65 | 30.52 | 36.86 | 59.37 | 42.47 |