Ad-Injected Response Generation on MT-Human
88.8AccuracyGI-R
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GI-RJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 88.8 | 49.8 | 82.8 | 75.6 | 32.4 | 62.4 | 65.3 | |
| GIR-RJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 88.2 | 53.4 | 78.6 | 74.4 | 43.2 | 66.6 | 67.4 | |
| GIR-PJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 87.6 | 54 | 76.8 | 71.4 | 29.4 | 65.4 | 64.1 | |
| GIR-RBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Embedding model=text-embedding-3-small, Retrieval strategy=response-based, Final rewriting=included2025.09 | 85.8 | 52.8 | 74.4 | 73.2 | 82.8 | 79.2 | 74.7 | |
| GI-RJudgement LLM=kimi-k22025.09 | 84.6 | 36.6 | 71.4 | 46.2 | 25.2 | 52.8 | 52.8 | |
| GI-RBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Embedding model=text-embedding-3-small, Retrieval strategy=response-based, Final rewriting=skipped2025.09 | 83.4 | 39.6 | 79.8 | 68.4 | 74.4 | 72.6 | 69.7 | |
| GIR-RJudgement LLM=kimi-k22025.09 | 83.4 | 46.2 | 72 | 61.2 | 46.2 | 57 | 61 | |
| GIR-PBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Embedding model=text-embedding-3-small, Retrieval strategy=prompt-based, Final rewriting=included2025.09 | 82.8 | 53.4 | 74.4 | 70.2 | 81.6 | 81 | 73.9 | |
| GI-RJudgement LLM=qwen-max2025.09 | 82.2 | 41.4 | 75 | 54.6 | 49.2 | 43.8 | 57.7 | |
| GIR-PJudgement LLM=kimi-k22025.09 | 81.6 | 46.8 | 70.8 | 54.6 | 36 | 52.8 | 57.1 | |
| GIR-RJudgement LLM=qwen-max2025.09 | 81 | 53.4 | 64.2 | 52.8 | 63.6 | 55.2 | 61.7 | |
| GIR-PJudgement LLM=qwen-max2025.09 | 76.2 | 52.2 | 64.2 | 56.4 | 63 | 53.4 | 60.9 | |
| Ad-ChatJudgement LLM=claude-3-5-haiku, Base LLM=doubao-1-5-lite2025.09 | 76.2 | 43.2 | 57.6 | 52.8 | 31.8 | 53.4 | 52.5 | |
| Ad-ChatBase LLM=doubao-1.5-lite-32k, Evaluator LLM=gpt-4.1-mini, Configuration=Original implementation2025.09 | 72.6 | 51 | 68.4 | 61.8 | 72.6 | 64.2 | 65.1 | |
| Ad-ChatJudgement LLM=qwen-max2025.09 | 71.4 | 51.6 | 62.4 | 53.4 | 55.8 | 46.8 | 56.9 | |
| Ad-ChatJudgement LLM=kimi-k22025.09 | 65.4 | 43.2 | 56.4 | 49.8 | 25.8 | 40.8 | 46.9 |