Single Ad Selection on Single Ad Insertion Simulated Scenarios (test)
100Complex Intent Analysis AccuracyLERA (LLM-only)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LERA (LLM-only)Embed Model=N/A, Gen Model=Qwen3-235B-A22B2026.05 | 100 | 100 | 100 | 100 | |
| LERA (LLM-only)Embed Model=N/A, Gen Model=Qwen3-30B-A3B2026.05 | 97.5 | 86.2 | 100 | 94.57 | |
| LERAEmbed Model=all-MiniLM-L6-v2, Gen Model=Qwen3-30B-A3B2026.05 | 96.2 | 100 | 100 | 98.73 | |
| LERAEmbed Model=all-MiniLM-L6-v2, Gen Model=Qwen3-235B-A22B2026.05 | 96.2 | 98.8 | 97.5 | 97.5 | |
| LERAEmbed Model=all-MiniLM-L6-v2, Gen Model=Qwen3-4B2026.05 | 91.2 | 97.5 | 95 | 94.6 | |
| LERA (Keyword-Match)Embed Model=all-MiniLM-L6-v2, Gen Model=Qwen3-30B-A3B2026.05 | 81.2 | 90 | 88.8 | 86.67 | |
| LERA (LLM-only)Embed Model=N/A, Gen Model=Qwen3-4B2026.05 | 76.2 | 71.2 | 72.5 | 73.3 | |
| LERA (Keyword-Match)Embed Model=all-MiniLM-L6-v2, Gen Model=Qwen3-235B-A22B2026.05 | 75 | 81.2 | 86.2 | 80.8 | |
| Embedding-onlyEmbed Model=all-MiniLM-L6-v2, Gen Model=N/A2026.05 | 67.5 | 68.8 | 52.5 | 62.93 | |
| LERA (Keyword-Match)Embed Model=all-MiniLM-L6-v2, Gen Model=Qwen3-4B2026.05 | 67.5 | 73.8 | 82.5 | 74.6 | |
| Embedding-onlyEmbed Model=Qwen3-Embedding-8B, Gen Model=N/A2026.05 | 55 | 60 | 53.8 | 56.3 |