DSL Generation on RED-Community 0916
90.16TBREDPARROT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| REDPARROTEncoding=Entity-agnostic embedding model (Qwen3-embedding-0.6B backbone), LLM=Qwen2.5-72b2026.03 | 90.16 | 81.97 | 77.05 | 86.89 | 59.02 | 7.74 | |
| REDPARROT-MANEncoding=Manually labeled skeletons (Qwen3-embedding-0.6B), LLM=Qwen2.5-72b2026.03 | 88.52 | 81.97 | 73.77 | 80.33 | 54.1 | 11.12 | |
| REDPARROT-RAWEncoding=Raw user queries (Qwen3-embedding-0.6B), LLM=Qwen2.5-72b2026.03 | 86.89 | 77.05 | 72.13 | 86.89 | 52.46 | 6.98 | |
| LONGCHAIN-DSReasoning Model=DeepSeek-V32026.03 | 68.85 | 68.85 | 60.65 | 95.08 | 60.67 | 34 | |
| LONGCHAIN-QWENReasoning Model=Qwen2.5-72b, Configuration Model=Qwen2.5-72b2026.03 | 49.18 | 63.93 | 59.02 | 95.08 | 47.54 | 25 |