DSL Generation on RED-Community-095
90.91TBREDPARROT-RAW
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| REDPARROT-RAWEncoding=Raw user queries (Qwen3-embedding-0.6B), LLM=Qwen2.5-72b2026.03 | 90.91 | 90.91 | 66.67 | 83.33 | 53.03 | 7.8 | |
| REDPARROTEncoding=Entity-agnostic embedding model (Qwen3-embedding-0.6B backbone), LLM=Qwen2.5-72b2026.03 | 90.91 | 98.48 | 78.79 | 95.45 | 66.67 | 7.8 | |
| REDPARROT-MANEncoding=Manually labeled skeletons (Qwen3-embedding-0.6B), LLM=Qwen2.5-72b2026.03 | 89.39 | 98.48 | 78.79 | 95.45 | 65.15 | 10.66 | |
| LONGCHAIN-DSReasoning Model=DeepSeek-V32026.03 | 68.18 | 68.18 | 59.09 | 89.39 | 59.09 | 26 | |
| LONGCHAIN-QWENReasoning Model=Qwen2.5-72b, Configuration Model=Qwen2.5-72b2026.03 | 51.52 | 68.18 | 57.57 | 92.42 | 43.94 | 30 |