DSL Generation on RED-Trading 095
83.87TBREDPARROT-RAW
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| REDPARROT-RAWEncoding=Raw user queries (Qwen3-embedding-0.6B), LLM=Qwen2.5-72b2026.03 | 83.87 | 90.32 | 61.29 | 90.32 | 38.71 | 7.82 | |
| REDPARROT-MANEncoding=Manually labeled skeletons (Qwen3-embedding-0.6B), LLM=Qwen2.5-72b2026.03 | 83.87 | 77.42 | 54.84 | 93.55 | 38.71 | 10.6 | |
| REDPARROTEncoding=Entity-agnostic embedding model (Qwen3-embedding-0.6B backbone), LLM=Qwen2.5-72b2026.03 | 83.87 | 77.42 | 58.06 | 83.87 | 38.71 | 7.82 | |
| LONGCHAIN-DSReasoning Model=DeepSeek-V32026.03 | 67.74 | 67.74 | 22.58 | 96.77 | 22.58 | 24 | |
| LONGCHAIN-QWENReasoning Model=Qwen2.5-72b, Configuration Model=Qwen2.5-72b2026.03 | 29.03 | 51.62 | 25.81 | 87.1 | 16.13 | 24 |