DSL Generation on RED-Trading 0916
79.17TB ScoreREDPARROT-RAW
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| REDPARROT-RAWEncoding=Raw user queries (Qwen3-embedding-0.6B), LLM=Qwen2.5-72b2026.03 | 79.17 | 87.5 | 20.83 | 91.67 | 20.83 | 8.85 | |
| REDPARROT-MANEncoding=Manually labeled skeletons (Qwen3-embedding-0.6B), LLM=Qwen2.5-72b2026.03 | 79.17 | 87.5 | 20.83 | 91.67 | 20.83 | 12.01 | |
| REDPARROTEncoding=Entity-agnostic embedding model (Qwen3-embedding-0.6B backbone), LLM=Qwen2.5-72b2026.03 | 75 | 87.5 | 33.33 | 91.67 | 33.33 | 9.19 | |
| LONGCHAIN-DSReasoning Model=DeepSeek-V32026.03 | 41.67 | 41.67 | 20.83 | 83.33 | 20.83 | 29 | |
| LONGCHAIN-QWENReasoning Model=Qwen2.5-72b, Configuration Model=Qwen2.5-72b2026.03 | 12.5 | 37.5 | 20.84 | 87.5 | 8.33 | 24 |