Single-turn data-to-logic tasks on DTLBench
90.29DDXPlus ScoreCASCADE
Evaluation Results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CASCADEBackbone=gemini-2.0-flash2026.05 | 90.29 | — | — | — | 67.13 | 61.99 | 83.33 | 40.79 | 75.96 | 86.29 | 83.05 | 64.42 | 72.58 | |
| NP-CBRBackbone=gemini-2.0-flash2026.05 | 86.91 | — | — | — | 66.33 | 62.05 | 81.94 | 37.64 | 69.02 | 85.11 | 82.85 | 64.3 | 70.68 | |
| CASCADEBackbone=Qwen3-32B2026.05 | 84.54 | 63.98 | 56.79 | 52.25 | 62.44 | 60.26 | 84.66 | 47.72 | 70.76 | 86.15 | 75.41 | 55.18 | 66.68 | |
| NP-CBRBackbone=Qwen3-32B2026.05 | 80.84 | 60.58 | 56.29 | 51.83 | 60.93 | 59.56 | 82.77 | 45.35 | 65.08 | 85.39 | 66.87 | 49.6 | 63.76 | |
| REINFORCE+LoRABackbone=Qwen3-32B2026.05 | 76.9 | 13.88 | 57.47 | 53.97 | 60.32 | 52.63 | 73.87 | 47.26 | 58.2 | 86.8 | 73.54 | 52.35 | 58.93 | |
| CASCADEBackbone=Qwen3-14B2026.05 | 75.66 | 62.35 | 57.76 | 54.55 | 56.95 | 59.06 | 82.16 | 45.1 | 73.68 | 85.61 | 74.66 | 53.68 | 65.1 | |
| ICRLBackbone=gemini-2.0-flash2026.05 | 73.79 | — | — | — | 64.98 | 55.77 | 74.1 | 30.07 | 28.5 | 84.7 | 80.62 | 62.32 | 61.65 | |
| ICRLPlusBackbone=gemini-2.0-flash2026.05 | 73.52 | — | — | — | 66 | 56.55 | 73.92 | 31.82 | 30.09 | 84.6 | 80.61 | 61.99 | 62.12 | |
| REINFORCE+LoRABackbone=Qwen3-14B2026.05 | 71.3 | 11.07 | 58.21 | 52.39 | 53.72 | 41.38 | 72.3 | 44.42 | 54.11 | 86.47 | 72.07 | 49.17 | 55.55 | |
| NP-CBRBackbone=Qwen3-14B2026.05 | 70.69 | 61.28 | 56.77 | 53.15 | 56.33 | 56.11 | 81.02 | 42.98 | 66.41 | 85.31 | 71.57 | 52.14 | 62.81 | |
| CASCADEBackbone=Qwen3-4B2026.05 | 70.04 | — | 57.43 | 36.33 | 46.87 | 49.12 | 80.1 | 37.96 | 57.45 | 86.45 | 75.9 | 54.59 | 59.29 | |
| Zero-shotBackbone=gemini-2.0-flash2026.05 | 69.67 | — | — | — | 59.61 | 39.32 | 73.61 | 12.92 | 28.06 | 83.16 | 80.66 | 62.19 | 56.58 | |
| CASCADEBackbone=Qwen3-8B2026.05 | 69.13 | 54.88 | 63.97 | 51.2 | 47.9 | 47.75 | 80.78 | 39.77 | 66.91 | 86.4 | 72.7 | 48.66 | 60.84 | |
| ICRLBackbone=Qwen3-32B2026.05 | 66.11 | 12.95 | 50.82 | 50.23 | 57.03 | 44.06 | 67.59 | 33.77 | 41.83 | 83.55 | 70.95 | 50.86 | 52.48 | |
| REINFORCE+LoRABackbone=Qwen3-8B2026.05 | 65.55 | 4.94 | 64.16 | 52.04 | 48.85 | 51.67 | 68.84 | 43.19 | 54.17 | 86.37 | 69.08 | 44.77 | 54.47 | |
| NP-CBRBackbone=Qwen3-8B2026.05 | 65.44 | 52.85 | 62.73 | 49.91 | 45.45 | 46.86 | 78.92 | 38.04 | 60.93 | 85.3 | 66.74 | 48.11 | 58.44 | |
| ICRLPlusBackbone=Qwen3-32B2026.05 | 64.91 | 30.29 | 52.24 | 51.05 | 58.33 | 43.22 | 67.61 | 35.39 | 43.34 | 83.91 | 71.11 | 50.52 | 54.33 | |
| Zero-shotBackbone=Qwen3-32B2026.05 | 63.77 | 12.79 | 51.36 | 48.56 | 25.53 | 39.12 | 66.66 | 34.3 | 45.36 | 84.05 | 63.84 | 44.65 | 48.33 | |
| NP-CBRBackbone=Qwen3-4B2026.05 | 62.69 | — | 56.56 | 34.93 | 44.84 | 45.92 | 77.36 | 35.19 | 53.7 | 85.42 | 73.43 | 52.93 | 56.63 | |
| REINFORCE+LoRABackbone=Qwen3-4B2026.05 | 61.39 | — | 59.65 | 44.7 | 47.69 | 45.89 | 65.89 | 35.66 | 49.31 | 85.57 | 73.42 | 53.12 | 56.57 | |
| ICRLBackbone=Qwen3-14B2026.05 | 56.5 | 1.18 | 54.65 | 48.47 | 49.39 | 40.52 | 66.99 | 32.44 | 32.03 | 82.12 | 69.29 | 48.59 | 48.51 | |
| ICRLPlusBackbone=Qwen3-14B2026.05 | 56.16 | 35.39 | 55.39 | 49.7 | 50.09 | 41.91 | 67.14 | 34.71 | 38.47 | 82.52 | 69.39 | 48.33 | 52.43 | |
| Zero-shotBackbone=Qwen3-14B2026.05 | 55.75 | 11.91 | 52.92 | 40.92 | 50.84 | 38.65 | 65.86 | 33.03 | 36.66 | 78.41 | 66.29 | 48.75 | 48.33 | |
| ICRLBackbone=Qwen3-8B2026.05 | 49.5 | 3.19 | 53.4 | 46.76 | 40.68 | 29.4 | 60.43 | 26.12 | 21.11 | 81.53 | 67.66 | 43.25 | 43.59 | |
| ICRLPlusBackbone=Qwen3-8B2026.05 | 48.57 | 23.36 | 55.23 | 48.75 | 42.48 | 31.69 | 60.45 | 27.97 | 28.55 | 80.98 | 68.19 | 44.86 | 46.76 | |
| Zero-shotBackbone=Qwen3-8B2026.05 | 45.2 | 5.94 | 50.49 | 35.6 | 39.76 | 36.22 | 61.37 | 26.49 | 38.08 | 74.61 | 60.96 | 43.96 | 43.22 | |
| ICRLBackbone=Qwen3-4B2026.05 | 44.52 | — | 51.01 | 28.43 | 35.9 | 21.6 | 62.92 | 27.15 | 21.72 | 83.31 | 71.69 | 51.71 | 45.45 | |
| ICRLPlusBackbone=Qwen3-4B2026.05 | 44.11 | — | 50.1 | 31.69 | 38.89 | 27.88 | 63.04 | 27.99 | 26.05 | 83.37 | 71.67 | 52.86 | 47.06 | |
| Zero-shotBackbone=Qwen3-4B2026.05 | 36.58 | — | 50.53 | 26.85 | 34.72 | 19.24 | 62.33 | 24.72 | 23.52 | 81.92 | 70.15 | 52.07 | 43.88 |