Reasoning on Software Domain 20 tasks (test)
0.06Total Cost (USD)Force Weak
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Force Weak2026.01 | 0.06 | 0.003 | 92.4 | |
| CASTER2026.01 | 0.36 | 0.018 | 54.4 | |
| Force Strong2026.01 | 0.79 | 0.04 | — |
Loading the SOTA2 catalog…
| Method | Links | |||
|---|---|---|---|---|
| Force Weak2026.01 | 0.06 | 0.003 | 92.4 | |
| CASTER2026.01 | 0.36 | 0.018 | 54.4 | |
| Force Strong2026.01 | 0.79 | 0.04 | — |