OpenFOAM Case Authoring on n30 hybrid 30-task study derived from FoamGPT
87Mean ScoreSIGA R+S
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SIGA R+SModel=deepseek-v4-flash, Run count=single run2026.06 | 87 | 1 | 0 | 263.5 | 2.61 | |
| SIGA S+XModel=deepseek-v4-flash, Run count=single run2026.06 | 86.6 | 1 | 0 | 469.8 | 3.68 | |
| SIGA R+S+X+MModel=deepseek-v4-flash, Run count=single run2026.06 | 85.4 | 1 | 0 | 376.9 | 3.81 | |
| SIGA S+X+MModel=deepseek-v4-flash, Run count=single run2026.06 | 84.7 | 1 | 0 | 495.4 | 3.65 | |
| SIGA S+MModel=deepseek-v4-flash, Run count=single run2026.06 | 83.5 | 1 | 0 | 429.5 | 3.29 | |
| SIGA X+MModel=deepseek-v4-flash, Run count=single run2026.06 | 69.7 | 1 | 0 | 236 | 2.04 | |
| SIGA R+MModel=deepseek-v4-flash, Run count=single run2026.06 | 68.9 | 1 | 0 | 220.8 | 1.15 | |
| SIGA R+XModel=deepseek-v4-flash, Run count=single run2026.06 | 68.5 | 1 | 0 | 179.2 | 1.62 | |
| SIGA VanillaModel=deepseek-v4-flash, Run count=single run2026.06 | 68.1 | 1 | 0 | 196.3 | 1.36 | |
| Foam-Agent 2.0 (lint)Model=deepseek-v4-flash, Run count=single run, Execution mode=lint-only2026.06 | 51.6 | 0.6333 | 0.2667 | 373 | 0.3 | |
| MetaOpenFOAM (lint)Model=deepseek-v4-flash, Run count=single run, Execution mode=lint-only2026.06 | 37.9 | 0.3333 | 0.4 | 431.5 | 0.17 |