Photonic Inverse Design on PCF design tasks
32.85F1 ScoreSkillPCF
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| SkillPCFExecutor Backbone=MiniMax-M2.52026.05 | 32.85 | — | 6.92 | 24.35 | 62.24 | 84.85 | 82.35 | 52.18 | 68.45 | 1.02 | |
| SkillPCFExecutor Backbone=Qwen2.5-72B2026.05 | 25.12 | — | 7.95 | 29.85 | 69.35 | 72.45 | 78.92 | 48.76 | 65.28 | 1.02 | |
| ReadAgentExecutor Backbone=MiniMax-M2.52026.05 | 23.67 | — | 5.96 | 18.92 | 56.45 | 69.39 | 37.76 | 7.06 | 49.98 | 1.13 | |
| MemoryOSExecutor Backbone=MiniMax-M2.52026.05 | 23.26 | — | 5.46 | 19.11 | 54.91 | 67.35 | 38.78 | 14.76 | 55.1 | 1.15 | |
| LangMemExecutor Backbone=MiniMax-M2.52026.05 | 23.05 | — | 5.44 | 17.16 | 54.91 | 69.39 | 38.78 | 8.56 | 44.9 | 1.2 | |
| A-MEMExecutor Backbone=MiniMax-M2.52026.05 | 22.98 | — | 6.02 | 19.11 | 53.51 | 67.35 | 50 | 8.22 | 48.98 | 1.15 | |
| MemoryBankExecutor Backbone=MiniMax-M2.52026.05 | 22.58 | — | 6.18 | 19.85 | 55.93 | 70.41 | 46.94 | 9.12 | 56.63 | 1.12 | |
| RAGExecutor Backbone=MiniMax-M2.52026.05 | 22.41 | — | 6.08 | 19.76 | 55.62 | 68.37 | 42.86 | 6.88 | 57.14 | 1.13 | |
| ReadAgentExecutor Backbone=Qwen2.5-72B2026.05 | 22.34 | — | 5.58 | 19.04 | 56.23 | 68.37 | 33.16 | 6.5 | 46.94 | 1.13 | |
| Mem0Executor Backbone=Qwen2.5-72B2026.05 | 22.07 | — | 6.3 | 18.43 | 47.44 | 68.37 | 12.76 | 3.01 | 39.8 | 1.11 | |
| RAGExecutor Backbone=Qwen2.5-72B2026.05 | 21.77 | — | 5.87 | 20.2 | 56.08 | 68.37 | 35.71 | 5.54 | 54.08 | 1.13 | |
| LangMemExecutor Backbone=Qwen2.5-72B2026.05 | 21.59 | — | 6.01 | 19.87 | 55.41 | 67.35 | 41.33 | 6.77 | 57.14 | 1.17 | |
| A-MEMExecutor Backbone=Qwen2.5-72B2026.05 | 21.21 | — | 6 | 20.79 | 57.12 | 70.41 | 41.33 | 8.95 | 52.04 | 1.1 | |
| Mem0Executor Backbone=MiniMax-M2.52026.05 | 21.16 | — | 6.39 | 17.31 | 42.38 | 71.43 | 3.06 | 0.98 | 16.33 | 1.03 | |
| MemoryBankExecutor Backbone=Qwen2.5-72B2026.05 | 21.05 | — | 6.11 | 19.46 | 54.83 | 69.39 | 43.67 | 8.66 | 54.08 | 1.12 | |
| MemoryOSExecutor Backbone=Qwen2.5-72B2026.05 | 20.55 | — | 5.68 | 18.12 | 54.47 | 67.35 | 24.49 | 8.97 | 60.2 | 1.15 | |
| CoNExecutor Backbone=MiniMax-M2.52026.05 | 20.12 | — | 6.21 | 20.58 | 54.91 | 66.33 | 40.82 | 7.21 | 55.1 | 1.12 | |
| CoNExecutor Backbone=Qwen2.5-72B2026.05 | 19.15 | — | 6.03 | 21.9 | 56.77 | 65.31 | 35.71 | 6.58 | 54.08 | 1.12 | |
| RandomSearchExecutor Backbone=MiniMax-M2.52026.05 | 15 | — | 1.6 | 12.9 | 11.5 | 19.4 | 92.9 | 7.3 | 64.3 | 100 | |
| RandomSearchExecutor Backbone=Qwen2.5-72B2026.05 | 15 | — | 1.6 | 12.9 | 11.5 | 19.4 | 92.9 | 7.3 | 64.3 | 100 | |
| NNPredictorExecutor Backbone=MiniMax-M2.52026.05 | 13.6 | — | 1.8 | 12.7 | 15.7 | 19.4 | 0 | 0 | 63.3 | 100 | |
| NNPredictorExecutor Backbone=Qwen2.5-72B2026.05 | 13.6 | — | 1.8 | 12.7 | 15.7 | 19.4 | 0 | 0 | 63.3 | 100 | |
| NelderMeadExecutor Backbone=MiniMax-M2.52026.05 | 12.2 | — | 2.4 | 12.4 | 31.4 | 19.4 | 91.8 | 81.6 | 64.3 | 135 | |
| NelderMeadExecutor Backbone=Qwen2.5-72B2026.05 | 12.2 | — | 2.4 | 12.4 | 31.4 | 19.4 | 91.8 | 81.6 | 64.3 | 135 | |
| A-MEMExecutor Backbone=Human2026.05 | — | 6.88 | — | — | — | — | — | — | — | — | |
| CoNExecutor Backbone=Human2026.05 | — | 6.98 | — | — | — | — | — | — | — | — | |
| LangMemExecutor Backbone=Human2026.05 | — | 6.79 | — | — | — | — | — | — | — | — | |
| Mem0Executor Backbone=Human2026.05 | — | 5.92 | — | — | — | — | — | — | — | — | |
| MemoryBankExecutor Backbone=Human2026.05 | — | 7.22 | — | — | — | — | — | — | — | — | |
| MemoryOSExecutor Backbone=Human2026.05 | — | 6.83 | — | — | — | — | — | — | — | — | |
| NelderMeadExecutor Backbone=Human2026.05 | — | 6.42 | — | — | — | — | — | — | — | — | |
| NNPredictorExecutor Backbone=Human2026.05 | — | 4.62 | — | — | — | — | — | — | — | — | |
| RAGExecutor Backbone=Human2026.05 | — | 7.1 | — | — | — | — | — | — | — | — | |
| RandomSearchExecutor Backbone=Human2026.05 | — | 6.05 | — | — | — | — | — | — | — | — | |
| ReadAgentExecutor Backbone=Human2026.05 | — | 6.86 | — | — | — | — | — | — | — | — | |
| SkillPCFExecutor Backbone=Human2026.05 | — | 9.12 | — | — | — | — | — | — | — | — |