Skill invocation in LLM agents on 30 artifact-rich tasks (test)
18Data Computation WinsQuality-First
Evaluation Results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Quality-FirstEcosystem Size (|S|)=2002026.03 | 18 | 1 | 5 | 12 | 6 | 6 | 11 | 7 | 6 | 17 | 7 | 0 | 20 | 3 | 1 | 100 | |
| Quality-FirstEcosystem Size (|S|)=200K2026.03 | 15 | 7 | 2 | 15 | 5 | 4 | 13 | 3 | 8 | 9 | 4 | 11 | 18 | 1 | 5 | 100 | |
| Quality-FirstEcosystem Size (|S|)=1K2026.03 | 14 | 4 | 6 | 11 | 7 | 6 | 13 | 2 | 9 | 9 | 10 | 5 | 20 | 2 | 2 | 100 | |
| Efficiency-FirstEcosystem Size (|S|)=2002026.03 | 13 | 2 | 9 | 7 | 10 | 7 | 6 | 9 | 9 | 11 | 8 | 5 | 14 | 3 | 7 | 58.5 | |
| w/ Full Pool.Ecosystem Size (|S|)=1K2026.03 | 11 | 1 | 12 | 9 | 5 | 10 | 14 | 2 | 8 | 10 | 4 | 10 | 4 | 4 | 16 | 48.1 | |
| Efficiency-FirstEcosystem Size (|S|)=200K2026.03 | 11 | 7 | 6 | 12 | 7 | 5 | 14 | 4 | 6 | 10 | 5 | 9 | 16 | 2 | 6 | 89 | |
| Simplicity-FirstEcosystem Size (|S|)=2002026.03 | 10 | 4 | 10 | 7 | 9 | 8 | 8 | 7 | 9 | 12 | 6 | 6 | 13 | 3 | 8 | 53.6 | |
| Efficiency-FirstEcosystem Size (|S|)=1K2026.03 | 10 | 4 | 10 | 11 | 5 | 8 | 15 | 1 | 8 | 8 | 10 | 6 | 10 | 6 | 8 | 76.1 | |
| Simplicity-FirstEcosystem Size (|S|)=1K2026.03 | 9 | 3 | 12 | 10 | 5 | 9 | 7 | 3 | 14 | 9 | 8 | 7 | 11 | 6 | 7 | 56.3 | |
| VanillaEcosystem Size (|S|)=1K2026.03 | 9 | 2 | 13 | 5 | 6 | 13 | 6 | 2 | 16 | 5 | 6 | 13 | 4 | 4 | 16 | 0 | |
| Simplicity-FirstEcosystem Size (|S|)=200K2026.03 | 8 | 3 | 13 | 11 | 5 | 8 | 8 | 3 | 13 | 12 | 5 | 7 | 14 | 3 | 7 | 56 | |
| w/ Full Pool.Ecosystem Size (|S|)=200K2026.03 | 7 | 4 | 13 | 4 | 5 | 15 | 12 | 2 | 10 | 8 | 3 | 13 | 4 | 6 | 14 | 17.2 | |
| VanillaEcosystem Size (|S|)=2002026.03 | 6 | 5 | 13 | 4 | 5 | 15 | 5 | 8 | 11 | 2 | 5 | 17 | 2 | 5 | 17 | 0 | |
| VanillaEcosystem Size (|S|)=200K2026.03 | 6 | 5 | 13 | 3 | 8 | 13 | 6 | 2 | 16 | 10 | 5 | 9 | 0 | 4 | 20 | 0 | |
| w/ Full Pool.Ecosystem Size (|S|)=2002026.03 | 5 | 4 | 15 | 11 | 8 | 5 | 13 | 3 | 8 | 3 | 4 | 17 | 2 | 4 | 18 | 24.3 |