Tool Shortlisting on AppWorld v1.0 (test) benchmark leaderboard · SOTA2 Research

Tool Shortlisting on AppWorld v1.0 (test)

0.71R-precision (AZ)TabAgent

Evaluation Results

Method	Links
TabAgent2026.02		0.71	0.66	0.9	0.96	0.61	0.95	0.93	0.98	1	0.88	0.97	0.95	1	1	0.92
DSR-FT2026.02		0.69	0.56	0.85	0.92	0.64	0.93	0.9	0.98	1	0.85	0.95	0.93	0.99	1	0.88
TabAgent2026.02		0.57	0.58	0.66	0.62	0.7	0.91	0.88	0.98	1	0.88	0.93	0.91	1	1	0.9
Llama-3.1-8B-Instruct2026.02		0.56	0.45	0.38	0.7	0.67	0.56	0.5	0.65	0.71	0.67	0.56	0.5	0.65	0.73	0.69
DSR-FT2026.02		0.45	0.52	0.57	0.63	0.46	0.89	0.84	0.95	0.99	0.82	0.9	0.87	0.95	0.99	0.85
DSR2026.02		0.39	0.18	0.3	0.55	0.3	0.65	0.46	0.82	0.89	0.62	0.67	0.53	0.88	0.96	0.7
Llama-3.2-3B-Instruct2026.02		0.38	0.34	0.4	0.42	0.39	0.6	0.58	0.63	0.76	0.64	0.66	0.66	0.66	0.79	0.67
BM252026.02		0.28	0.19	0.35	0.42	0.28	0.48	0.51	0.54	0.72	0.6	0.54	0.58	0.57	0.74	0.71
Llama-3.2-1B-Instruct2026.02		0.02	0.03	0.06	0.07	0.09	0.29	0.27	0.29	0.45	0.33	0.32	0.29	0.33	0.46	0.34