List your company
ResearchFormerly WizWand

E-commerce Agent Interaction on WebShop

70.4Average RewardMCTS
-2.46895216.44894935.3668554.284751Jul 25, 2025Sep 15, 2025Nov 6, 2025Dec 28, 2025Feb 18, 2026Apr 11, 2026Jun 2, 2026
Updated 5mo ago

Evaluation results

MethodLinks
MCTSBase LLM=Llama2-70B2025.0770.4
BPOApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.0867.45
Tree DPOBase LLM=Llama2-70B2025.0765.9
SFTBase LLM=Llama2-70B2025.0765.1
MPOApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.0855.2
Qwen-2.5-7B-InstructApproach=System-1, Evaluation Protocol=Zero-shot2025.0854.28
SFTApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.0852.94
ETOApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.0852.08
o3-miniApproach=System-2, Evaluation Protocol=Zero-shot2025.0849.02
Qwen-3-ThinkingApproach=System-2, Evaluation Protocol=Zero-shot2025.0842.16
Deepseek-R1Approach=System-2, Evaluation Protocol=Zero-shot2025.0840.31
Llama-3.1-8B-InstructApproach=System-1, Evaluation Protocol=Zero-shot2025.0833.93
DELTAMEMLLM=DeepSeek-V4-flash, Decoding Temperature=0, Prompt Framework=ReAct, Node Embeddings=E5-base-v2, Learning Protocol=Online learning, in-context demonstration=nicl = 12026.060.6254
SynapseLLM=DeepSeek-V4-flash, Decoding Temperature=0, Prompt Framework=ReAct, Node Embeddings=E5-base-v2, Learning Protocol=Online learning, in-context demonstration=nicl = 12026.060.589
No MemoryLLM=DeepSeek-V4-flash, Decoding Temperature=0, Prompt Framework=ReAct, Node Embeddings=E5-base-v2, Learning Protocol=Zero-shot, in-context demonstration=nicl = 12026.060.3674
RBankLLM=DeepSeek-V4-flash, Decoding Temperature=0, Prompt Framework=ReAct, Node Embeddings=E5-base-v2, Learning Protocol=Online learning, in-context demonstration=nicl = 12026.060.3498
AWMLLM=DeepSeek-V4-flash, Decoding Temperature=0, Prompt Framework=ReAct, Node Embeddings=E5-base-v2, Learning Protocol=Online learning, in-context demonstration=nicl = 12026.060.3337