Scientific hypothesis discovery on TOMATO-Chem2 (51 top-tier papers)
26.96RecallMC2_with_strong_feedback_x4_oracle_rank
Evaluation Results
| Method | Links | |
|---|---|---|
| MC2_with_strong_feedback_x4_oracle_rankDescription=MC + initial blueprint + (oracle-ranking) + MC2 + [(oracle ranking + strong feedback) + MC2]x4, # Search Steps=90.12026.05 | 26.96 | |
| MC2_with_strong_feedback_x3_oracle_rankDescription=MC + initial blueprint + (oracle-ranking) + MC2 + [(oracle ranking + strong feedback) + MC2]x3, # Search Steps=123.12026.05 | 25.7 | |
| MC2_with_strong_feedback_x2_oracle_rankDescription=MC + initial blueprint + (oracle-ranking) + MC2 + [(oracle ranking + strong feedback) + MC2]x2, # Search Steps=102.12026.05 | 25.54 | |
| MC2_with_strong_feedback_oracle_rankDescription=MC + initial blueprint + (oracle-ranking) + MC2 + [(oracle ranking + strong feedback) + MC2]x1, # Search Steps=129.42026.05 | 23.1 | |
| MC2_with_feedback_x3_oracle_rankDescription=MC + initial blueprint + (oracle-ranking) + MC2 + [(oracle ranking + feedback) + MC2]x3, # Search Steps=123.12026.05 | 22.35 | |
| MC2_with_feedback_oracle_rankDescription=MC + initial blueprint + (oracle-ranking) + MC2 + [(oracle ranking + feedback) + MC2]x1, # Search Steps=166.12026.05 | 21.98 | |
| MC2_with_feedback_x2_oracle_rankDescription=MC + initial blueprint + (oracle-ranking) + MC2 + [(oracle ranking + feedback) + MC2]x2, # Search Steps=149.92026.05 | 21.91 | |
| MC2_with_feedback_x4_oracle_rankDescription=MC + initial blueprint + (oracle-ranking) + MC2 + [(oracle ranking + feedback) + MC2]x4, # Search Steps=107.42026.05 | 20.99 | |
| MC2_with_MC_input_oracle_rankDescription=MC + initial blueprint + (oracle-ranking) + MC2, # Search Steps=336.62026.05 | 18.26 | |
| MC_with_feedback_with_hintDescription=MC + initial blueprint + (oracle ranking + feedback) + MC2026.05 | 16.93 | |
| MC_with_soft_feedback_with_hintDescription=MC + initial blueprint + (oracle ranking + soft feedback) + MC2026.05 | 16.78 | |
| MC_with_hintDescription=MC + initial blueprint2026.05 | 15.37 | |
| MC2_with_MC_input_self_rankDescription=MC + initial blueprint + (self-ranking) + MC2, # Search Steps=321.22026.05 | 12.74 | |
| baseline_MCDescription=MC2026.05 | 11.44 | |
| baseline_MC2Description=MC2, # Search Steps=478.62026.05 | 10.33 |