Preference-driven Tool Calling on MPT Context-Guided, Preference Induction
37.95P-EMPREFINE
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| PREFINEBase LLM=GPT-52026.04 | 37.95 | 65.87 | 64.8 | |
| PREFINEBase LLM=GPT-5-mini2026.04 | 32.97 | 67.71 | 65.16 | |
| GPT-5Methodology=Base Prompting2026.04 | 32.42 | 65.34 | 64.01 | |
| PREFINEBase LLM=Gemini-3-Flash2026.04 | 29.76 | 69.98 | 67.17 | |
| Gemini-3-FlashMethodology=Base Prompting2026.04 | 28.67 | 69.66 | 66.49 | |
| PREFINEBase LLM=GPT-4o-mini2026.04 | 28.12 | 70.73 | 65.03 | |
| Mem0Methodology=Memory-Augmented, Backbone=Gemini-3-Flash2026.04 | 27.99 | 65.52 | 62.05 | |
| LangMemMethodology=Memory-Augmented, Backbone=Gemini-3-Flash2026.04 | 26.62 | 69.1 | 63.56 | |
| RAG (Top-5)Methodology=Memory-Augmented, Backbone=Gemini-3-Flash2026.04 | 24.91 | 67.6 | 61.34 | |
| GPT-5-miniMethodology=Base Prompting2026.04 | 23.21 | 63.46 | 61.78 | |
| PREFINEBase LLM=R1-Distill-Llama-8B2026.04 | 22.12 | 62.07 | 58.22 | |
| R1-Distill-Llama-8BMethodology=Base Prompting2026.04 | 18.43 | 62.6 | 58.02 | |
| GPT-4o-miniMethodology=Base Prompting2026.04 | 18.43 | 62.46 | 57.34 | |
| PREFINEBase LLM=R1-Distill-Qwen-7B2026.04 | 17.2 | 58.93 | 51.2 | |
| PREFINEBase LLM=CodeGemma-7B2026.04 | 16.38 | 65.86 | 61 | |
| R1-Distill-Qwen-7BMethodology=Base Prompting2026.04 | 7.17 | 27.88 | 25.5 | |
| PREFINEBase LLM=Gemma-3-12B2026.04 | 5.67 | 74.11 | 63.24 | |
| CodeGemma-7BMethodology=Base Prompting2026.04 | 4.1 | 32.78 | 30.35 | |
| Gemma-3-12BMethodology=Base Prompting2026.04 | 2.73 | 57.64 | 48.16 |