Preference-driven Tool Calling on MPT Context-Guided, Preference Transfer
26.19P-EMPREFINE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PREFINEBase LLM=GPT-52026.04 | 26.19 | 67.23 | 59.29 | |
| GPT-5Methodology=Base Prompting2026.04 | 23.94 | 64.27 | 55.47 | |
| PREFINEBase LLM=GPT-5-mini2026.04 | 21.02 | 67.23 | 58.47 | |
| PREFINEBase LLM=Gemini-3-Flash2026.04 | 18.81 | 70.55 | 59.62 | |
| Mem0Methodology=Memory-Augmented, Backbone=Gemini-3-Flash2026.04 | 16.31 | 65.93 | 54.85 | |
| Gemini-3-FlashMethodology=Base Prompting2026.04 | 14.62 | 69.68 | 56.54 | |
| GPT-5-miniMethodology=Base Prompting2026.04 | 11.65 | 61.09 | 52.25 | |
| PREFINEBase LLM=GPT-4o-mini2026.04 | 9.19 | 69.97 | 56.99 | |
| RAG (Top-5)Methodology=Memory-Augmented, Backbone=Gemini-3-Flash2026.04 | 8.26 | 69.4 | 55.88 | |
| LangMemMethodology=Memory-Augmented, Backbone=Gemini-3-Flash2026.04 | 6.57 | 57.59 | 46.79 | |
| R1-Distill-Llama-8BMethodology=Base Prompting2026.04 | 6.14 | 59.37 | 49.57 | |
| GPT-4o-miniMethodology=Base Prompting2026.04 | 4.87 | 61.98 | 48.94 | |
| PREFINEBase LLM=R1-Distill-Llama-8B2026.04 | 4.83 | 52.22 | 42.95 | |
| PREFINEBase LLM=R1-Distill-Qwen-7B2026.04 | 3.6 | 47.38 | 37.81 | |
| PREFINEBase LLM=CodeGemma-7B2026.04 | 1.61 | 67.2 | 53.97 | |
| CodeGemma-7BMethodology=Base Prompting2026.04 | 0.64 | 37.19 | 29.37 | |
| R1-Distill-Qwen-7BMethodology=Base Prompting2026.04 | 0.64 | 25.87 | 20.12 | |
| PREFINEBase LLM=Gemma-3-12B2026.04 | 0.21 | 75.38 | 63.54 | |
| Gemma-3-12BMethodology=Base Prompting2026.04 | 0 | 55.86 | 46.22 |