Utterance Retrieval on WeChat English-translated (test)
64.18PrecisionGPT-4o
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4ozero-shot inference=true, MLDR fine-tuning=false2026.06 | 64.18 | 34.54 | 44.91 | 42.23 | |
| Gemini-1.5zero-shot inference=true, MLDR fine-tuning=false2026.06 | 59.94 | 26.56 | 36.81 | 35 | |
| Claude-3.5zero-shot inference=true, MLDR fine-tuning=false2026.06 | 59.09 | 39.46 | 47.32 | 42.82 | |
| F2RVLM-Qwen2-VL-7BMLDR fine-tuning=true2026.06 | 50.19 | 53.81 | 51.94 | 45.23 | |
| Qwen2-VL-7BMLDR fine-tuning=true2026.06 | 38.85 | 56.66 | 46.09 | 38.14 | |
| Mimo-7B-RLMLDR fine-tuning=true2026.06 | 37.44 | 40.92 | 39.1 | 30.49 | |
| Qwen2.5-VL-7BMLDR fine-tuning=true2026.06 | 34.82 | 63.52 | 44.98 | 37.27 | |
| Qwen2.5-VL-72Bzero-shot inference=true, MLDR fine-tuning=false2026.06 | 30.31 | 32.04 | 31.15 | 21.57 | |
| F2RVLM-Qwen2.5-VL-3BMLDR fine-tuning=true2026.06 | 30.25 | 70.18 | 42.28 | 35 | |
| DeepSeek-VL2-Small-16BMLDR fine-tuning=true2026.06 | 26.43 | 14.66 | 18.86 | 11.97 | |
| Qwen2.5-VL-3BMLDR fine-tuning=true2026.06 | 23.78 | 76.42 | 36.27 | 28.87 | |
| mPLUG-Owl3-2BMLDR fine-tuning=true2026.06 | 22.32 | 16.92 | 19.25 | 10.14 | |
| F2RVLM-Qwen2-VL-2BMLDR fine-tuning=true2026.06 | 17.03 | 86.27 | 28.45 | 19.51 | |
| Qwen2-VL-2BMLDR fine-tuning=true2026.06 | 15.9 | 86.34 | 26.86 | 16.69 |