Domain-Specific Knowledge Retrieval on AgriIR Evaluation Set 191 queries 1.0 (human-annotated)
3.36Answer MeanChatGPT-4o
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ChatGPT-4oComposite Score Lambda=λ=0.72026.03 | 3.36 | 0.94 | 88.5 | — | — | — | 0.84 | 0.233 | 1 | |
| AgriIR_Gemma3:27BBackbone=Gemma3, Model Scale=27B, Composite Score Lambda=λ=0.72026.03 | 3.24 | 0.87 | 86.9 | 1.69 | 0.55 | 73 | 0.82 | 0.208 | 2 | |
| Gemini 2.5 FlashComposite Score Lambda=λ=0.72026.03 | 3.12 | 1.01 | 78.7 | — | — | — | 0.779 | 0.25 | 3 | |
| GPT-OSS-120BModel Scale=120B, Composite Score Lambda=λ=0.72026.03 | 2.82 | 0.99 | 70.5 | — | — | — | 0.705 | 0.246 | 5 | |
| AgriIR_Llama3.2:3B (w/o DB)Backbone=Llama3.2, Model Scale=3B, Database Integration=w/o DB, Composite Score Lambda=λ=0.72026.03 | 2.72 | 1.12 | 68 | 1.62 | 0.64 | 69.7 | 0.718 | 0.252 | 4 | |
| AgriIR_Llama3.2:3B (w/ DB)Backbone=Llama3.2, Model Scale=3B, Database Integration=w/ DB, Composite Score Lambda=λ=0.72026.03 | 2.62 | 1.09 | 63.9 | 1.51 | 0.68 | 59 | 0.684 | 0.258 | 6 | |
| AgriIR_Gemma3:1B (w/o DB)Backbone=Gemma3, Model Scale=1B, Database Integration=w/o DB, Composite Score Lambda=λ=0.72026.03 | 2.44 | 1.05 | 49.2 | 1.47 | 0.65 | 54.9 | 0.648 | 0.233 | 7 |