Structured Metadata Retrieval on DEVDATABENCH Multilingual raw-schema (holdout)
52.8R@1harrier-270m-ft
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| harrier-270m-fttype=ft, multilingual, Number of Parameters=270M, Serialization=raw-schema2026.06 | 52.8 | 88.1 | 65.3 | 70.6 | |
| multilingual-e5-small-fttype=ft, multilingual, Number of Parameters=118M, Serialization=raw-schema2026.06 | 52.2 | 89.1 | 65.2 | 70.7 | |
| paraphrase-ml-minilm-fttype=ft, multilingual, Number of Parameters=118M, Serialization=raw-schema2026.06 | 47.6 | 87.9 | 61.6 | 67.8 | |
| jina-embeddings-v5-nanotype=zero-shot open, Serialization=raw-schema2026.06 | 39.7 | 73.1 | 51.1 | 56.1 | |
| text-embedding-3-largetype=zero-shot API, Serialization=raw-schema2026.06 | 38.6 | 73.5 | 49.8 | 55.6 | |
| granite-97m-fttype=ft, multilingual, Number of Parameters=97M, Serialization=raw-schema2026.06 | 37.4 | 74.7 | 50.1 | 55.6 | |
| harrier-oss-v1-0.6btype=zero-shot open, Number of Parameters=600M, Serialization=raw-schema2026.06 | 36.9 | 70.1 | 48.1 | 53 | |
| BM25 (lexical)type=lexical, Serialization=raw-schema2026.06 | 36 | 47.4 | 40.2 | 41.3 | |
| noinstruct-small-fttype=ft, English-vocab, Number of Parameters=33M, Serialization=raw-schema2026.06 | 30.4 | 68 | 42.7 | 48.3 | |
| text-embedding-3-smalltype=zero-shot API, Serialization=raw-schema2026.06 | 25.3 | 52 | 34.6 | 38.1 |