Relation Extraction on WIKIDATA one-shot (out-of-distribution)
66.66PrecisionCOGRE
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| COGREBackbone=Qwen2.5-14B-Instruct, Method Strategy=After RL, Training Reward Protocol=HIT@DICT + Acc2025.10 | 66.66 | 23.52 | 34.78 | |
| COGREBackbone=Phi-4, Method Strategy=After RL, Training Reward Protocol=HIT@DICT + Acc2025.10 | 37.84 | 27.45 | 31.81 | |
| Direct MatchingBackbone=Qwen2.5-14B-Instruct2025.10 | 23.33 | 13.73 | 17.28 | |
| COGREBackbone=Phi-4, Method Strategy=After RL, Training Reward Protocol=Accuracy (Acc)2025.10 | 14.29 | 31.37 | 19.63 | |
| COGREBackbone=Qwen2.5-14B-Instruct, Method Strategy=After RL, Training Reward Protocol=Accuracy (Acc)2025.10 | 12.5 | 31.37 | 17.88 | |
| COGREBackbone=Qwen2.5-14B-Instruct, Method Strategy=Vanilla Prompting2025.10 | 11.57 | 27.45 | 16.28 | |
| Semantic Rule Matcher2025.10 | 6.54 | 6.08 | 6.31 | |
| COGREBackbone=Phi-4, Method Strategy=Vanilla Prompting2025.10 | 5.08 | 52.94 | 9.28 | |
| Simple ReasoningBackbone=Qwen2.5-14B-Instruct2025.10 | 3.96 | 54.91 | 7.39 | |
| Simple ReasoningBackbone=Phi-42025.10 | 1.99 | 56.86 | 3.85 | |
| SUMASKBackbone=Phi-42025.10 | 1.48 | 27.45 | 2.8 | |
| Direct MatchingBackbone=Phi-42025.10 | 1.13 | 49.02 | 2.22 |