Patent Claim Evaluation on Patent-CE 1.0 (test)
54.8Completeness AccuracyG-Eval-4
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| G-Eval-4Type=LLM-as-a-judge2025.05 | 54.8 | 54.8 | 55.6 | 55.9 | 45.9 | 43.7 | 54.8 | 54.6 | 49.6 | 46.9 | |
| PatClaimEvalType=Ours2025.05 | 52.7 | 53.2 | 60.3 | 59.5 | 50 | 49.3 | 52.7 | 54.7 | 56.5 | 57.4 | |
| UniEvalType=Miscellaneous2025.05 | 52.2 | 44.1 | 53.3 | 45.8 | 46.7 | 38.3 | 52.2 | 45 | 52.7 | 44.8 | |
| METEORType=N-gram2025.05 | 51.1 | 43.2 | 54.9 | 47.2 | 48.9 | 40.1 | 53.3 | 46 | 50.5 | 42.8 | |
| BLEU-1Type=N-gram2025.05 | 50.5 | 42.6 | 54.3 | 46.7 | 47.8 | 39.2 | 53.8 | 46.5 | 52.2 | 44.3 | |
| ROUGE-1Type=N-gram2025.05 | 50.5 | 42.8 | 52.2 | 44.8 | 45.7 | 37.5 | 52.2 | 45 | 50.5 | 42.9 | |
| ROUGE-2Type=N-gram2025.05 | 50.5 | 42.8 | 50.5 | 43.4 | 44.6 | 36.6 | 50.5 | 43.6 | 49.5 | 42 | |
| ROUGE-LType=N-gram2025.05 | 49.5 | 41.8 | 50.5 | 43.5 | 46.7 | 38.3 | 54.3 | 46.9 | 51.1 | 43.4 | |
| BLEU-4Type=N-gram2025.05 | 48.9 | 41.4 | 50.5 | 43.5 | 45.1 | 37 | 50 | 43.2 | 49.5 | 42 | |
| BERTScoreType=Embedding2025.05 | 46.7 | 39.1 | 48.9 | 42.2 | 45.7 | 37.6 | 51.1 | 44.8 | 48.4 | 41.8 | |
| MoverScoreType=Embedding2025.05 | 45.1 | 38.4 | 46.7 | 41.1 | 44.6 | 36.8 | 48.4 | 42 | 46.2 | 39.5 | |
| SimCSEType=Embedding2025.05 | 44.6 | 38.4 | 45.7 | 40.7 | 41.3 | 34.6 | 48.4 | 42.6 | 44.6 | 38.6 | |
| BARTScoreType=Embedding2025.05 | 43.5 | 35.9 | 42.9 | 36.3 | 44.6 | 36.4 | 47.3 | 40.7 | 44.6 | 37.5 | |
| AlignScoreType=Miscellaneous2025.05 | 42.9 | 36.4 | 44 | 38 | 46.7 | 38.3 | 47.3 | 40.8 | 47.3 | 40.1 |