Innovation Prediction on Patents Publication 2024 (test)
0.936AUC-ROC (0.005%)TechToken BERT (IPC)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TechToken BERT (IPC)Backbone Model=BERT, Embedding Method (IPC vs CLS)=TechToken IPC, Pooling Strategy=Average of closest technological tokens2026.05 | 0.936 | 0.938 | 0.94 | 0.94 | |
| TechToken BERT (CLS)Backbone Model=BERT, Embedding Method (IPC vs CLS)=TechToken CLS, Pooling Strategy=Average embedding using CLS token2026.05 | 0.908 | 0.911 | 0.903 | 0.904 | |
| LLaMA PatentsBackbone Model=LLaMA 3.1 8B, Fine-tuning status=true (cutoff Dec 2023), Embedding Method=Average embedding2026.05 | 0.856 | 0.832 | 0.846 | 0.84 | |
| BERT4Patents FTBackbone Model=BERT4Patents, Fine-tuning status=true, Embedding Method=Average embedding2026.05 | 0.765 | 0.761 | 0.753 | 0.745 | |
| BERT4PatentsBackbone Model=BERT4Patents, Fine-tuning status=false, Embedding Method=Average embedding2026.05 | 0.725 | 0.716 | 0.693 | 0.682 |