Binary Code Similarity Detection on BinaryCorp-VirtualAssembly (test)
73.1Recall@1Trex
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TrexSetting=O2+L12026.01 | 73.1 | 66.5 | |
| jTransSetting=O2+L1, Evaluation Protocol=Linear Probe2026.01 | 71.5 | 64.4 | |
| jTransSetting=O2+L1, Evaluation Protocol=Zero Shot2026.01 | 69.3 | 62.2 | |
| PalmTreeSetting=O2+L12026.01 | 67.7 | 60.9 | |
| Asm2VecSetting=O2+L12026.01 | 66.9 | 58.9 | |
| GNNSetting=O2+L12026.01 | 64.3 | 59.2 | |
| SAFESetting=O2+L12026.01 | 64.3 | 57.5 | |
| jTransSetting=O1+L3, Evaluation Protocol=Linear Probe2026.01 | 60.8 | 52.6 | |
| jTransSetting=O2+L3, Evaluation Protocol=Linear Probe2026.01 | 60.1 | 52 | |
| OrderMattersSetting=O2+L12026.01 | 60 | 53.5 | |
| jTransSetting=O1+L1, Evaluation Protocol=Linear Probe2026.01 | 57.3 | 49.4 | |
| TrexSetting=O2+L32026.01 | 51.3 | 43.6 | |
| TrexSetting=O1+L32026.01 | 51.1 | 42.2 | |
| Asm2VecSetting=O2+L32026.01 | 51 | 42.6 | |
| jTransSetting=O2+L3, Evaluation Protocol=Zero Shot2026.01 | 51 | 43 | |
| PalmTreeSetting=O2+L32026.01 | 49.6 | 42 | |
| jTransSetting=O1+L1, Evaluation Protocol=Zero Shot2026.01 | 49 | 41.2 | |
| ShieldedCodeSetting=O0+L12026.01 | 48.8 | 57.5 | |
| GraphEmbSetting=O2+L12026.01 | 48.6 | 44.7 | |
| TrexSetting=O1+L12026.01 | 47.7 | 38.8 | |
| jTransSetting=O1+L3, Evaluation Protocol=Zero Shot2026.01 | 47.2 | 39.3 | |
| ShieldedCodeSetting=O2+L32026.01 | 46.9 | 47.9 | |
| Asm2VecSetting=O1+L12026.01 | 44.9 | 36.7 | |
| GeminiSetting=O2+L12026.01 | 41.6 | 36.7 | |
| Asm2VecSetting=O1+L32026.01 | 40.9 | 33.2 | |
| jTransSetting=O0+L3, Evaluation Protocol=Linear Probe2026.01 | 40.4 | 30.9 | |
| PalmTreeSetting=O1+L12026.01 | 40.3 | 32.6 | |
| SAFESetting=O2+L32026.01 | 37.7 | 28.3 | |
| PalmTreeSetting=O1+L32026.01 | 35.5 | 28.1 | |
| SAFESetting=O1+L12026.01 | 34.5 | 24.7 | |
| ShieldedCodeSetting=O1+L32026.01 | 34.4 | 46.9 | |
| jTransSetting=O0+L1, Evaluation Protocol=Linear Probe2026.01 | 33.3 | 24.5 | |
| SAFESetting=O1+L32026.01 | 32.1 | 22.1 | |
| OrderMattersSetting=O1+L12026.01 | 31.9 | 24.8 | |
| DeGPTSetting=O0+L12026.01 | 31.2 | 26.7 | |
| ShieldedCodeSetting=O2+L12026.01 | 30.9 | 43 | |
| ShieldedCodeSetting=O1+L12026.01 | 30.6 | 47.5 | |
| ShieldedCodeSetting=O0+L32026.01 | 27.2 | 39.7 | |
| CodeBERT-BinarySetting=O0+L12026.01 | 26.5 | 22.3 | |
| OrderMattersSetting=O1+L32026.01 | 26 | 17.8 | |
| DeGPTSetting=O2+L32026.01 | 24.8 | 21.4 | |
| OrderMattersSetting=O2+L32026.01 | 23.3 | 15.8 | |
| GraphEmbSetting=O2+L32026.01 | 22.2 | 16.6 | |
| CodeBERT-BinarySetting=O2+L32026.01 | 22.1 | 18.7 | |
| GraphEmbSetting=O1+L12026.01 | 21.7 | 15.4 | |
| GNNSetting=O2+L32026.01 | 21.4 | 17.5 | |
| BinDiffSetting=O0+L12026.01 | 19.8 | 14.2 | |
| GNNSetting=O1+L12026.01 | 19.7 | 15.5 | |
| GeminiSetting=O2+L32026.01 | 19.5 | 15.1 | |
| GraphEmbSetting=O1+L32026.01 | 19.5 | 13.5 | |
| GNNSetting=O1+L32026.01 | 18.7 | 14.6 | |
| jTransSetting=O0+L3, Evaluation Protocol=Zero Shot2026.01 | 18.2 | 12.2 | |
| GeminiSetting=O1+L12026.01 | 16.1 | 12.2 | |
| BinDiffSetting=O2+L32026.01 | 15.6 | 10.8 | |
| PalmTreeSetting=O0+L32026.01 | 15.2 | 9.7 | |
| TrexSetting=O0+L32026.01 | 14.8 | 8.8 | |
| SAFESetting=O0+L32026.01 | 14.7 | 7.9 | |
| jTransSetting=O0+L1, Evaluation Protocol=Zero Shot2026.01 | 13.7 | 8.8 | |
| GeminiSetting=O1+L32026.01 | 13.3 | 9.9 | |
| PalmTreeSetting=O0+L12026.01 | 13 | 8 | |
| SAFESetting=O0+L12026.01 | 12.7 | 6.8 | |
| TrexSetting=O0+L12026.01 | 11.8 | 7.3 | |
| GraphEmbSetting=O0+L32026.01 | 11 | 6.3 | |
| GraphEmbSetting=O0+L12026.01 | 8.7 | 5 | |
| Asm2VecSetting=O0+L32026.01 | 8.3 | 5.2 | |
| OrderMattersSetting=O0+L32026.01 | 7.5 | 4 | |
| Asm2VecSetting=O0+L12026.01 | 7.2 | 4.6 | |
| OrderMattersSetting=O0+L12026.01 | 6.2 | 4 | |
| GNNSetting=O0+L32026.01 | 6.1 | 4.1 | |
| GeminiSetting=O0+L32026.01 | 4.9 | 3 | |
| GNNSetting=O0+L12026.01 | 4.8 | 3.6 | |
| GeminiSetting=O0+L12026.01 | 3.7 | 2.4 |