Knowledge Editing on VLKEB
98.77ReliabilityLiveEdit
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| LiveEditBackbone=BLIP2 OPT (2.7B), # Edit=12024.11 | 98.77 | 98.08 | 94.89 | 100 | 100 | 98.35 | |
| LiveEditBackbone=BLIP2 OPT (2.7B), # Edit=10002024.11 | 97 | 91.92 | 87.53 | 100 | 100 | 95.29 | |
| LiveEditBackbone=LLaVA-V1.5 (7B), # Edit=12024.11 | 96.43 | 95.22 | 93.72 | 100 | 100 | 97.08 | |
| LiveEditBackbone=LLaVA-V1.5 (7B), # Edit=102024.11 | 95.54 | 94.52 | 91.25 | 100 | 100 | 96.26 | |
| MENDBackbone=BLIP2 OPT (2.7B), # Edit=12024.11 | 94.91 | 93.81 | 93.84 | 94.98 | 86.54 | 92.82 | |
| LEMoEBackbone=LLaVA-V1.5 (7B), # Edit=12024.11 | 94.85 | 93.09 | 91.67 | 87.03 | 87.88 | 90.9 | |
| LEMoEBackbone=BLIP2 OPT (2.7B), # Edit=12024.11 | 94.59 | 93.14 | 92.37 | 94.53 | 61.53 | 87.23 | |
| LTEBackbone=LLaVA-V1.5 (7B), # Edit=12024.11 | 94.42 | 93.57 | 93.22 | 86.84 | 79.69 | 89.55 | |
| FT-LBackbone=LLaVA-V1.5 (7B), # Edit=12024.11 | 94.29 | 87 | 92.22 | 91.16 | 91.37 | 91.21 | |
| LTEBackbone=BLIP2 OPT (2.7B), # Edit=12024.11 | 94.13 | 91.93 | 92.23 | 93.89 | 92.27 | 92.89 | |
| RECIPEBackbone=LLaVA-V1.5 (7B), # Edit=12024.11 | 92.67 | 92.35 | 91.01 | 89.67 | 82.85 | 89.71 | |
| FT-MBackbone=BLIP2 OPT (2.7B), # Edit=12024.11 | 92.64 | 80.97 | 63.62 | 100 | 83.02 | 84.05 | |
| RECIPEBackbone=BLIP2 OPT (2.7B), # Edit=12024.11 | 92.38 | 89.74 | 89.17 | 97.13 | 94.46 | 92.58 | |
| LiveEditBackbone=LLaVA-V1.5 (7B), # Edit=10002024.11 | 92.22 | 83.97 | 82.75 | 100 | 100 | 91.79 | |
| MENDBackbone=LLaVA-V1.5 (7B), # Edit=12024.11 | 92.13 | 91.28 | 90.22 | 89.19 | 90.13 | 90.59 | |
| LEMoEBackbone=LLaVA-V1.5 (7B), # Edit=102024.11 | 91.55 | 84.58 | 81.03 | 67.19 | 72.81 | 79.43 | |
| LTEBackbone=LLaVA-V1.5 (7B), # Edit=102024.11 | 90.06 | 81.52 | 88.11 | 83.4 | 81.48 | 84.91 | |
| SERACBackbone=LLaVA-V1.5 (7B), # Edit=12024.11 | 89.77 | 89.11 | 87.92 | 66.68 | 14.2 | 69.54 | |
| FT-LBackbone=LLaVA-V1.5 (7B), # Edit=102024.11 | 88.05 | 85.32 | 85.23 | 74.53 | 85.74 | 83.77 | |
| SERACBackbone=BLIP2 OPT (2.7B), # Edit=12024.11 | 87.95 | 84.67 | 85.2 | 68.1 | 17.75 | 68.73 | |
| RECIPEBackbone=LLaVA-V1.5 (7B), # Edit=102024.11 | 83.92 | 76.23 | 82.84 | 86.33 | 83.69 | 82.6 | |
| SERACBackbone=LLaVA-V1.5 (7B), # Edit=102024.11 | 81.55 | 74.49 | 80.24 | 54.71 | 13.15 | 60.83 | |
| FT-MBackbone=LLaVA-V1.5 (7B), # Edit=12024.11 | 76.31 | 65.57 | 59.43 | 100 | 92.35 | 78.73 | |
| FT-MBackbone=LLaVA-V1.5 (7B), # Edit=102024.11 | 68.63 | 57.57 | 56.56 | 100 | 82.99 | 73.15 | |
| FT-LBackbone=LLaVA-V1.5 (7B), # Edit=10002024.11 | 68.14 | 66.38 | 66.98 | 65.61 | 75.35 | 68.49 | |
| LEMoEBackbone=LLaVA-V1.5 (7B), # Edit=10002024.11 | 67.97 | 61.07 | 58.16 | 48.48 | 44.06 | 55.95 | |
| LTEBackbone=LLaVA-V1.5 (7B), # Edit=10002024.11 | 64.51 | 56.26 | 64.8 | 80.85 | 76.52 | 68.59 | |
| RECIPEBackbone=LLaVA-V1.5 (7B), # Edit=10002024.11 | 62 | 56.84 | 61.5 | 85.37 | 82.07 | 69.56 | |
| LTEBackbone=BLIP2 OPT (2.7B), # Edit=10002024.11 | 61.67 | 51.05 | 61.6 | 94.78 | 90.94 | 72.01 | |
| SERACBackbone=LLaVA-V1.5 (7B), # Edit=10002024.11 | 60.93 | 56.49 | 60.06 | 52.94 | 15.04 | 49.09 | |
| FT-LBackbone=BLIP2 OPT (2.7B), # Edit=10002024.11 | 55.39 | 54.34 | 53.87 | 50.8 | 54 | 53.68 | |
| RECIPEBackbone=BLIP2 OPT (2.7B), # Edit=10002024.11 | 54.64 | 46.54 | 54.1 | 94.6 | 96.93 | 69.37 | |
| FT-LBackbone=BLIP2 OPT (2.7B), # Edit=12024.11 | 54.31 | 54.27 | 54.08 | 98.4 | 94.37 | 71.09 | |
| SERACBackbone=BLIP2 OPT (2.7B), # Edit=10002024.11 | 53.58 | 45.78 | 52.42 | 56.81 | 16.9 | 45.1 | |
| FT-MBackbone=LLaVA-V1.5 (7B), # Edit=10002024.11 | 53.41 | 48.8 | 43.16 | 100 | 57.03 | 60.48 | |
| TPBackbone=BLIP2 OPT (2.7B), # Edit=12024.11 | 50.98 | 49.47 | 50.88 | 94.76 | 78.57 | 64.93 | |
| TPBackbone=LLaVA-V1.5 (7B), # Edit=12024.11 | 50.77 | 55.7 | 51.65 | 87.93 | 90.43 | 67.3 | |
| FT-MBackbone=BLIP2 OPT (2.7B), # Edit=10002024.11 | 47.03 | 49.68 | 46.99 | 100 | 41.41 | 57.02 | |
| TPBackbone=LLaVA-V1.5 (7B), # Edit=102024.11 | 44.56 | 47.52 | 45.36 | 52.21 | 66.61 | 51.25 | |
| MENDBackbone=BLIP2 OPT (2.7B), # Edit=10002024.11 | 37.22 | 38.03 | 37.19 | 91.49 | 84.1 | 57.61 | |
| LEMoEBackbone=BLIP2 OPT (2.7B), # Edit=10002024.11 | 34.74 | 33.43 | 32.05 | 55.55 | 50.04 | 41.16 | |
| TPBackbone=BLIP2 OPT (2.7B), # Edit=10002024.11 | 24.36 | 24.21 | 24.25 | 16.37 | 19.96 | 21.83 | |
| TPBackbone=LLaVA-V1.5 (7B), # Edit=10002024.11 | 5.46 | 4.81 | 5.51 | 2.77 | 7.19 | 5.15 | |
| MENDBackbone=LLaVA-V1.5 (7B), # Edit=102024.11 | 0.18 | 0.24 | 0.05 | 0.03 | 0.19 | 0.14 | |
| MENDBackbone=LLaVA-V1.5 (7B), # Edit=10002024.11 | 0.03 | 0.05 | 0.07 | 0.06 | 0.08 | 0.06 |