Lifelong Knowledge Editing on E-VQA Lifelong Sequential
96.67Rel. ScoreLiveEdit
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| LiveEditBackbone=BLIP2 (2.7B), Number of Edits=12024.11 | 96.67 | 94.2 | 93.82 | 100 | 100 | 96.94 | |
| LTEBackbone=BLIP2 (2.7B), Number of Edits=12024.11 | 95.74 | 93.86 | 86.9 | 97.93 | 87.97 | 92.48 | |
| LiveEditBackbone=BLIP2 (2.7B), Number of Edits=102024.11 | 95.65 | 93.46 | 90.98 | 100 | 100 | 96.02 | |
| LiveEditBackbone=BLIP2 (2.7B), Number of Edits=1002024.11 | 95.25 | 92.94 | 85.48 | 100 | 99.76 | 94.69 | |
| LTEBackbone=BLIP2 (2.7B), Number of Edits=102024.11 | 94.96 | 92.43 | 85.23 | 94.19 | 87.01 | 90.76 | |
| LiveEditBackbone=BLIP2 (2.7B), Number of Edits=10002024.11 | 94.42 | 91.98 | 84.65 | 100 | 97.38 | 93.69 | |
| LiveEditBackbone=LLaVA (7B), Number of Edits=12024.11 | 94.28 | 94.51 | 88.01 | 100 | 100 | 95.36 | |
| LTEBackbone=LLaVA (7B), Number of Edits=12024.11 | 94.16 | 93.57 | 93.59 | 94.08 | 86.26 | 92.33 | |
| FT-LBackbone=LLaVA (7B), Number of Edits=12024.11 | 93.88 | 87.98 | 80.25 | 99.61 | 94.78 | 91.3 | |
| LiveEditBackbone=LLaVA (7B), Number of Edits=102024.11 | 93.79 | 93.21 | 86.42 | 100 | 100 | 94.68 | |
| LEMoEBackbone=LLaVA (7B), Number of Edits=12024.11 | 93.6 | 92.77 | 89.99 | 99.28 | 96.98 | 94.52 | |
| LEMoEBackbone=BLIP2 (2.7B), Number of Edits=12024.11 | 93.56 | 92.23 | 91.4 | 98.5 | 85.21 | 92.18 | |
| LiveEditBackbone=LLaVA (7B), Number of Edits=1002024.11 | 93.54 | 92.34 | 85.89 | 100 | 99.31 | 94.21 | |
| MENDBackbone=BLIP2 (2.7B), Number of Edits=12024.11 | 93.13 | 92.76 | 93.07 | 92 | 75.81 | 89.35 | |
| LiveEditBackbone=LLaVA (7B), Number of Edits=10002024.11 | 92.93 | 90.16 | 84.3 | 100 | 96.43 | 92.76 | |
| LTEBackbone=LLaVA (7B), Number of Edits=102024.11 | 92.83 | 91.41 | 90.82 | 86.38 | 85.52 | 89.39 | |
| LTEBackbone=BLIP2 (2.7B), Number of Edits=1002024.11 | 92.78 | 90.36 | 83.88 | 94.33 | 81.79 | 88.63 | |
| LEMoEBackbone=LLaVA (7B), Number of Edits=102024.11 | 91.95 | 86.54 | 79.82 | 85.19 | 49.81 | 78.66 | |
| FT-MBackbone=BLIP2 (2.7B), Number of Edits=12024.11 | 91.7 | 87.24 | 33.3 | 100 | 85.22 | 79.49 | |
| LEMoEBackbone=BLIP2 (2.7B), Number of Edits=102024.11 | 91.59 | 85.92 | 86.82 | 83.64 | 29.71 | 75.54 | |
| RECIPEBackbone=LLaVA (7B), Number of Edits=12024.11 | 91.37 | 86.51 | 87.73 | 94.27 | 88.88 | 89.75 | |
| MENDBackbone=LLaVA (7B), Number of Edits=12024.11 | 91.23 | 90.05 | 91.29 | 91.02 | 90.22 | 90.76 | |
| FT-LBackbone=LLaVA (7B), Number of Edits=102024.11 | 90.57 | 84.14 | 73.21 | 95.56 | 81.5 | 85 | |
| RECIPEBackbone=LLaVA (7B), Number of Edits=102024.11 | 90.22 | 85.92 | 86.24 | 90.34 | 88.11 | 88.17 | |
| RECIPEBackbone=LLaVA (7B), Number of Edits=1002024.11 | 89.86 | 83.32 | 84.82 | 87.37 | 85.08 | 86.09 | |
| RECIPEBackbone=BLIP2 (2.7B), Number of Edits=12024.11 | 89.42 | 86.24 | 87.53 | 99.87 | 89.16 | 90.45 | |
| SERACBackbone=LLaVA (7B), Number of Edits=12024.11 | 89.33 | 83.72 | 84.97 | 82.05 | 23.78 | 72.77 | |
| LTEBackbone=BLIP2 (2.7B), Number of Edits=10002024.11 | 89.32 | 82.82 | 81.51 | 94.86 | 69.83 | 83.67 | |
| LTEBackbone=LLaVA (7B), Number of Edits=1002024.11 | 88.92 | 87.89 | 85.72 | 84.34 | 81.6 | 85.69 | |
| SERACBackbone=BLIP2 (2.7B), Number of Edits=12024.11 | 88.39 | 84.5 | 84.25 | 85.82 | 26 | 73.79 | |
| RECIPEBackbone=BLIP2 (2.7B), Number of Edits=102024.11 | 88.27 | 85.37 | 84.8 | 97.78 | 88.49 | 88.94 | |
| RECIPEBackbone=BLIP2 (2.7B), Number of Edits=1002024.11 | 88.12 | 82.49 | 83.1 | 98.97 | 86.53 | 87.84 | |
| SERACBackbone=LLaVA (7B), Number of Edits=102024.11 | 88.09 | 83.4 | 83.57 | 64.91 | 15.5 | 67.1 | |
| SERACBackbone=LLaVA (7B), Number of Edits=1002024.11 | 88.08 | 81.53 | 82.48 | 62.13 | 12.9 | 65.42 | |
| FT-MBackbone=LLaVA (7B), Number of Edits=12024.11 | 87.29 | 76.11 | 53.23 | 100 | 96.95 | 82.72 | |
| RECIPEBackbone=LLaVA (7B), Number of Edits=10002024.11 | 87 | 76.81 | 83.09 | 86.95 | 87.03 | 84.18 | |
| SERACBackbone=BLIP2 (2.7B), Number of Edits=1002024.11 | 86.98 | 81.27 | 80.97 | 71.04 | 15.55 | 67.17 | |
| SERACBackbone=BLIP2 (2.7B), Number of Edits=102024.11 | 86.12 | 82.24 | 82.52 | 70.24 | 16.06 | 67.43 | |
| SERACBackbone=LLaVA (7B), Number of Edits=10002024.11 | 85.57 | 75.58 | 82.01 | 62.46 | 15.69 | 64.26 | |
| FT-MBackbone=BLIP2 (2.7B), Number of Edits=102024.11 | 85.27 | 81.09 | 34.6 | 100 | 49 | 69.99 | |
| RECIPEBackbone=BLIP2 (2.7B), Number of Edits=10002024.11 | 84.99 | 74.2 | 82.04 | 96.82 | 87.73 | 85.16 | |
| FT-MBackbone=LLaVA (7B), Number of Edits=102024.11 | 84.9 | 73.53 | 49.99 | 100 | 55.98 | 72.88 | |
| LTEBackbone=LLaVA (7B), Number of Edits=10002024.11 | 83.93 | 82.55 | 81.34 | 83.97 | 73.09 | 80.98 | |
| SERACBackbone=BLIP2 (2.7B), Number of Edits=10002024.11 | 83.35 | 70.8 | 80.32 | 67.66 | 13.13 | 63.05 | |
| FT-MBackbone=LLaVA (7B), Number of Edits=1002024.11 | 82.9 | 72.83 | 47.26 | 100 | 43.39 | 69.28 | |
| FT-LBackbone=LLaVA (7B), Number of Edits=1002024.11 | 79.67 | 70.05 | 64.07 | 83.47 | 54.44 | 70.34 | |
| FT-LBackbone=LLaVA (7B), Number of Edits=10002024.11 | 71.39 | 59.83 | 57.41 | 55.55 | 48.99 | 58.63 | |
| TPBackbone=BLIP2 (2.7B), Number of Edits=12024.11 | 70.14 | 65.8 | 53.05 | 98.11 | 85.33 | 74.49 | |
| FT-MBackbone=LLaVA (7B), Number of Edits=10002024.11 | 69.57 | 56.34 | 44.07 | 100 | 41.47 | 62.29 | |
| TPBackbone=BLIP2 (2.7B), Number of Edits=102024.11 | 55.86 | 52.39 | 43 | 91.26 | 63.62 | 61.22 | |
| FT-LBackbone=BLIP2 (2.7B), Number of Edits=102024.11 | 53.77 | 48.91 | 37.66 | 97.68 | 80.4 | 63.68 | |
| FT-LBackbone=BLIP2 (2.7B), Number of Edits=12024.11 | 52.86 | 48.8 | 32.94 | 98.24 | 94.27 | 65.42 | |
| FT-LBackbone=BLIP2 (2.7B), Number of Edits=1002024.11 | 52.47 | 43.68 | 39.43 | 91.13 | 47.72 | 54.88 | |
| FT-MBackbone=BLIP2 (2.7B), Number of Edits=1002024.11 | 48.04 | 40.71 | 29.86 | 100 | 33.25 | 50.37 | |
| FT-LBackbone=BLIP2 (2.7B), Number of Edits=10002024.11 | 45.1 | 34.62 | 35.42 | 48.42 | 41.24 | 40.96 | |
| TPBackbone=BLIP2 (2.7B), Number of Edits=1002024.11 | 44.26 | 38.24 | 33.28 | 43.81 | 38.54 | 39.63 | |
| LEMoEBackbone=LLaVA (7B), Number of Edits=1002024.11 | 42.41 | 36.6 | 34.33 | 78.57 | 53.28 | 49.04 | |
| MENDBackbone=BLIP2 (2.7B), Number of Edits=102024.11 | 40.4 | 32.21 | 27.73 | 87.92 | 67.74 | 51.2 | |
| FT-MBackbone=BLIP2 (2.7B), Number of Edits=10002024.11 | 40.4 | 31.46 | 27.85 | 100 | 27.44 | 45.43 | |
| TPBackbone=LLaVA (7B), Number of Edits=12024.11 | 35.95 | 36.12 | 28.65 | 93.87 | 97.61 | 58.44 | |
| TPBackbone=LLaVA (7B), Number of Edits=102024.11 | 32.71 | 31.23 | 28.58 | 75.1 | 91.17 | 51.76 | |
| LEMoEBackbone=LLaVA (7B), Number of Edits=10002024.11 | 30.8 | 25.75 | 24.32 | 71.45 | 46.23 | 39.71 | |
| LEMoEBackbone=BLIP2 (2.7B), Number of Edits=1002024.11 | 29.61 | 21.7 | 27.05 | 79.48 | 32.61 | 38.09 | |
| TPBackbone=LLaVA (7B), Number of Edits=1002024.11 | 29.37 | 28.72 | 24.66 | 14.64 | 45.01 | 28.48 | |
| TPBackbone=BLIP2 (2.7B), Number of Edits=10002024.11 | 20.63 | 15.09 | 18.41 | 8.65 | 8.25 | 14.21 | |
| LEMoEBackbone=BLIP2 (2.7B), Number of Edits=10002024.11 | 19.73 | 17.34 | 18.22 | 72.01 | 31.06 | 31.67 | |
| MENDBackbone=BLIP2 (2.7B), Number of Edits=1002024.11 | 17.69 | 16.39 | 18.31 | 91.52 | 67.94 | 42.37 | |
| TPBackbone=LLaVA (7B), Number of Edits=10002024.11 | 16.56 | 16.8 | 15.65 | 7.28 | 15.6 | 14.38 | |
| MENDBackbone=BLIP2 (2.7B), Number of Edits=10002024.11 | 15.84 | 14.35 | 17.73 | 91.74 | 70.17 | 41.97 | |
| MENDBackbone=LLaVA (7B), Number of Edits=102024.11 | 3.58 | 3.55 | 3.53 | 2.1 | 1.26 | 2.8 | |
| MENDBackbone=LLaVA (7B), Number of Edits=1002024.11 | 2.22 | 2.2 | 2.21 | 0.21 | 0.62 | 1.49 | |
| MENDBackbone=LLaVA (7B), Number of Edits=10002024.11 | 0.04 | 0.05 | 0.05 | 0.08 | 0.09 | 0.06 |