List your company

ResearchFormerly WizWand

OVEN

Benchmarks

TaskDatasetSOTA resultTrendResults

Knowledge-Based Visual Question Answering

OVEN (val)

25.1Accuracy (All)

20

Visual Entity Recognition

OVEN

28.5HM (Unseen)

15

Visual Question Answering

OVEN Query 1.0 (test)

30.9HM

15

Fine-grained Entity Recognition

OVEN Entity 1.0 (test)

29.6HM

15

(Image, Text)-to-Multimodal Retrieval

OVEN

75.3R@5

14

(Image, Text)-to-Text Retrieval

OVEN

57.8Recall@5

14

Visual Entity Recognition

OVEN entity (test)

65Top-1 Accuracy (Seen)

11

Open-Vocabulary Entity Recognition

OVEN

0.789EM

8

Multi-modal retrieval (Image-Text to Text/Image-Text)

OVEN QS

8.39Recall@5

7

Visual Entity Recognition

OVEN (test)

33.6Top-1 Acc (Seen)

7

Multi-modal to multi-modal retrieval ((q^i, q^t) -> (c^i, c^t))

OVEN (held-out)

67.6R@5

6

Multi-modal to text retrieval ((q^i, q^t) -> c^t)

OVEN (held-out)

46.5R@5

6

Multimodal Retrieval

OVEN-8

75.98R@5

6

Multimodal Retrieval

OVEN-6

58.17R@5

6

Visual Question Answering

OVEN

15.88EM

6

Open-domain Visual Entity Recognition

OVEN Wiki (human evaluation set)

76.1Score (Seen Entities)

6

OVEN M2KR

42.8R@1

4

Image-text-to-multimodal retrieval

OVEN M-BEIR (test)

67.6Recall@5

4

Image-text-to-text retrieval

OVEN M-BEIR (test)

46.9Recall@5

4

Open-Vocabulary Entity Grounding

OVEN (test)

23.1Accuracy

2