List your company
ResearchFormerly WizWand

Document Visual Question Answering on VisualMRC (test)

55.76ROUGE-LLayoutLLM-7B
28.688835.716942.74549.7731Apr 8, 2024
Updated 8mo ago

Evaluation results

MethodLinks
LayoutLLM-7BModel Category=Ours, Evaluation Protocol=Zero-shot, Backbone Initialization=Vicuna-1.5-7B2024.0455.76
LayoutLLM-7BModel Category=Ours, Evaluation Protocol=Zero-shot, Backbone Initialization=Llama2-7B-chat2024.0455.73
Vicuna-7BModel Category=LLM, Input Representation=Plain Text, Evaluation Protocol=Zero-shot2024.0453.63
Llama2-7B-chatModel Category=LLM, Input Representation=Plain Text, Evaluation Protocol=Zero-shot2024.0452.84
Vicuna-7BModel Category=LLM, Input Representation=Layout Text (Text + Box), Evaluation Protocol=Zero-shot2024.0452.55
Vicuna-1.5-7BModel Category=LLM, Input Representation=Plain Text, Evaluation Protocol=Zero-shot2024.0452.13
Llama2-7B-chatModel Category=LLM, Input Representation=Layout Text (Text + Box), Evaluation Protocol=Zero-shot2024.0449.26
Vicuna-1.5-7BModel Category=LLM, Input Representation=Layout Text (Text + Box), Evaluation Protocol=Zero-shot2024.0447.22
Qwen-VL-7BModel Category=MLLM, Evaluation Protocol=Fine-tuned, Implementation=Original Paper2024.0442.52
LLaVAR-7BModel Category=MLLM, Evaluation Protocol=Zero-shot, Implementation=Original Paper2024.0436.37
LLaVA-1.5-7BModel Category=MLLM, Evaluation Protocol=Zero-shot2024.0435.23
Llama2-7BModel Category=LLM, Input Representation=Layout Text (Text + Box), Evaluation Protocol=Zero-shot2024.0433.82
Llama2-7BModel Category=LLM, Input Representation=Plain Text, Evaluation Protocol=Zero-shot2024.0429.73