[185] LLaVA-OneVision: Easy Visual Task Transfer

TL;DR

I read this because.. : 높은 mathvista 성능
task : MLLM
problem : multi-image, video 까지 한번에 커버하는 VLM
idea : anyres를 domain 별로 조금씩 다르게 설정. 데이터 잘 모아서 학습!
input/output : {image or images or video, question} -> answer
architecture : SigLIP SO400M + 2 layer MLP + Qwen2 {0.5B, 7.6B, 72.7B}
objective : CE loss
baseline : QwenVL, Gemini-Pro, Claude 3.5 Sonnet, GPT4V, GPT4o, VILA, Cambrian, InternVL
data : stage 1.0(여전히 LCS-553K), stage 1.5(3.5M llava recap, UReader, SynDog, chinese ShareGPT4V), stage 2.0(curated Single Image 3.2M and OneVision 1.6M)
evaluation : AI2D, ChartQA, DocVQA, InfoVQA, Mathverse, Mathvista, MMBench, MME, MMStar, MMMU, MMVet, SeedBench, ScienceQA, ImageDC, RealWorldQA, … Multi-image benchs(5), Video Benchs(9)
result : single image eval 관련해서 동일 스케일인 Intern2-VL-8B와 비교해봤을 때 유의미하게 높은건 MathVista 정도 인듯? (63.2), multi-image, video bench에서 준수한 성능
contribution : 빠르게 비디오 벤치 찍음.
etc. :