🍎 Paper Today I Read 🦔

[222] Qwen3-VL Technical Report

[221] Scaling Synthetic Data Creation with 1,000,000,000 Personas

dataset LLM 2024Q3

[220] VideoRoPE: What Makes for Good Video Rotary Position Embedding?

MLLM 2025Q1 video

[219] GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning

RL MLLM 2025Q3

[218] Qwen2.5-VL Technical Report

alibaba MLLM 2025Q2 qwen

[217] PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding

meta 2025Q2 video

[216] Emerging Properties in Unified Multimodal Pretraining

MLLM 2025Q2 WORLD-MODEL

[215] Group Sequence Policy Optimization

LLM RL 2025Q3

[214] Learning to Model the World With Language

ICML RL 2023Q3 WORLD-MODEL

[213] Skywork-R1V3 Technical Report

MLLM reasoning 2025Q3

[211] Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought

MLLM reasoning 2025Q2

[212] MiMo-VL Technical Report

MLLM reasoning 2025Q2

[210] Weight Ensembling Improves Reasoning in Language Models

RL reasoning 2025Q2

[209] SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training

google RL Berkley 2025Q1

[208] FastCuRL: Curriculum Reinforcement Learning with Progressive Context Extension for Efficient Training R1-like Reasoning Models

25min RL 2025Q1

[206] Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models

25min RL MLLM 2025Q1

[207] MM-EUREKA: Exploring Visual Aha Moment with Rule-based Large-scale Reinforcement Learning

RL MLLM 2025Q1

[205] LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!

Berkley reasoning 2025Q1

[204] DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

25min RL reasoning 2025Q1

[203] DeepSeek-V3 Technical Report

WIP 25min LLM RL 2024Q4

[202] s1: Simple test-time scaling

25min test-time-scaling reasoning 2025Q1

[201] VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment

RL reasoning 2025Q1

[200] Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling

25min RL 2025Q1 THU

[199] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

RL reasoning 2025Q1

[198] Kimi k1.5: Scaling Reinforcement Learning with LLMs

multimodal RL reasoning 2025Q1

[197] Free Process Rewards without Process Labels

25min RL 2024Q4

[196] Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations

ACL RL 2023Q4 reasoning

[195] STaR: Self-Taught Reasoner Bootstrapping Reasoning With Reasoning

2022Q1 google 25min reasoning

[194] Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters

DeepMind 2024Q3 reasoning

[193] Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective

survey 2024Q4 reasoning

[192] Scaling Test-time Compute with Open Models (hf blog)

2024Q4 test-time-scaling reasoning

[191] Critique-out-Loud Reward Models

AllenAI LLM RL 2024Q3

[190] Solving math word problems with process and outcome-based feedback

DeepMind 2022Q4 RL

[189] Training Verifiers to Solve Math Word Problems

2021Q4 openAI 25min RL

read torch titan

distributed

[188] LLaVA-CoT: Let Vision Language Models Reason Step-by-Step

MLLM 2024Q4

[187] Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization

RL MLLM 2024Q4 SHU

[186] The Llama 3 Herd of Models

LLM meta 2024Q3

[185] LLaVA-OneVision: Easy Visual Task Transfer

25min MLLM 2024Q4

[184] Improve Vision Language Model Chain-of-thought Reasoning

CMU MLLM 2024Q3

[183] MultiMath: Bridging Visual and Mathematical Reasoning for Large Language Models

MLLM 2024Q3 STEM

[182] Calibrated Self-Rewarding Vision Language Models

NeurIPS 25min RL MLLM 2024Q2

[181] Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs

LLM RL 2023Q3

[180] Phantom of Latent for Large Language and Vision Models

MLLM 2024Q3

[179] Aligning Large Multimodal Models with Factually Augmented RLHF

25min RL 2023Q3 MLLM Berkley

[178] RLAIF-V: Aligning MLLMs through Open-Source AI Feedback for Super GPT-4V Trustworthiness

RL MLLM 2024Q2

[177] Fine-grained Image Captioning with CLIP Reward

2022Q2 25min RL NAACL

[176] Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive

LLM RL 2024Q1

[175] Dense Reward for Free in Reinforcement Learning from Human Feedback

ICML LLM RL 2024Q3

[174] Evaluations for Object Hallucinations

survey evaluation MLLM

[171] CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs

ECCV RL MLLM 2024Q3

[172] RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

CVPR RL MLLM 2024Q2

[173] Detecting and Preventing Hallucinations in Large Vision Language Models

AAAI RL 2023Q3 MLLM ScaleAI

[170] Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback

RL AI2 2024Q2

[169] Direct Preference Optimization: Your Language Model is Secretly a Reward Model

2023Q2 RL

[168] Proximal Policy Optimization Algorithms

2017 RL

[167] Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation

NeurIPS 2023Q4 generation

[166] Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models

MLLM 2024Q1

[165] Rich Human Feedback for Text-to-Image Generation

CVPR 2023Q4 evaluation

[163] What You See is What You Read? Improving Text-Image Alignment Evaluation

google NeurIPS 2023Q2 evaluation

[164] TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering

ICCV evaluation 2023Q3

[162] CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attention

AAAI 2022Q3 25min CLIP

[161] MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks

25min 2022Q4 XAI ACL

[160] ALOHa: A New Measure for Hallucination in Captioning Models

evaluation 2024Q2 NAACL

[159] Long-CLIP: Unlocking the Long-Text Capability of CLIP

25min CLIP 2024Q1

[158] A Mathematical Framework for Transformer Circuits

2021Q4 XAI anthropic

feat: add text span

ICLR 2023Q4

[156] Interpreting CLIP's Image Representation via Text-Based Decomposition

ICLR CLIP XAI 2023Q4

[157] LeGrad: An Explainability Method for Vision Transformers via Feature Formation Sensitivity

CLIP XAI 2024Q2

feat: add LeGrad

2024Q2

[155] Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings

google evaluation generation 2024Q2

feat: llava next hf implementation

[154] Mismatch Quest: Visual and Textual Feedback for Image-Text Misalignment

google XAI evaluation 2024Q2

[153] Contrastive Explanations for Model Interpretability

2021Q1 XAI emnlp AI2

[152] Sigmoid Loss for Language Image Pre-Training

25min CLIP 2023Q1

[151] FOIL it! Find One mismatch between Image and Language caption

dataset 2017 XAI evaluation

[150] Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization

2016 25min XAI

[149] Noise-aware Learning from Web-crawled Image-Text Data for Image Captioning

ICCV 25min 2022Q4 kakao

[148] I Can't Believe There's No Images! Learning Visual Tasks Using only Language Supervision

ICCV 25min CLIP 2023Q3 AI2

[147] Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers

ICCV 2021Q1 XAI

[146] Transformer Interpretability Beyond Attention Visualization

2020Q1 CVPR XAI

[145] CLIPScore: A Reference-free Evaluation Metric for Image Captioning

2021Q2 CLIP emnlp evaluation AI2

[144] Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

multilingual alibaba 2023Q3 MLLM qwen

[143] Honeybee: Locality-enhanced Projector for Multimodal LLM

kakao 2023Q4 MLLM

[142] Trust Region Policy Optimization

2015 RL

[141] Alpha-CLIP: A CLIP Model Focusing on Wherever You Want

multimodal CLIP 2023Q4

[140] Improved Baselines with Visual Instruction Tuning

multimodal LLM 2023Q3 MLLM

[139] Davidsonian Scene Graph: Improving Reliability in Fine-Grained Evaluation for Text-to-Image Generation

google 2023Q4 evaluation generation

[138] ShareGPT4V: Improving Large Multi-Modal Models with Better Captions

multimodal dataset 2023Q4 MLLM

[137] mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration

multimodal LLM 2023Q4 alibaba

[136] Visually-Situated Natural Language Understanding with Contrastive Reading Model and Frozen Large Language Models

multimodal naver 2021Q3 document emnlp

[135] Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text

multimodal dataset NeurIPS 2023Q2

[134] Asynchronous Methods for Deep Reinforcement Learning

2016 DeepMind RL

[133] DataComp: In search of the next generation of multimodal datasets

dataset CLIP 2023Q2

[132] Hyperbolic Image-Text Representations

ICML CLIP 2023Q2 meta

[131] Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels

2021Q1 CVPR naver

[129] Grounding Language Models to Images for Multimodal Inputs and Outputs

ICML 25min 2023Q1 CMU

[130] Segment Anything

segmentation 2023Q2 meta

[128] Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding

ICML google 2022Q3 document

[127] Linearly Mapping from Image to Text Space

multimodal ICLR 2023Q1

[126] ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision

multimodal 2021Q1 25min kakao

[125] RILS: Masked Visual Reconstruction in Language Semantic Space

CVPR CLIP 2023Q1

feat: add sparse rcnn

CVPR 2020Q4

[124] LiT: Zero-Shot Transfer with Locked-image text Tuning

2021Q4 google CLIP

[123] Robust fine-tuning of zero-shot models

openAI google CVPR 2022Q3 CLIP domainshift

[122] Neural Architecture Search without Training

ICML 2020Q2 NAS

[121] Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entities

multimodal CLIP 2023Q1 retrieval

feat: add open-clip

2021Q1 CLIP

[120] Large-scale Bilingual Language-Image Contrastive Learning

2022Q1 CLIP multilingual

[119] Visual Instruction Tuning

multimodal NeurIPS 2023Q2

[118] PaLI-X: On Scaling up a Multilingual Vision and Language Model

multimodal google 2023Q2

[117] Multimodal Chain-of-Thought Reasoning in Language Models

multimodal 2023Q1

[116] Data Distributional Properties Drive Emergent In-Context Learning in Transformers

DeepMind NeurIPS 2022Q2

[115] ImageBind: One Embedding Space To Bind Them All

multimodal 25min 2023Q2 meta

[114] MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks

multimodal google 2023Q1

[113] BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

multimodal 2023Q1 salesforce

[112] RoFormer: Enhanced Transformer with Rotary Position Embedding

PE 2021Q2

[111] Perceiver IO: A General Architecture for Structured Inputs & Outputs

multimodal 2021Q2 ICLR DeepMind MTL

[110] Understanding the Role of Self Attention for Efficient Speech Recognition

2022Q1 ICLR 25min transformer

[109] 🦩 Flamingo: a Visual Language Model for Few-Shot Learning

multimodal DeepMind LLM

[108] Unsupervised Vision-Language Parsing: Seamlessly Bridging Visual Scene Graphs with Language Structures via Dependency Relationships

2022Q1 dataset CVPR graph

[107] Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models

GPT 2023Q1

[106] Prefix-Tuning: Optimizing Continuous Prompts for Generation

2021Q1 25min finetuning LLM ACL

[105] LoRA: Low-Rank Adaptation of Large Language Models

2021Q2 microsoft finetuning LLM

[104] GPT Understands, too

2021Q1 prompt GPT finetuning LLM

[103] Deep Sets

NeurIPS 2017

[102] Attention Augmented Convolutional Networks

attention 2019 25min

[101] Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

2017 uncertainty MTL

[100] An Overview of Multi-Task Learning in Deep Neural Networks

2017 survey MTL

[99] LinkNet: Relational Embedding for Scene Graph

NeurIPS 2017 SGG

[98] Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection

NeurIPS object detection 2022Q3 CLIP

[97] Contrastive Language-Image Pre-Training with Knowledge Graph

multimodal NeurIPS graph 2022Q4 CLIP

[96] Vision GNN: An Image is Worth Graph of Nodes

backbone 2022Q1 NeurIPS graph

[95] Pixels to Graphs by Associative Embedding

NeurIPS 2017 SGG one-stage

[94] Recipe for a General, Powerful, Scalable Graph Transformer

long NeurIPS graph 25min transformer

[93] Mining the Benefits of Two-stage and One-stage HOI Detection

2021Q2 NeurIPS 25min HOI

[92] Long-Tail Learning via Logit Adjustment

2020Q3 google 25min imbalance

[91] Deep Residual Learning for Image Recognition

fundamental microsoft 2015

[90] Neural Collaborative Graph Machines for Table Structure Recognition

2021Q4 CVPR graph document

[89] Relational Attention: Generalizing Transformers for Graph-Structured Tasks

microsoft graph 2022Q4 transformer

[88] Relation Networks for Object Detection

2017 microsoft object detection

[87] Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation

2021Q4 CVPR SGG imbalance

[86] Graph R-CNN for Scene Graph Generation

2018 SGG graph two-stage

[85] Dynamic Head: Unifying Object Detection Heads with Attentions

2021Q2 CVPR microsoft object detection

[84] DiffusionDet: Diffusion Model for Object Detection

object detection generative 2022Q4

[83] Variance Networks: When Expectation Does Not Meet Your Expectations

2018 ICLR uncertainty later.. bayesian

[82] Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectors

2021Q1 ICLR object detection uncertainty later..

[81] Equalization Loss for Long-Tailed Object Recognition

2020Q1 object detection SenseTime imbalance

add deformable detr repo

2020Q2 object detection

[80] Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection

2020Q3 object detection imbalance uncertainty

[79] FCOS: Fully Convolutional One-Stage Object Detection

2019 fundamental object detection

[78] Localization Uncertainty Estimation for Anchor-Free Object Detection

2020Q2 25min uncertainty

[77] Interpretable Image Classification with Differentiable Prototype Assignment

2022Q3 ECCV XAI

[75] SESS: Saliency Enhancing with Scaling and Sliding

2022Q3 25min ECCV XAI

[76] Long-tail Detection with Effective Class-Margins

2022Q3 imbalance ECCV

[74] “This is my unicorn, Fluffy”: Personalizing frozen vision-language representations

dataset 2022Q3 25min ECCV nvidia CLIP

[73] Simple Open-Vocabulary Object Detection with Vision Transformers

google object detection 2022Q2 25min ECCV OV

[72] Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsity

2021Q4 ICLR object detection sparse kakao

[71] Large Models are Parsimonious Learners: Activation Sparsity in Trained Transformers

25min sparse 2022Q4 transformer

[70] SSD: Single Shot MultiBox Detector

object detection 2015

[69] End-to-End Object Detection with Transformers

facebook 2020Q2 object detection re-read

[68] Iterative Scene Graph Generation

SGG 2022Q3 one-stage

[67] Deformable DETR: Deformable Transformers for End-to-End Object Detection

2020Q3 ICLR long object detection SenseTime

huggingface DeformableDetr code reading

2020Q2 object detection huggingface

[66] Pointly-Supervised Instance Segmentation

2022Q2 25min ECCV annotation segmentation

[65] Margin Calibration for Long-Tailed Visual Recognition

2021Q4 25min imbalance ECCV

[64] Open-Vocabulary DETR with Conditional Matching

2022Q1 object detection ECCV OV

[63] Masked Autoencoders Are Scalable Vision Learners

2021Q4 SSL 25min

[62] What to Hide from Your Students: Attention-Guided Masked Image Modeling

SSL 2022Q1 25min ECCV

[61] Generative Modeling by Estimating Gradients of the Data Distribution

generative 2020Q4

[60] Efficient Sparsely Activated Transformers

MoE 2022Q3 25min AutoML

[59] MLP-Mixer: An all-MLP Architecture for Vision

backbone 2021Q2 google 25min

[58] MetaFormer Is Actually What You Need for Vision

2021Q4 backbone 25min

[57] Learning Transferable Architectures for Scalable Image Recognition

fundamental 2017 25min AutoML

[56] NICE: Non-linear Independent Components Estimation

fundamental generative 2014

[55] Position Prediction as an Effective Pretraining Strategy

ViT PE 2022Q2 25min apple

[54] Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models

LM MoE 2022Q3 25min

[53] InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

openAI 2016 fundamental generative

[51] Structured Sparse R-CNN for Direct Scene Graph Generation

2020Q2 CVPR SGG one-stage

[52] Sparse R-CNN: End-to-End Object Detection with Learnable Proposals

object detection 2020Q4

[50] Generative Adversarial Networks

fundamental generative re-read 2014

[49] Sparse Graph Attention Networks

2019 graph IEEE

[48] SAC: Accelerating and Structuring Self-Attention via Sparse Adaptive Connection

2020Q1 long NeurIPS graph 25min

[47] Recovering the Unbiased Scene Graphs from the Biased Ones

SGG 25min 2021Q3 imbalance

[45] BGT-Net: Bidirectional GRU Transformer Network for Scene Graph Generation

2021Q4 SGG graph

[46] ReFormer: The Relational Transformer for Image Captioning

SGG 2021Q3 captioning

[44] Context-Aware Scene Graph Generation With Seq2Seq Transformers

ICCV 2021Q4 SGG graph

[41] Panoptic Scene Graph Generation

dataset SGG 2022Q3 25min

[42] DETRs with Hybrid Matching

object detection 2022Q3 25min DETR

[43] Relation Transformer Network

2020Q2 SGG graph

[40] Neural Discrete Representation Learning

DeepMind 2017 generative

[39] Auto-Encoding Variational Bayes

fundamental generative 2013

[38] Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queries

ICCV 2021Q2 SGG one-stage

[37] Relationformer: A Unified Framework for Image-to-Graph Generation

2022Q1 SGG graph one-stage ECCV

RelTR code reading

2022Q1 SGG

[36] SGTR: End-to-end Scene Graph Generation with Transformer

2022Q1 CVPR SGG graph one-stage

[35] RelTR: Relation Transformer for Scene Graph Generation

2022Q1 SGG graph one-stage

[34] What Regularized Auto-Encoders Learn from the Data Generating Distribution

fundamental 2012 generative

[33] Learning to Prompt for Continual Learning

2021Q4 google CVPR continual learning

[32] ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision

multimodal 2021Q2 naver

[31] GIT: A Generative Image-to-text Transformer for Vision and Language

multimodal microsoft 2022Q2

[30] CoCa: Contrastive Captioners are Image-Text Foundation Models

multimodal backbone google 2022Q2

[29] Grounded Language-Image Pre-training

multimodal 2021Q4 few-shot zero-shot microsoft object detection

[28] Learning to Compare: Relation Network for Few-Shot Learning

few-shot zero-shot 2018 CVPR

[27] Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation

few-shot 2020Q1 ICLR

MoEBERT code reading

2022Q1 MoE

[26] Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts

2018 MoE KDD

Sparse MoE code reading

2017 MoE

[25] Intriguing Properties of Vision Transformers

ViT WIP 2020Q2 NeurIPS

[24] DINO: Emerging Properties in Self-Supervised Vision Transformers

ViT SSL 2021Q2 facebook

[23] Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

SSL 2020Q2 google DeepMind

[22] Transformers without Tears: Improving the Normalization of Self-Attention

NLP 2019 fundamental norm

[21] cosFormer: Rethinking Softmax in Attention

NLP attention 2022Q1 ICLR long

[20] Memorizing Transformer

NLP 2022Q1 google ICLR long

[19] Multimodal Explanations: Justifying Decisions and Pointing to the Evidence

multimodal 2018 dataset

[18] Deep Learning with Differential Privacy

WIP privacy 2016 google

[17] Membership Inference Attacks Against Machine Learning Models

privacy 2016

[16] Counterfactual Memorization in Neural Language Models

NLP 2021Q4 privacy LM

[15] Quantifying Memorization Across Neural Language Models

NLP 2022Q1 privacy LM

[14] Longformer: The Long-Document Transformer

NLP AllenAI 2020Q1 long

[13] GPT-3 : Language Models are Few-Shot Learners

NLP few-shot zero-shot openAI 2020Q2

[12] BBPE: Neural Machine Translation with Byte-Level Subwords

NLP 2019 tokenizing facebook AAAI

[11] DALL-E : Zero-Shot Text-to-Image Generation

multimodal 2021Q1 zero-shot openAI

[10] CLIP: Connecting Text and Images

multimodal 2021Q1 few-shot SSL zero-shot CLIP

[9] SimCLR : A Simple Framework for Contrastive Learning of Visual Representations

few-shot SSL 2020Q3 ICML google

[8] SimVLM: Simple Visual Language Model Pretraining with Weak Supervision

multimodal SSL 2021Q2 zero-shot

[7] SLIP: Self-supervision meets Language-Image Pre-training

multimodal 2021Q4 few-shot SSL

[6] Crossing the Format Boundary of Text and Boxes: Towards Unified Vision-Language Modeling

multimodal 2021Q4 backbone multitask

[5] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

ViT backbone 2021Q1 re-read

[4] Conditional Positional Encodings for Vision Transformers

ViT 2021Q1 PE

[3] Twins: Revisiting the Design of Spatial Attention in Vision Transformers

ViT backbone 2021Q1

[2] ELSA: Enhanced Local Self-Attention for Vision Transformer

2021Q4 ViT attention

[1] Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

2021Q4 ViT backbone