ViT | 🍎 Paper Today I Read 🦔

[55] Position Prediction as an Effective Pretraining Strategy

ViT PE 2022Q2 25min apple

[25] Intriguing Properties of Vision Transformers

ViT WIP 2020Q2 NeurIPS

[24] DINO: Emerging Properties in Self-Supervised Vision Transformers

ViT SSL 2021Q2 facebook

[5] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

ViT backbone 2021Q1 re-read

[4] Conditional Positional Encodings for Vision Transformers

[3] Twins: Revisiting the Design of Spatial Attention in Vision Transformers

ViT backbone 2021Q1

[2] ELSA: Enhanced Local Self-Attention for Vision Transformer

2021Q4 ViT attention

[1] Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

2021Q4 ViT backbone