{"id":2317,"date":"2025-06-11T12:12:33","date_gmt":"2025-06-11T06:42:33","guid":{"rendered":"https:\/\/texpertssolutions.com\/notes\/?p=2317"},"modified":"2026-10-21T11:47:54","modified_gmt":"2026-10-21T06:17:54","slug":"what-are-visual-transformer","status":"publish","type":"post","link":"https:\/\/texpertssolutions.com\/notes\/2025\/06\/11\/what-are-visual-transformer\/","title":{"rendered":"Vision Transformers (ViT): How They Work in Computer Vision"},"content":{"rendered":"\n

Let\u2019s break down Visual Transformers<\/strong> (also called Vision Transformers<\/strong> or ViTs<\/strong>) in a super simple and clear way\ud83e\udde0\ud83d\udc41\ufe0f\u200d\ud83d\udde8\ufe0f\ud83d\udcca<\/p>\n\n\n\n

\ud83e\udde0 What Are Visual Transformers?<\/h2>\n\n\n\n

Visual Transformers<\/strong> are deep learning models that use the Transformer architecture<\/strong> (originally made for text \ud83d\udcc4) to analyze images \ud83d\uddbc\ufe0f<\/strong> instead of using CNNs (Convolutional Neural Networks).<\/p>\n\n\n\n

They were introduced in the paper “An Image is Worth 16×16 Words”<\/strong> by Google in 2020 \ud83d\udcda\ud83d\udd0d<\/p>\n\n\n\n

\n\n\n\n

\ud83d\udce6 Basic Idea:<\/h3>\n\n\n\n

Think of an image as a puzzle<\/strong> \ud83e\udde9:<\/p>\n\n\n\n

    \n
  • We break the image into small patches<\/strong> (like mini-images)<\/li>\n\n\n\n
  • Each patch is treated like a “word” in NLP \ud83e\uddfe<\/li>\n\n\n\n
  • Then the Transformer<\/strong> learns relationships between these patches (like it does with words in a sentence!)<\/li>\n<\/ul>\n\n\n\n
    \n\n\n\n

    \ud83d\udd0d How It Works (Simply Explained):<\/h2>\n\n\n\n

    Step 1\ufe0f\u20e3: \ud83e\udde9 Image to Patches<\/strong><\/h3>\n\n\n\n
      \n
    • Divide the image into small square patches (e.g., 16\u00d716 pixels each)<\/li>\n\n\n\n
    • Flatten each patch into a vector \ud83d\udccf<\/li>\n<\/ul>\n\n\n\n

      Step 2\ufe0f\u20e3: \ud83e\udde0 Embed the Patches<\/strong><\/h3>\n\n\n\n
        \n
      • Just like word embeddings in text, each patch is turned into a numeric vector<\/li>\n\n\n\n
      • Add position info<\/strong> so the model knows where<\/strong> each patch belongs in the image \ud83e\udded<\/li>\n<\/ul>\n\n\n\n

        Step 3\ufe0f\u20e3: \ud83d\udd01 Transformer Encoder<\/strong><\/h3>\n\n\n\n
          \n
        • Feed all patch vectors into the transformer<\/li>\n\n\n\n
        • It uses self-attention<\/strong> to learn relationships between different parts of the image \ud83d\udc41\ufe0f\u200d\ud83d\udde8\ufe0f<\/li>\n<\/ul>\n\n\n\n

          Step 4\ufe0f\u20e3: \ud83e\uddfe Classification or Output<\/strong><\/h3>\n\n\n\n
            \n
          • The final output is usually a classification token<\/strong> (like in BERT for text) that predicts the image class (e.g., “cat” \ud83d\udc31 or “car” \ud83d\ude97)<\/li>\n<\/ul>\n\n\n\n
            \n\n\n\n

            \ud83c\udd9a ViT vs CNN (What\u2019s the Difference?)<\/h2>\n\n\n\n
            Feature<\/th>CNN \ud83e\udde0<\/th>ViT \ud83e\udd16<\/th><\/tr><\/thead>
            Works with<\/td>Pixels directly<\/td>Image patches<\/td><\/tr>
            Learns using<\/td>Convolutions (filters)<\/td>Attention (relationships)<\/td><\/tr>
            Position Awareness<\/td>Built-in (via structure)<\/td>Needs positional encoding<\/td><\/tr>
            Data Requirement<\/td>Works well on small data<\/td>Needs lots of data<\/strong> or pretraining<\/td><\/tr>
            Interpretability<\/td>Less clear<\/td>More explainable with attention \u2728<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
            \n\n\n\n

            \ud83e\uddea Where Are Visual Transformers Used?<\/h2>\n\n\n\n

            \u2705 Image classification (e.g., cat vs dog) \ud83d\udc36\ud83d\udc31
            \u2705 Object detection \ud83e\uddcd\ud83d\udce6
            \u2705 Image segmentation \ud83e\udde0\ud83e\udde9
            \u2705 Medical imaging \ud83e\uddec
            \u2705 Video analysis \ud83c\udfa5<\/p>\n\n\n\n

            \n\n\n\n

            \ud83d\udcc8 Pros and Cons<\/h2>\n\n\n\n
            \u2705 Pros<\/th>\u274c Cons<\/th><\/tr><\/thead>
            Great at global understanding<\/strong> \ud83c\udf0d<\/td>Needs more data<\/strong> to train<\/td><\/tr>
            Works better with pretraining<\/strong> \ud83c\udfcb\ufe0f<\/td>Slower than CNNs on small tasks \ud83d\udc0c<\/td><\/tr>
            Easy to scale \ud83c\udfd7\ufe0f<\/td>More complex to implement \ud83d\udcbb<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
            \n\n\n\n

            \ud83c\udfaf TL;DR:<\/h3>\n\n\n\n

            Visual Transformers = Transformer + Images<\/strong><\/p>\n\n\n\n

            They slice images into patches \ud83e\udde9, treat them like words \ud83d\udcd6, and use attention to “understand” the image \ud83e\udde0\ud83d\udc41\ufe0f\u200d\ud83d\udde8\ufe0f \u2014 just like how Transformers understand sentences!<\/p>\n\n\n\n

            \n\n\n\n

            <\/p>\n","protected":false},"excerpt":{"rendered":"

            Let\u2019s break down Visual Transformers (also called Vision Transformers or ViTs) in a super simple and clear way\ud83e\udde0\ud83d\udc41\ufe0f\u200d\ud83d\udde8\ufe0f\ud83d\udcca \ud83e\udde0 What Are Visual Transformers? Visual Transformers are deep learning models that use the Transformer architecture (originally made for text \ud83d\udcc4) to analyze images \ud83d\uddbc\ufe0f instead of using CNNs (Convolutional Neural Networks). They were introduced in the…<\/p>\n","protected":false},"author":1,"featured_media":2350,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[641],"tags":[],"class_list":["post-2317","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-machine-learning"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t