Applications in Computer Vision, NLP & Transformer Self-Attention
Idea in plain words
Computer Vision Applications: Object Detection (YOLO - You Only Look Once, Faster R-CNN) and Semantic Segmentation (U-Net).
Exam tip
High-yield exam topic (Very High).
Explanation
Transformers revolutionized natural language processing by overcoming the sequential processing bottleneck of RNNs. Because self-attention calculates pairwise token interactions across the entire sentence in parallel matrix multiplications, training scales efficiently on GPUs. Queries (Q), Keys (K), and Values (V) are projected into multiple attention heads, enabling the model to jointly attend to information from different representation subspaces. Vision Transformers (ViT) extend this paradigm to computer vision by treating 16x16 image patches as word tokens.
Key points
- YOLO frames object detection as a single regression problem, predicting bounding boxes and class probabilities simultaneously in real time.
- Natural Language Processing: Sequence-to-sequence models, Machine Translation, and Speech-to-Text (Whisper).
- Transformer Architecture (Vaswani et al., 2017): Eliminates recurrence entirely in favor of parallel Self-Attention mechanisms.
- Scaled Dot-Product Attention: Attention(Q, K, V) = softmax((Q * K^T) / sqrt(d_k)) * V.
- Foundation Models: Pre-trained Large Language Models (LLMs) like GPT and BERT fine-tuned for downstream NLP tasks.
Example
Example
In the sentence 'The bank of the river overflowed', self-attention links 'bank' with 'river' rather than financial institutions, correctly resolving word sense disambiguation.
Code examples
/*
Scaled Dot-Product Self-Attention Formula:
Attention(Q, K, V) = Softmax( (Q * K^T) / sqrt(d_k) ) * V
Where:
- Q (Query): What the current word is looking for
- K (Key): What each word in the sequence offers
- V (Value): The actual information content
- sqrt(d_k): Scaling factor to prevent dot products from growing excessively large
*/
Exam questions
- [10 Marks] Explain the Scaled Dot-Product Attention mechanism in the Transformer architecture. Detail Queries, Keys, Values, and the role of the scaling factor sqrt(d_k).
- [8 Marks] Explain the working of real-time object detection using YOLO (You Only Look Once). How does it differ from two-stage detectors like Faster R-CNN?