NLP-CV.AJ1 ISBN: 979-8-90059-025-7
Transformers for Natural Language Processing and Computer Vision
Bridge the gap between language and sight! Master the state-of-the-art Transformer Models that are currently redefining the boundaries of AI.
What you will be able to do
- Architectural Mastery: Understand the core mechanics of Transformer Models, including positional encoding, multi-head attention, and the evolution from CNNs/RNNs to the modern Encoder-Decoder stack.
- NLP & Semantic Analysis: Master BERT and GPT-4 for tasks like SRL (Semantic Role Labeling), text summarization with T5, and complex NLU benchmarks like GLUE and SuperGLUE.
- Vision & Multimodal AI: Implement the Vision Transformer (ViT), explore CLIP for image-text pairing, and master the multi-phase workflow of Stable Diffusion for creative content generation.
- Production & Risk Management: Learn Fine-tuning Transformers for custom datasets, implement RAG to mitigate hallucinations, and establish robust guardrails for Ethical AI and risk management.
Intermediate Self-paced · 1 year access
27 Hands-On LiveLabs
Practice real IT tasks in guided environments.
- Real environments
- Auto-graded
- No installation
01 / About
About This Course
Are you ready to stop treating text and images as separate worlds? The "Transformer Revolution" has unified the way we build intelligence. This course is your deep-dive into the most powerful architecture in AI history—the Transformer.
We move beyond the basics of BERT and GPT-4 to show you how the same fundamental principles power both human-like text generation and superhuman computer vision. You’ll explore the Transformer ecosystem, mastering everything from the initial Encoder-Decoder stack to advanced Vision Transformer (ViT) implementations.
Whether you’re building a multi-modal assistant, fine-tuning an LLM using RAG (Retrieval-Augmented Generation), or generating photorealistic art with Stable Diffusion, this course provides the blueprint. We don't just talk about theory; we dive into 30+ virtual labs where you’ll perform Fine-tuning Transformers for real-world production. This is your chance to master Generative AI across both pixels and prose.
02 / Lessons & labs
See exactly what you will learn and practice
Lessons
21 Interactive Lessons · 135 topics01 Preface 2 topics +
- Who this course is for
- What this course covers
02 What are Transformers? 6 topics · 1 LiveLab +
- Foundation Models
- A brief history of how transformers were born
- The new role of AI professionals
- The rise of seamless transformer APIs
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
03 Getting Started with the Architecture of the Transformer Model 5 topics · 2 LiveLab +
- The rise of the Transformer: Attention Is All You Need
- Training and performance
- Hugging Face transformer models
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
04 Emergent vs Downstream Tasks: The Unseen Depths of Transformers 5 topics · 2 LiveLab +
- The paradigm shift: What is an NLP task?
- Investigating the potential of downstream tasks
- Running downstream tasks
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
05 Advancements in Translations with Google Trax, Google Translate, and Gemini 7 topics · 1 LiveLab +
- Defining machine translation
- Evaluating machine translations
- Translations with Google Trax
- Translation with Google Translate
- Translation with Gemini
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
06 Diving into Fine-Tuning through BERT 5 topics · 1 LiveLab +
- The architecture of BERT
- Fine-tuning BERT
- Building a Python interface to interact with the model
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
07 Pretraining a Transformer from Scratch through RoBERTa 6 topics · 2 LiveLab +
- Training a tokenizer and pretraining a transformer
- Building KantaiBERT from scratch
- Pretraining a Generative AI customer support model on X data
- Next steps
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
08 The Generative AI Revolution with ChatGPT 7 topics · 3 LiveLab +
- GPTs as GPTs
- The architecture of OpenAI GPT transformer models
- OpenAI models as assistants
- Getting started with the GPT-4 API
- Retrieval Augmented Generation (RAG) with GPT-4
- Summary
- References
3 LiveLab in this lesson — see the labs panel →
09 Fine-Tuning OpenAI GPT Models 9 topics +
- Risk management
- Fine-tuning a GPT model for completion (generative)
- Preparing the dataset
- Fine-tuning an original model
- Running the fine-tuned GPT model
- Managing fine-tuned jobs and models
- Before leaving
- Summary
- References
10 Shattering the Black Box with Interpretable Tools 6 topics · 2 LiveLab +
- Transformer visualization with BertViz
- Interpreting Hugging Face transformers with SHAP
- Transformer visualization via dictionary learning
- Other interpretable AI tools
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
11 Investigating the Role of Tokenizers in Shaping Transformer Models 4 topics · 1 LiveLab +
- Matching datasets and tokenizers
- Exploring sentence and WordPiece tokenizers to u...fficiency of subword tokenizers for transformers
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
12 Leveraging LLM Embeddings as an Alternative to Fine-Tuning 6 topics · 2 LiveLab +
- LLM embeddings as an alternative to fine-tuning
- Fundamentals of text embedding with NLTK and Gensim
- Implementing question-answering systems with embedding-based search techniques
- Transfer learning with Ada embeddings
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
13 Toward Syntax-Free Semantic Role Labeling with ChatGPT and GPT-4 9 topics · 1 LiveLab +
- Getting started with cutting-edge SRL
- Entering the syntax-free world of AI
- Defining SRL
- SRL experiments with ChatGPT with GPT-4
- Questioning the scope of SRL
- Redefining SRL
- From task-specific SRL to emergence with ChatGPT
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
14 Summarization with T5 and ChatGPT 8 topics · 1 LiveLab +
- Designing a universal text-to-text model
- The rise of text-to-text transformer models
- A prefix instead of task-specific formats
- The T5 model
- Text summarization with T5
- From text-to-text to new word predictions with OpenAI ChatGPT
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
15 Exploring Cutting-Edge LLMs with Vertex AI and PaLM 2 6 topics +
- Architecture
- Assistants
- Vertex AI PaLM 2 API
- Fine-tuning
- Summary
- References
16 Guarding the Giants: Mitigating Risks in Large Language Models 9 topics · 3 LiveLab +
- The emergence of functional AGI
- Cutting-edge platform installation limitations
- Auto-BIG-bench
- WandB
- When will AI agents replicate?
- Risk management
- Risk mitigation tools with RLHF and RAG
- Summary
- References
3 LiveLab in this lesson — see the labs panel →
17 Beyond Text: Vision Transformers in the Dawn of Revolutionary AI 7 topics · 2 LiveLab +
- From task-agnostic models to multimodal vision transformers
- ViT – Vision Transformer
- CLIP
- DALL-E 2 and DALL-E 3
- GPT-4V, DALL-E 3, and divergent semantic association
- Summary
- References
2 LiveLab in this lesson — see the labs panel →
18 Transcending the Image-Text Boundary with Stable Diffusion 6 topics · 1 LiveLab +
- Transcending image generation boundaries
- Part I: Defining text-to-image with Stable Diffusion
- Part II: Running text-to-image with Stable Diffusion
- Part III: Video
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
19 Hugging Face AutoTrain: Training Vision Models without Coding 8 topics · 1 LiveLab +
- Goal and scope of this lesson
- Getting started
- Uploading the dataset
- Training models with AutoTrain
- Deploying a model
- Running our models for inference
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
20 On the Road to Functional AGI with HuggingGPT and its Peers 8 topics · 1 LiveLab +
- Defining F-AGI
- Installing and importing
- Validation set
- HuggingGPT
- CustomGPT
- Model Chaining with Runway Gen-2
- Summary
- References
1 LiveLab in this lesson — see the labs panel →
21 Beyond Human-Designed Prompts with Generative Ideation 6 topics +
- Part I: Defining generative ideation
- Part II: Automating prompt design for generative image design
- Part III: Automated generative ideation with Stable Diffusion
- The future is yours!
- Summary
- References
Hands-On Labs Our edge
27 LiveLabs- Training, Evaluating, and Visualizing a Machine Learning Classifier
- Implementing Multi-Head Attention and Post-Layer Normalization
- Exploring Positional Encoding in Transformer Models
- Visualizing Decision Boundaries with k-NN Using 1000 Random Samples
- Running Downstream Transformer Tasks
- Preprocessing the WMT14 French-English Dataset and Evaluating with BLEU
- Fine-Tuning BERT for Sentence Classification Using the CoLA Dataset
- Building and Training KantaiBERT for Token Classification
- Building a Customer-Support Assistant Using a Transformer Model
- Analyzing GPT Transformer Architecture and OpenAI Model APIs
- Getting Started with OpenAI GPT-4 for NLP Tasks
- Implementing RAG Using GPT-4
- Visualizing Transformer Attention with BertViz
- Interpreting Transformer Predictions Using SHAP
- Exploring Tokenizers in Modern NLP Using HuggingFace
- Building Word Embeddings Using NLTK and Gensim
- Building an Embedding-Based Question-Answering and Transfer-Learning Pipeline
- Performing Zero-Shot SRL Using GPT-4 Via Prompting
- Building and Evaluating Text Summarization Systems
- Evaluating Auto-BIG-bench Tasks
- Evaluating and Mitigating Hallucination in RAG Systems
- Mitigating Risks in Generative AI Systems
- Exploring Vision-Language Models with CLIP and ViT
- Generating and Interpreting AI-Driven Visual Content Using GPT-4V and DALL·E
- Generating Images with Stable Diffusion Using Keras
- Training NLP Models Automatically with Hugging Face AutoTrain
- Analyzing Images Using ViT Models
03 / FAQs
Questions before you start
What makes Transformers better than CNNs for Computer Vision?+
Will I learn about RAG (Retrieval-Augmented Generation)?+
How difficult is Fine-tuning Transformers in this course?+
Does this course cover image generation?+
Keep going
You might also like
Ready to Build the Future of Multimodal AI?
The line between text and vision is disappearing. Start your journey to becoming a lead AI architect and master Transformers for NLP and CV to stay ahead in the rapidly evolving Generative AI landscape.
- 1 year of full access
- 27 LiveLab included
- Certificate of completion
No credit card required