Run this notebook: Open in Colab Open in Kaggle

Phase 24: Advanced Deep Learning — Start Here

Master the research-level techniques behind modern generative AI — GANs, diffusion models, transformers, meta-learning, and beyond.

What’s in This Phase

38 notebooks covering the mathematical foundations and implementations of state-of-the-art deep learning architectures.

Tracks

Generative Models

Notebook	Topic
01_gan_mathematics.ipynb	GAN theory: minimax game, JS divergence
02_wgan_theory_implementation.ipynb	Wasserstein GAN — stable training
05_info_gan.ipynb	InfoGAN — disentangled representations
08_conditional_gan.ipynb	cGAN — controlled generation
19_stylegan.ipynb	StyleGAN2 — state-of-the-art face generation
24_cyclegan.ipynb	CycleGAN — unpaired image translation
25_progressive_gan.ipynb	Progressive growing for high-res images
11_diffusion_models.ipynb	DDPM — the math behind Stable Diffusion
33_score_based_models.ipynb	Score matching and stochastic processes
34_energy_based_models.ipynb	EBM — unnormalized density estimation

Variational Methods

Notebook	Topic
03_variational_autoencoders_advanced.ipynb	VAE theory: ELBO, reparameterization
07_hierarchical_vae.ipynb	NVAE, VDVAE — multi-scale latents
09_vq_vae.ipynb	VQ-VAE — discrete latent spaces
10_normalizing_flows.ipynb	Invertible transforms, exact likelihoods

Transformer Architectures

Notebook	Topic
06_vision_transformers.ipynb	ViT — images as sequences of patches
12_bert_architecture.ipynb	BERT internals and pre-training
13_gpt_architecture.ipynb	GPT — autoregressive transformer
14_efficient_transformers.ipynb	Longformer, Linformer, FlashAttention
30_attention_mechanisms.ipynb	Multi-head attention deep dive
35_mixture_of_experts.ipynb	MoE — how GPT-4 scales

Meta & Continual Learning

Notebook	Topic
16_maml_meta_learning.ipynb	MAML — learn to learn
17_prototypical_networks.ipynb	Few-shot learning
27_curriculum_learning.ipynb	Train on easy examples first
28_continual_learning.ipynb	Overcome catastrophic forgetting
29_neural_architecture_search.ipynb	AutoML for architectures

Other Advanced Topics

Notebook	Topic
04_neural_ode.ipynb	Neural ODEs — continuous-depth networks
15_graph_neural_networks.ipynb	GNN — learning on graph-structured data
18_neural_radiance_fields.ipynb	NeRF — 3D scene representation
20_contrastive_learning.ipynb	SimCLR, MoCo — self-supervised vision
21_adversarial_robustness.ipynb	PGD attacks, adversarial training
22_knowledge_distillation.ipynb	Compress big models into small ones
23_point_cloud_networks.ipynb	PointNet for 3D data
26_model_interpretability.ipynb	SHAP, LIME, attention visualization
31_memory_networks.ipynb	External memory and attention
32_capsule_networks.ipynb	Hinton’s capsules
36_implicit_neural_representations.ipynb	SIREN, NeRF-style networks
37_gaussian_processes.ipynb	GP — probabilistic function estimation
38_bayesian_neural_networks.ipynb	Uncertainty-aware deep learning

Prerequisites

• Neural Networks (Phase 06)

• Strong calculus and linear algebra (Phase 03)

• PyTorch fundamentals

Recommended Starting Points

For generative AI: 11_diffusion_models.ipynb → 01_gan_mathematics.ipynb

For LLM internals: 12_bert_architecture.ipynb → 13_gpt_architecture.ipynb

For probabilistic ML: 03_variational_autoencoders_advanced.ipynb → 37_gaussian_processes.ipynb