Deep Learning @ VU

1. Introduction

• 1.1 Neural networks
• 1.2 Classification and regression
• 1.3 Autoencoders

2. Backpropagation

• 2.1 Scalar backpropagation
• 2.2 Tensor backpropagation
• 2.3 Automatic differentiation
• 2.4 Tensor details*

3. Convolutions

• 3.1 Introduction
• 3.2 Conv1D (a)
• 3.3 Conv1D (b)
• 3.3 Conv2D, Conv3D, ConvND

3. Tools of the trade

• 4.1 Deep learning in practice
• 4.2 Why does any of this work at all?
• 4.3 Understanding optimizers
• 4.4 The bag of tricks

5. Sequences

• 5.1 Learning from sequences
• 5.2 Recurrent neural networks
• 5.3 LSTMs and friends
• 5.4 CNNs for sequential data
• 5.5 ELMo, a case study

6. Latent Variable Models

• 6.1 Why Generative Modeling
• 6.2 Autoencoders
• 6.3 Variational Autoencoders

7. Unsupervised representation learning

• 7.0 Introduction
• 7.1 VAE Implementation
• 7.2 KL Divergence
• 7.3 MMD-VAE

8. Learning with graphs

• 8.1 Introduction - Graphs (1A)
• 8.2 Introduction - Embeddings (1B)
• 8.3 Graph Embedding Techniques
• 8.4 Graph Neural Networks
• 8.5 Query embedding

9. Transformers and self-attention

• 9.1 Self-attention
• 9.2 Transformers
• 9.3 Famous transformers
• 9.4 Scaling up

10. Reinforcement learning

11. Diffusion models

• 11.1 Naive diffusion
• 11.2 Understanding Gaussians
• 11.3 Gaussian diffusion

12. Generalization

• 12.0 Introduction
• 12.1 Review
• 12.2 Problem
• 12.3 Generalization Bound