Theory is not enough, you must apply. The objective of this workshop is to give you hands on experience in building models using the pytorch's core component called Tensor . Beleive me, everything you are gonna build is simply stacking or connecting copies of this single core component!. It make sense as all the models takes in Tensors (data) as input and manipulate them and finally outputs a Tensor. In this workshop we start with very foundational concepts like tensors and computing gradient of tensors. I try my best to correlate the theory of back-proagation for fully conncected layers, convolutional layers, recurrent layers with the pytorch implementation (which uses AutoGrad).
-
You can access the slide deck here: https://iitm-pod.slides.com/arunprakash_ai/pytorch
-
Read the contents in the slide deck before using the following colab notebooks.
-
I strongly believe that once you get a good grip on the first four modules, you can refer to documenetation or others code easily. I will keep updating this post.
Colab Notebooks
-
The Fuel: Tensors
- Understand the Pytorch architecture
- Create Tensors of 0d,1d,2d,3d,... (a multidomensional array in numpy)
- Understand the attributes : storage, stride, offset, device`
- Manipulate tensor dimensions
- Operations on tensors
-
The Engine: Autograd
- A few more attributes of tensor : requires_grad, grad, grad_fn, _saved_tensors, backward, retain_grad, zero_grad
- Computation gradph: Leaf node (parameters) vs non-leaf node (intermediate computation)
- Accumulate gradient and update with context manager (torch.no_grad)
- Implementating a neural network from scratch
-
The factory: nn.Module , Data utils
- Brief tour into the source code of nn.Module
- Everything is a module (layer in other frameworks)
- Stack modules by subclassing nn.Module and build any neural network
- Managing data with dataset class and DataLoader class
-
Convolutional Neural Netowrk Image Classification
- Using torchvision for datasets
- build CNN and move it to GPU
- Train and test
- Transfer learning
- Image segmentation
-
Recurrent Neural Netowrk Sequence Classification
- torchdata
- torchtext
- Word Embedding
- Build RNN
- Train,Test,Evaluate