To study the basics of Neural Networks and their various variants such as the Convolutional Neural Networks and Recurrent Neural Networks, to study the different ways in which they can be used to solve problems in various domains such as Computer Vision, Speech and NLP

Lectures-Tutorials-Practicals-Credits: 4-0-0-4

Week 1: Deep learning history; McCulloch-Pitts neuron; Threshold logic; Perceptron learning and convergence.

Week 2: Multilayer Perceptrons and their representation power; Sigmoid neurons; Gradient descent.

Week 3: Feedforward Neural Networks - representation power, backpropagation.

Week 4: Gradient descent recap; Contour plots; Momentum based and Nesterov accelerated GD; Adaptive optimization methods; Learning rate scheduling.

Week 5: Adaptive gradient methods; Adam and variants; NAdam; Learning rate schemes.

Week 6: Bias-variance; Training vs test error; Model complexity; Regularization; Noise injection; Early stopping; Ensembles; Dropout.

Week 7: Deep learning revival; Pre-training; Optimization vs regularization; Activation functions; Maxout; GELU to SiLU.

Week 8: CNNs; Convolution and architectures; ImageNet models; Visualisation techniques; Image generation; Deep Dream and style transfer; Adversarial examples.

Week 9: Word representations; SVD embeddings; CBOW and Skip-Gram; Optimization techniques; GloVe; Evaluation and Word2Vec.

Week 10: Sequence learning; Recurrent Neural Networks (RNN); Backpropagation Through Time (BPTT); Gradient issues; LSTM and GRU.

Week 11: Encoder-decoder models; Attention Mechanism; Transformers.

Week 12: Transformers; Self-attention; Multi-head attention; Decoder