Degree Level Course

Deep Learning

To study the basics of Neural Networks and their various variants such as the Convolutional Neural Networks and Recurrent Neural Networks, to study the different ways in which they can be used to solve problems in various domains such as Computer Vision, Speech and NLP.

Taught by Prof. Mitesh M.Khapra

Code BSCS3004

Credits 4 Credits

Type Core Option II

Prerequisites None

Core Competencies

What You'll Learn

View Course Videos

A brief history of deep learning and its success stories.
Perceptrons, Sigmoid neurons and Multi-Layer Perceptrons (MLP) with specific emphasis on their representation power and algorithms used for training them (such as Perceptron Learning Algorithm and Backpropagation).
Gradient Descent (GD) algorithm and its variants like Momentum based GD,AdaGrad, Adam etc Principal Component Analysis and its relation to modern Autoencoders.
The bias variance tradeoff and regularisation techniques used in DNNs (such as L2 regularisation, noisy data augmentation, dropout, etc).
Different activation functions and weight initialization strategies
Convolutional Neural Networks (CNNs) such as AlexNet, ZFNet, VGGNet, InceptionNet and ResNet.
Recurrent Neural Network (RNNs) and their variants such as LSTMs and GRUs (in particular, understanding the vanishing/exploding gradient problem and how LSTMs overcome the vanishing gradient problem)
Applications of CNN and RNN models for various computer vision and Natural Language Processing (NLP) problems.

12-Week Roadmap

Course Structure & Syllabus

For details of standard term assessment timelines and exam structures, visit our Academics page.

WEEK 1

History of Deep Learning, McCulloch Pitts Neuron, Thresholding Logic, Perceptron Learning Algorithm and Convergence

WEEK 2

Multilayer Perceptrons (MLPs), Representation Power of MLPs, Sigmoid Neurons, Gradient Descent

WEEK 3

Feedforward Neural Networks, Representation Power of Feedforward Neural Networks, Backpropagation

WEEK 4

Gradient Descent(GD), Momentum Based GD, Nesterov Accelerated GD, Stochastic GD, Adagrad, AdaDelta,RMSProp, Adam,AdaMax,NAdam, learning rate schedulers

Reading List

Prescribed Books & References

Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning. An MIT Press book. 2016.
Charu C. Aggarwal. Neural Networks and Deep Learning: A Textbook. Springer. 2019.

Faculty & Experts

About the Instructors

Prof. Mitesh M.Khapra

Associate Professor , Department of Computer Science and Engineering , IIT Madras

Mitesh M. Khapra is an Associate Professor in the Department of Computer Science and Engineering at IIT Madras and is affiliated with the Robert Bosch Centre for Data Science and AI. He is also a co-founder of One Fourth Labs, a startup whose mission is to design and deliver affordable hands-on courses on AI and related topics. He is also a co-founder of AI4Bharat, a voluntary community with an aim to provide AI-based solutions to India-specific problems. His research interests span the areas of Deep Learning, Multimodal Multilingual Processing, Natural Language Generation, Dialog systems, Question Answering and Indic Language Processing. Prior to IIT Madras, he was a Researcher at IBM Research India for four and a half years, where he worked on several interesting problems in the areas of Statistical Machine Translation, Cross Language Learning, Multimodal Learning, Argument Mining and Deep Learning. Prior to IBM, he completed his PhD and M.Tech from IIT Bombay in Jan 2012 and July 2008 respectively.During his PhD he was a recipient of the IBM PhD Fellowship (2011) and the Microsoft Rising Star Award (2011). He is also a recipient of the Google Faculty Research Award (2018), the IITM Young Faculty Recognition Award (2019) and the Prof. B. Yegnanarayana Award for Excellence in Research and Teaching (2020).

Other Courses by Instructor:

BSDA5013 • Deep Learning Practice

Back to Academics Overview View all Degree Level courses