Deep Learning (BIT423CO)
Neural network architectures — multilayer perceptrons, deep CNNs, RNNs, LSTMs, generative belief nets, and TensorFlow vision/speech applications.
Unit 1: Basics of Artificial Neural Networks4 hrs
- Biological neuron inspiration and artificial neuron models
- Perceptron model and perceptron learning algorithm
- Linear separability and the XOR problem
- Activation functions: Sigmoid, Tanh, ReLU, Leaky ReLU, Softmax
Unit 2: Feedforward Neural Networks and Backpropagation5 hrs
- Multi-layer perceptron (MLP) architecture
- Forward propagation matrix formulations
- Loss functions: Mean Squared Error, Cross-Entropy loss
- Backpropagation algorithm, chain rule derivations, and gradient descent optimization
Unit 3: Deep Neural Networks and Optimization8 hrs
- Vanishing and exploding gradient problems
- Optimization algorithms: Momentum, RMSProp, Adam, AdaGrad
- Regularization techniques: L1/L2 weight decay, Dropout, Early Stopping
- Batch Normalization, Layer Normalization, and hyperparameter tuning
Unit 4: Convolutional Neural Networks (CNNs)8 hrs
- Convolution operation, kernels, stride, padding, and feature maps
- Pooling layers: Max pooling, Average pooling
- CNN architectures: LeNet-5, AlexNet, VGGNet, ResNet, Inception
- Transfer learning and fine-tuning pretrained CNNs for image classification
Unit 5: Recurrent Neural Networks (RNNs)7 hrs
- Sequential data modeling and recurrent neuron architecture
- Backpropagation Through Time (BPTT)
- Long Short-Term Memory (LSTM) cell architecture and gates
- Gated Recurrent Unit (GRU) architecture
- Bidirectional RNNs and sequence-to-sequence architectures
Unit 6: Generative Models7 hrs
- Energy-based models and Restricted Boltzmann Machines (RBM)
- Deep Belief Networks (DBN) training and contrastive divergence
- Autoencoders and Variational Autoencoders (VAE)
- Generative Adversarial Networks (GAN) generator and discriminator dynamics
Unit 7: Applications in Vision, Speech and NLP6 hrs
- Object detection and image segmentation (YOLO, Mask R-CNN concepts)
- Acoustic modeling and speech recognition overview
- Word embeddings (Word2Vec, GloVe) and Transformer attention mechanisms in NLP
Reference books
- Goodfellow, Ian, Yoshua Bengio, and Aaron Courville, Deep Learning, MIT Press.
- Geron, Aurelien, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, O'Reilly Media.
- Chollet, Francois, Deep Learning with Python, Manning Publications.
- Zhang, Aston, Zachary C. Lipton, Mu Li, and Alexander J. Smola, Dive into Deep Learning, Cambridge University Press.