2024: Introduction to Deep Learning
CS551: Introduction to Deep Learning (Spring 2024)
|
|
This course will provide a basic understanding of deep learning and how to solve problems from varied domains. Open source tools will be used to demonstrate different applications.
|
Class schedule
|
|
Tuesday — 1500-1600; Wednesday — 1600-1700; Thursday — 1700-1800;
|
|
Venue — R410;
|
Instructors & TAs
|
|
Instructors
|
TAs
|
Syllabus
|
|
Brief introduction of big data problem. Overview of linear algebra, probability, numerical computation. Basics of Machine learning/Feature engineering. Neural network. Tutorial for Tools. Deep learning network - Shallow vs Deep network, Deep feedforward network, Gradient based learning - Cost function, soft max, sigmoid function, Hidden unit - ReLU, Logistic sigmoid, hyperbolic tangent Architecture design, SGD, Unsupervised learning - Deep Belief Network, Deep Boltzmann Machine, Factor analysis, Autoencoders. Regularization. Optimization for training deep model. Advanced topics - Convolutional Neural Network, Recurrent Neural Network/ Sequence modeling, LSTM, Reinforcement learning. Practical applications – Vision, speech, NLP, etc.
|
Books
|
- Ian Goodfellow, Yoshua Bengio and Aaron Courville, “Deep Learning”, Book in preparation for MIT Press, 2016. (available online)
- Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie, “The elements of statistical learning”, Springer Series in Statistics, 2009.
- Charu C Aggarwal, “Neural Networks and Deep Learning”, Springer.
- Aston Zhang, Zachary C. Lipton, Mu Li, Alexander J. Smola, "Dive into Deep Learning" (avilable online)
- Iddo Drori, "The Science of Deep Learning", Cambridge University Press
- Simon O. Haykin, "Neural Networks and Learning Machines", Pearson Education India
- Richard S. Sutton, Andrew G. Barto, "Reinforcement Learning: An Introduction", MIT Press
- Christopher M. Bishop, Hugh Bishop, "Deep Learning: Foundations and Concepts", Springer
- Simon J. D. Prince, "Understanding Deep Learning", MIT Press
|
Slides
|
| Topic |
Slides |
Lecture delivered by |
|
|
Introduction
|
pdf
|
AM |
|
Neural networks-I
|
pdf
|
AM |
|
Neural networks-II
|
pdf
|
AM |
|
Deep feedforward network
|
pdf
|
JC |
|
Backpropagation
|
pdf
|
JC |
|
Regularization
|
pdf
|
JC |
|
Optimization
|
pdf
|
AM |
|
CNN
|
pdf
|
AM |
|
RNN
|
pdf
|
JC |
|
Transformer
|
pdf
|
AM |
|
Practical methodology
|
pdf
|
AM |
|
Autoencoders
|
pdf
|
JC |
|