This is an undergraduate course on the introduction of basic mathematical, numerical and practical aspects of deep learning techniques. It will provide students with the mathematical background and also practical tools needed to understand, to analyze and to further develop numerical methods for deep learning and applications. The course is simultaneously geared towards math students who want to learn about the emerging technology of deep learning and also towards students from other fields who are interested in deep learning application but would like to strengthen their theoretical foundation and mathematical understanding. This course will allow students to fulfill 400-level math course requirement for Math Majors/Minors (or for other Majors as approved by student advisor).
The course will cover some basic deep learning models such as the basic deep neural networks, convolutional neural networks, training algorithms such as stochastic gradient descent methods, popular data bases such as MNIST and CIFAR and specific applications such as image classifications. Traditional numerical methods such as finite element and multigrid method will also be introduced to motivate and to understand how and why deep neural networks work. ore specifically, the following topics will be covered:
Class time will be split between lectures providing the necessary theoretical background and hands-on coding exercises for python programming for deep learning algorithms. Pre-recorded video lectures, copy of slides, typed lecture notes, live discussions and programming subroutines and exercises will be provided in an integrated fashion through Jupyter (see jupyter.org ) and Piazza. Both math and computing homework problems will be assigned. Research opportunities and possible supports will also be available to interested students.
A similar course was also offerred in 2019 summer under the Education Abroad Program of Penn State, for details and student experiences, see 2019 Education Abroad at Peking University: Math497 . Unfortunately, the 2020 Education Abroad at Peking University: Math497 was cancelled due to the COVID-19 and students who originally enrolled for this Education Abroad Program are especially encouraged to take this course.
Course Information, Prerequisites
1. Linear algebra (MATH 220), multi-variable calculus (MATH 230 or 231).
- Hastie T, Tibshirani R, Friedmann J. The Element of Statistical Learning[M]. Springer, 2001.
- Haykin S. Neural Networks: A Comprehensive Foundation (3rd Edition)[M]. Prentice-Hall, Inc. 2007.
- Goodfellow I, Bengio Y, Courville A. Deep learning[M]. MIT press, 2016. (URL: http://www.deeplearningbook.org)
- Xu J, Zikatanov L. Algebraic multigrid methods[J]. Acta Numerica, 2017, 26: 591-721(URL: https://doi.org/10.1017/S0962492917000083)
- Stanford Course: CS231n. (URL: http://cs231n.stanford.edu)