Online Course Math497: Deep Learning Algorithms and Analysis
(Time: May 18 - Jun. 26, 2020)   


Click Here for detailed lecture videos and notes.

    Course Description


This is an undergraduate course on the introduction of basic mathematical, numerical and practical aspects of deep learning techniques. It will provide students with the mathematical background and also practical tools needed to understand, to analyze and to further develop numerical methods for deep learning and applications. The course is simultaneously geared towards math students who want to learn about the emerging technology of deep learning and also towards students from other fields who are interested in deep learning application but would like to strengthen their theoretical foundation and mathematical understanding. This course will allow students to fulfill 400-level math course requirement for Math Majors/Minors (or for other Majors as approved by student advisor).

The course will cover some basic deep learning models such as the basic deep neural networks, convolutional neural networks, training algorithms such as stochastic gradient descent methods, popular data bases such as MNIST and CIFAR and specific applications such as image classifications. Traditional numerical methods such as finite element and multigrid method will also be introduced to motivate and to understand how and why deep neural networks work. ore specifically, the following topics will be covered:

  • Basics of machine learning and probability
  • Logistic regression and SVM
  • Training algorithms in machine learning
  • Polynomial approximation and Weierstrass theorem
  • Linear finite element spaces
  • Deep neural networks and mathematical properties
  • Convolutional neural networks
  • Multigrid methods and MgNet
  • PyTorch and deep learning for image classification
  • Other applications

  • Class time will be split between lectures providing the necessary theoretical background and hands-on coding exercises for python programming for deep learning algorithms. Pre-recorded video lectures, copy of slides, typed lecture notes, live discussions and programming subroutines and exercises will be provided in an integrated fashion through Jupyter (see jupyter.org ) and Piazza. Both math and computing homework problems will be assigned. Research opportunities and possible supports will also be available to interested students.

    A similar course was also offerred in 2019 summer under the Education Abroad Program of Penn State, for details and student experiences, see 2019 Education Abroad at Peking University: Math497 . Unfortunately, the 2020 Education Abroad at Peking University: Math497 was cancelled due to the COVID-19 and students who originally enrolled for this Education Abroad Program are especially encouraged to take this course.

      Course Lecturer


  • Prof. Jinchao Xu (Penn State University)


    • Teaching Assistants


  • Dr. Juncai He (Penn State University)
  • Dr. Qingguo Hong (Penn State University)
  • Dr. Jonathan Siegel (Penn State Univeristy)
  • Dr. Lian Zhang (Penn State University)

    • Course Information, Prerequisites


  • Six weeks for 3 credits: May 18 - Jun. 26, 2020
  • Course time: pre-recorded video will be online and can be watched anytime and live discussion sections will be held 10am-11am Monday and Friday through Zoom: https://psu.zoom.us/j/91432583217
  • Place: Canvas and Zoom: https://psu.zoom.us/j/91432583217
  • Prerequisites:
    1. Linear algebra (MATH 220), multi-variable calculus (MATH 230 or 231).
  • 2. Some programming experiences are helpful. For those who are not familiar with Python, please follow this tutorial: [Video,
    Jupyter version , html version ]


    References


    1. Hastie T, Tibshirani R, Friedmann J. The Element of Statistical Learning[M]. Springer, 2001.
    2. Haykin S. Neural Networks: A Comprehensive Foundation (3rd Edition)[M]. Prentice-Hall, Inc. 2007.
    3. Goodfellow I, Bengio Y, Courville A. Deep learning[M]. MIT press, 2016. (URL: http://www.deeplearningbook.org)
    4. Xu J, Zikatanov L. Algebraic multigrid methods[J]. Acta Numerica, 2017, 26: 591-721(URL: https://doi.org/10.1017/S0962492917000083)
    5. Stanford Course: CS231n. (URL: http://cs231n.stanford.edu)