Virtual On line
June 22, 2020
June 22, 2020
June 26, 2021
Mathematics Content of an Undergraduate Course on Deep Learning
Deep learning, aka neural networks, is a key AI technology that is increasingly important in science and engineering applications. Most of the undergraduate curriculum in science and engineering places emphasize on analytical models, often derived from first principles. In contrast, deep learning is based on empirical models learned from a large number of training examples. These empirical models are currently being used to solve challenging problems such as autonomous driving in unstructured environments or enabling robots to grasp arbitrary objects.
With the invention and wide spread use of computers, the applied mathematics curriculum has evolved to include courses like numerical analysis that supplement analytical modeling with simulation and numerical solutions to complex problems. Even introductory STEM courses such as calculus, differential equations, matrix algebra and statistics are turning to computer-based simulation and solutions. The bootstrap method, for example, has become a main topic in some introductory statistics courses. Wide spread adoption of deep learning will likely usher in another analogous evolution in the undergraduate mathematics curriculum.
Core mathematical concepts used in deep learning, such as linear functions, vector spaces, matrix arithmetic and the gradient vector, are familiar to engineering students. In this paper, the author summarizes his experiences teaching an undergraduate course in deep learning in a mathematics department.
The course begins by using the mean and the median of a set of numbers as the simplest examples of a neural network. By the middle of the course, many key concepts such as stochastic gradient descent, over vs underfitting and regularization, the softmax function and cross-entropy, have been covered using only linear and logistic regression. At this point students begin working on a course project. They have sufficient background information to begin using transfer learning and data augmentation to address challenging problems. Transfer learning involves re-purposing a pre-trained neural network to generate input features that can significantly improve the performance of linear and logistic regression. In the second half of the course, dense networks are introduced as a natural extension of linear and logistic regression. More challenging topics like convolutional neural networks and back propagation are also covered.
Shibberu, Y. (2020, June), Mathematics Content of an Undergraduate Course on Deep Learning Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--34957
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2020 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015