Mathematics Content of an Undergraduate Course on Deep Learning

Yosi Shibberu

Download Paper | Permalink

Conference: 2020 ASEE Virtual Annual Conference Content Access
Location: Virtual On line
Publication Date: June 22, 2020
Start Date: June 22, 2020
End Date: June 26, 2021
Conference Session: Mathematics Division Technical Session 2: Poster Presentations
Tagged Division: Mathematics
Tagged Topic: Diversity
Page Count: 9
DOI: 10.18260/1-2--34957
Permanent URL: https://peer.asee.org/34957
Download Count: 578

Paper Authors

biography

Yosi Shibberu Rose-Hulman Institute of Technology

visit author page

Dr. Yosi Shibberu is professor of mathematics at Rose-Hulman Institute of Technology. He has taught undergraduate courses on data mining, machine learning, bioinformatics and computational biology. Dr. Shibberu spent a year at Jimma University, Ethiopia, as a Fulbright Scholar and formerly held the endowed chair for innovation in science, engineering and mathematics education at Rose-Hulman Institute of Technology.

visit author page

Download Paper | Permalink

Abstract

Mathematics Content of an Undergraduate Course on Deep Learning

Deep learning, aka neural networks, is a key AI technology that is increasingly important in science and engineering applications. Most of the undergraduate curriculum in science and engineering places emphasize on analytical models, often derived from first principles. In contrast, deep learning is based on empirical models learned from a large number of training examples. These empirical models are currently being used to solve challenging problems such as autonomous driving in unstructured environments or enabling robots to grasp arbitrary objects.

With the invention and wide spread use of computers, the applied mathematics curriculum has evolved to include courses like numerical analysis that supplement analytical modeling with simulation and numerical solutions to complex problems. Even introductory STEM courses such as calculus, differential equations, matrix algebra and statistics are turning to computer-based simulation and solutions. The bootstrap method, for example, has become a main topic in some introductory statistics courses. Wide spread adoption of deep learning will likely usher in another analogous evolution in the undergraduate mathematics curriculum.

Core mathematical concepts used in deep learning, such as linear functions, vector spaces, matrix arithmetic and the gradient vector, are familiar to engineering students. In this paper, the author summarizes his experiences teaching an undergraduate course in deep learning in a mathematics department.

The course begins by using the mean and the median of a set of numbers as the simplest examples of a neural network. By the middle of the course, many key concepts such as stochastic gradient descent, over vs underfitting and regularization, the softmax function and cross-entropy, have been covered using only linear and logistic regression. At this point students begin working on a course project. They have sufficient background information to begin using transfer learning and data augmentation to address challenging problems. Transfer learning involves re-purposing a pre-trained neural network to generate input features that can significantly improve the performance of linear and logistic regression. In the second half of the course, dense networks are introduced as a natural extension of linear and logistic regression. More challenging topics like convolutional neural networks and back propagation are also covered.

Citation
Format

Shibberu, Y. (2020, June), Mathematics Content of an Undergraduate Course on Deep Learning Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--34957

TY - CPAPER
AB - Mathematics Content of an Undergraduate Course on Deep Learning

Deep learning, aka neural networks, is a key AI technology that is increasingly important in science and engineering applications. Most of the undergraduate curriculum in science and engineering places emphasize on analytical models, often derived from first principles. In contrast, deep learning is based on empirical models learned from a large number of training examples. These empirical models are currently being used to solve challenging problems such as autonomous driving in unstructured environments or enabling robots to grasp arbitrary objects.

With the invention and wide spread use of computers, the applied mathematics curriculum has evolved to include courses like numerical analysis that supplement analytical modeling with simulation and numerical solutions to complex problems. Even introductory STEM courses such as calculus, differential equations, matrix algebra and statistics are turning to computer-based simulation and solutions. The bootstrap method, for example, has become a main topic in some introductory statistics courses. Wide spread adoption of deep learning will likely usher in another analogous evolution in the undergraduate mathematics curriculum.

Core mathematical concepts used in deep learning, such as linear functions, vector spaces, matrix arithmetic and the gradient vector, are familiar to engineering students. In this paper, the author summarizes his experiences teaching an undergraduate course in deep learning in a mathematics department.

The course begins by using the mean and the median of a set of numbers as the simplest examples of a neural network. By the middle of the course, many key concepts such as stochastic gradient descent, over vs underfitting and regularization, the softmax function and cross-entropy, have been covered using only linear and logistic regression. At this point students begin working on a course project. They have sufficient background information to begin using transfer learning and data augmentation to address challenging problems. Transfer learning involves re-purposing a pre-trained neural network to generate input features that can significantly improve the performance of linear and logistic regression. In the second half of the course, dense networks are introduced as a natural extension of linear and logistic regression. More challenging topics like convolutional neural networks and back propagation are also covered.

AU - Yosi Shibberu
CY - Virtual On line
DA - 2020/06/22
PB - ASEE Conferences
TI - Mathematics Content of an Undergraduate Course on Deep Learning
UR - https://peer.asee.org/34957
DO - 10.18260/1-2--34957
ER -