Measuring Revealed Student Scheduling Preferences using Constrained Discrete Choice Models

Jacob Bailey; Matthew West; Craig Zilles

Download Paper | Permalink

Conference: 2017 ASEE Annual Conference & Exposition
Location: Columbus, Ohio
Publication Date: June 24, 2017
Start Date: June 24, 2017
End Date: June 28, 2017
Conference Session: Potpourri
Tagged Division: Computing & Information Technology
Tagged Topic: Diversity
Page Count: 14
DOI: 10.18260/1-2--28656
Permanent URL: https://peer.asee.org/28656
Download Count: 588

Paper Authors

biography

Jacob Bailey University of Illinois, Urbana-Champaign

visit author page

Jacob Bailey is an graduate student with a focus on computer science education at the University of Illinois at Urbana-Champaign.

visit author page

biography

Matthew West University of Illinois, Urbana-Champaign orcid.org/0000-0002-7605-0050

visit author page

Matthew West is an Associate Professor in the Department of Mechanical Science and Engineering at the University of Illinois at Urbana-Champaign. Prior to joining Illinois he was on the faculties of the Department of Aeronautics and Astronautics at Stanford University and the Department of Mathematics at the University of California, Davis. Prof. West holds a Ph.D. in Control and Dynamical Systems from the California Institute of Technology and a B.Sc. in Pure and Applied Mathematics from the University of Western Australia. His research is in the field of scientific computing and numerical analysis, where he works on computational algorithms for simulating complex stochastic systems such as atmospheric aerosols and feedback control. Prof. West is the recipient of the NSF CAREER award and is a University of Illinois Distinguished Teacher-Scholar and College of Engineering Education Innovation Fellow.

visit author page

biography

Craig Zilles University of Illinois, Urbana-Champaign

visit author page

Craig Zilles is an Associate Professor in the Computer Science department at the University of Illinois at Urbana-Champaign. His current research focuses on computer science education and computer architecture. His research has been recognized by two best paper awards from ASPLOS (2010 and 2013) and by selection for inclusion in the IEEE Micro Top Picks from the 2007 Computer Architecture Conferences. He received the IEEE Education Society's Mac Van Valkenburg Early Career Teaching Award in 2010, a (campus-wise) Illinois Student Senate Teaching Excellence award in 2013, the NSF CAREER award, and the Univerisity of Illinois College of Engineering's Rose Award and Everitt Award for Teaching Excellence. Prior to his work on education and computer architecture, he developed the first algorithm that allowed rendering arbitrary three-dimensional polygonal shapes for haptic interfaces (force-feedback human-computer interfaces). He holds 6 patents.

visit author page

Download Paper | Permalink

Abstract

For constrained student resources with large student populations it is often necessary to implement some form of reservation or scheduling system. Examples of scheduled-access resources can include one-on-one tutoring, machine shops or labs, and computer-based testing facilities. For planning and resource scheduling purposes it is important to be able to forecast demand, and thus it is important to understand what drives student preferences for particular scheduling time slots. Measuring these preferences can be challenging, however, for at least the following three reasons. (1) Revealed preferences (what students actually choose) can differ significantly from stated preferences (what they say they will want at a future time), requiring the use of actual scheduling data to infer preferences or utilities. (2) The utility that students derive from particular choices is multifactorial, so that in a computer-based testing facility, for example, students may prefer to take their exam mid-afternoon, but they may also prefer to take it as close to the end of the exam period as possible, and it can be difficult to disentangle these factors. (3) Capacity constraints will frequently lead to many time slots being fully reserved, making it unclear which slots were actually preferred.

This paper presents a general framework for measuring revealed student preferences from actual reservation or scheduling data. This framework is based on the theory of constrained discrete choice modeling, as used in economics for modeling consumer preferences. A multifactorial random utility model (RUM) is formulated for student scheduling preferences and the model is trained on scheduling data using maximum likelihood estimation (MLE) and cross-validated on multiple rounds of training/test data splits.

Results are presented using scheduling data from a computer-based testing facility with approximately 50,000 student reservations over three semesters (Spring 2015 to Spring 2016, inclusive). We show that this measurement methodology can accurately capture student preferences in real-world scheduling data, and can successfully separate out time-in-week preferences from time-within-exam preferences. Errors are quantified using both log-likelihood with per-reservation data and root mean square error (RMSE) with data aggregated to the time slot level. We discuss both estimation and simulation algorithms for constrained discrete choice models and show how Monte Carlo simulation can be used to obtain uncertainty predictions for predicting expected usage.

Citation
Format

Bailey, J., & West, M., & Zilles, C. (2017, June), Measuring Revealed Student Scheduling Preferences using Constrained Discrete Choice Models Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. 10.18260/1-2--28656

TY  - CPAPER
AB  - For constrained student resources with large student populations it is often necessary to implement some form of reservation or scheduling system. Examples of scheduled-access resources can include one-on-one tutoring, machine shops or labs, and computer-based testing facilities. For planning and resource scheduling purposes it is important to be able to forecast demand, and thus it is important to understand what drives student preferences for particular scheduling time slots. Measuring these preferences can be challenging, however, for at least the following three reasons. (1) Revealed preferences (what students actually choose) can differ significantly from stated preferences (what they say they will want at a future time), requiring the use of actual scheduling data to infer preferences or utilities. (2) The utility that students derive from particular choices is multifactorial, so that in a computer-based testing facility, for example, students may prefer to take their exam mid-afternoon, but they may also prefer to take it as close to the end of the exam period as possible, and it can be difficult to disentangle these factors. (3) Capacity constraints will frequently lead to many time slots being fully reserved, making it unclear which slots were actually preferred.

This paper presents a general framework for measuring revealed student preferences from actual reservation or scheduling data. This framework is based on the theory of constrained discrete choice modeling, as used in economics for modeling consumer preferences. A multifactorial random utility model (RUM) is formulated for student scheduling preferences and the model is trained on scheduling data using maximum likelihood estimation (MLE) and cross-validated on multiple rounds of training/test data splits.

Results are presented using scheduling data from a computer-based testing facility with approximately 50,000 student reservations over three semesters (Spring 2015 to Spring 2016, inclusive). We show that this measurement methodology can accurately capture student preferences in real-world scheduling data, and can successfully separate out time-in-week preferences from time-within-exam preferences. Errors are quantified using both log-likelihood with per-reservation data and root mean square error (RMSE) with data aggregated to the time slot level. We discuss both estimation and simulation algorithms for constrained discrete choice models and show how Monte Carlo simulation can be used to obtain uncertainty predictions for predicting expected usage.

AU  - Jacob Bailey
AU  - Matthew West
AU  - Craig Zilles
CY  - Columbus, Ohio
DA  - 2017/06/24
PB  - ASEE Conferences
TI  - Measuring Revealed Student Scheduling Preferences using Constrained Discrete Choice Models
UR  - https://peer.asee.org/28656
DO  - 10.18260/1-2--28656
ER  -