Asee peer logo

The Rise of Program Auto-grading in Introductory CS Courses: A Case Study of zyLabs

Download Paper |


2021 ASEE Virtual Annual Conference Content Access


Virtual Conference

Publication Date

July 26, 2021

Start Date

July 26, 2021

End Date

July 19, 2022

Conference Session

Computers in Education 9 - Technology 1

Tagged Division

Computers in Education

Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Chelsea L. Gordon zyBooks, A Wiley Brand

visit author page

Chelsea Gordon received her PhD in Cognitive Science at University of California, Merced in 2019. Chelsea works as a research scientist for zyBooks, a Wiley company that creates and publishes interactive, web-native textbooks in STEM.

visit author page


Roman Lysecky University of Arizona; zyBooks, A Wiley Brand

visit author page

Roman Lysecky is VP of Content at zyBooks, A Wiley Brand and a Professor of Electrical and Computer Engineering at the University of Arizona. He received his Ph.D. in Computer Science from the University of California, Riverside in 2005. His research focuses on embedded systems, cybersecurity, and STEM education. He has authored more than 100 research publications, received nine Best Paper Awards, is an inventor on multiple patents, and received multiple awards for Excellence at the Student Interface.

visit author page


Frank Vahid University of California, Riverside

visit author page

Frank Vahid is a Professor of Computer Science and Engineering at the Univ. of California, Riverside, and co-founder and chief learning officer of zyBooks.

visit author page

Download Paper |


In recent years, hundreds of college courses have switched how they grade programming assignments, from grading manually and/or using batch scripts, to using commercial cloud-based auto-graders with immediate score feedback to students and the ability to debug and resubmit for a higher score. This paper provides data on the rise in usage of one of the most widely-used program auto-graders, zyLabs, as one indicator of the strong shift in college course grading to the auto-grading paradigm. The number of courses, instructors, and students using zyLabs have increased dramatically since it was first introduced, such that from 2016 to 2020, the number of courses per year grew from 284 to 2,175, the number of students per year from 24,216 to 132,121, and the number of instructors per year from 364 to 2,866. Most instructors state they previously graded programs by hand and auto-grading saved an average of 9 hours per week. The result is a substantial shift in the classroom dynamic that enables instructors and students to spend more time on quality teaching and learning.

Gordon, C. L., & Lysecky, R., & Vahid, F. (2021, July), The Rise of Program Auto-grading in Introductory CS Courses: A Case Study of zyLabs Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--37887

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015