Asee peer logo

Assessing the Effectiveness of an Automated Problem Generator to Develop Course Content Rapidly and Minimize Student Cheating

Download Paper |


2020 ASEE Virtual Annual Conference Content Access


Virtual On line

Publication Date

June 22, 2020

Start Date

June 22, 2020

End Date

June 26, 2021

Conference Session

Mechanical Engineering Technical Session: Pedagogy I - Best Teaching Practices

Tagged Division

Mechanical Engineering

Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Philip Jackson University of Florida

visit author page

Dr. Philip B. Jackson earned B.S. degrees in Aerospace Engineering and Mechanical Engineering as well as an M.S. and Ph.D. in Mechanical Engineering, all from the University of Florida. He is currently a faculty member at the Institute for Excellence in Engineering Education at the University of Florida. There he specializes in implementing innovative methods of instruction in undergraduate courses on dynamics, heat transfer, and thermodynamics. His research interests include numerical heat transfer, fluids, and magnetohydrodynamic simulations and facilitating undergraduate students to engage in similar projects. He is also focused in the implementation of engineering freshman design experiences.

visit author page

author page

Ricker Lamphier University of Florida

Download Paper |


The education environment has taken a dramatic shift in the last decade with a greater focus in online delivery. In online and traditional classes alike, engineering faculty rely on textbooks, online publishers’ content, and problem repositories for class exercises as writing new problems every semester is prohibited. The problem with this is that both students and faculty have access to the same information on the web. With digital solution manuals or solution websites, the traditional delivery of homework has lost its integrity as the challenge of homework is no longer figuring out how to do the problem, it is now where does one find the online solution to a specific problem.

This project documents the development, implementation, and testing of a true random problem generator in Python to create original content quickly whose solutions are not ubiquitous or accessible via some online solution manual. With true random problem generation, the inputs define the problem as an archetype (where archetype is defined as a specific problem type (e.g. projectile motion, ideal gas piston cylinder, etc.)) and can have virtually an infinite set of problems generated from it. The script will read a user defined input file, randomize parameters, randomize values for those parameters, test for logical correctness (fidelity), and finally use a natural language algorithm to generate both problem text and solution.

To assess the effectiveness of problems created in this way, a survey was conducted having students rate the generator’s problems based on usefulness, clarity, and other qualitative features, as well as compare them to traditional, hand-written problems. A separate survey was conducted to score the accuracy of the generator’s difficulty heuristics by comparing the calculated difficulty of the software with the average of that perceived by the students. The assessment of the reduction of cheating was measured via blind survey and performance analysis. All-together, the functionality of this automated problem generator is established based on the qualitative measure of the aforementioned criteria, and overall perception of the potential future of online learning.

Jackson, P., & Lamphier, R. (2020, June), Assessing the Effectiveness of an Automated Problem Generator to Develop Course Content Rapidly and Minimize Student Cheating Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--34181

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2020 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015