Assessing the Effectiveness of an Automated Problem Generator to Develop Course Content Rapidly and Minimize Student Cheating

Philip Jackson; Ricker Lamphier

Download Paper | Permalink

Conference: 2020 ASEE Virtual Annual Conference Content Access
Location: Virtual On line
Publication Date: June 22, 2020
Start Date: June 22, 2020
End Date: June 26, 2021
Conference Session: Mechanical Engineering Technical Session: Pedagogy I - Best Teaching Practices
Tagged Division: Mechanical Engineering
Page Count: 21
DOI: 10.18260/1-2--34181
Permanent URL: https://peer.asee.org/34181
Download Count: 426

Request a correction

Paper Authors

biography

Philip Jackson University of Florida

visit author page

Dr. Philip B. Jackson earned B.S. degrees in Aerospace Engineering and Mechanical Engineering as well as an M.S. and Ph.D. in Mechanical Engineering, all from the University of Florida. He is currently a faculty member at the Institute for Excellence in Engineering Education at the University of Florida. There he specializes in implementing innovative methods of instruction in undergraduate courses on dynamics, heat transfer, and thermodynamics. His research interests include numerical heat transfer, fluids, and magnetohydrodynamic simulations and facilitating undergraduate students to engage in similar projects. He is also focused in the implementation of engineering freshman design experiences.

visit author page

author page

Ricker Lamphier University of Florida

Download Paper | Permalink

Abstract

The education environment has taken a dramatic shift in the last decade with a greater focus in online delivery. In online and traditional classes alike, engineering faculty rely on textbooks, online publishers’ content, and problem repositories for class exercises as writing new problems every semester is prohibited. The problem with this is that both students and faculty have access to the same information on the web. With digital solution manuals or solution websites, the traditional delivery of homework has lost its integrity as the challenge of homework is no longer figuring out how to do the problem, it is now where does one find the online solution to a specific problem.

This project documents the development, implementation, and testing of a true random problem generator in Python to create original content quickly whose solutions are not ubiquitous or accessible via some online solution manual. With true random problem generation, the inputs define the problem as an archetype (where archetype is defined as a specific problem type (e.g. projectile motion, ideal gas piston cylinder, etc.)) and can have virtually an infinite set of problems generated from it. The script will read a user defined input file, randomize parameters, randomize values for those parameters, test for logical correctness (fidelity), and finally use a natural language algorithm to generate both problem text and solution.

To assess the effectiveness of problems created in this way, a survey was conducted having students rate the generator’s problems based on usefulness, clarity, and other qualitative features, as well as compare them to traditional, hand-written problems. A separate survey was conducted to score the accuracy of the generator’s difficulty heuristics by comparing the calculated difficulty of the software with the average of that perceived by the students. The assessment of the reduction of cheating was measured via blind survey and performance analysis. All-together, the functionality of this automated problem generator is established based on the qualitative measure of the aforementioned criteria, and overall perception of the potential future of online learning.

Citation
Format

Jackson, P., & Lamphier, R. (2020, June), Assessing the Effectiveness of an Automated Problem Generator to Develop Course Content Rapidly and Minimize Student Cheating Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--34181

TY  - CPAPER
AB  - The education environment has taken a dramatic shift in the last decade with a greater focus in online delivery. In online and traditional classes alike, engineering faculty rely on textbooks, online publishers’ content, and problem repositories for class exercises as writing new problems every semester is prohibited. The problem with this is that both students and faculty have access to the same information on the web. With digital solution manuals or solution websites, the traditional delivery of homework has lost its integrity as the challenge of homework is no longer figuring out how to do the problem, it is now where does one find the online solution to a specific problem. 

This project documents the development, implementation, and testing of a true random problem generator in Python to create original content quickly whose solutions are not ubiquitous or accessible via some online solution manual. With true random problem generation, the inputs define the problem as an archetype (where archetype is defined as a specific problem type (e.g. projectile motion, ideal gas piston cylinder, etc.)) and can have virtually an infinite set of problems generated from it. The script will read a user defined input file, randomize parameters, randomize values for those parameters, test for logical correctness (fidelity), and finally use a natural language algorithm to generate both problem text and solution. 

To assess the effectiveness of problems created in this way, a survey was conducted having students rate the generator’s problems based on usefulness, clarity, and other qualitative features, as well as compare them to traditional, hand-written problems. A separate survey was conducted to score the accuracy of the generator’s difficulty heuristics by comparing the calculated difficulty of the software with the average of that perceived by the students. The assessment of the reduction of cheating was measured via blind survey and performance analysis. All-together, the functionality of this automated problem generator is established based on the qualitative measure of the aforementioned criteria, and overall perception of the potential future of online learning. 

AU  - Philip Jackson
AU  - Ricker Lamphier
CY  - Virtual On line 
DA  - 2020/06/22
PB  - ASEE Conferences
TI  - Assessing the Effectiveness of an Automated Problem Generator to Develop Course Content Rapidly and Minimize Student Cheating
UR  - https://peer.asee.org/34181
DO  - 10.18260/1-2--34181
ER  -