Exploring the Feasibility of an Educational Computer Game as a Novel Means of Assessing Problem Solving Competencies

Jacob R Grohs; Darren K Maczka; Michelle Soledad; Kiran Kelsey Bagalkotkar

Download Paper | Permalink

Conference: 2016 ASEE Annual Conference & Exposition
Location: New Orleans, Louisiana
Publication Date: June 26, 2016
Start Date: June 26, 2016
End Date: June 29, 2016
ISBN: 978-0-692-68565-5
ISSN: 2153-5965
Conference Session: Computer-Based Games and Animations in Education
Tagged Division: Computers in Education
Tagged Topic: Diversity
Page Count: 13
DOI: 10.18260/p.26869
Permanent URL: https://peer.asee.org/26869
Download Count: 530

Request a correction

Paper Authors

biography

Jacob R Grohs Virginia Tech

visit author page

Jacob Grohs is an Assistant Professor in Engineering Education at Virginia Tech with Affiliate Faculty status in Biomedical Engineering and Mechanics and the Learning Sciences and Technologies at Virginia Tech. He holds degrees in Engineering Mechanics (BS, MS) and in Educational Psychology (MAEd, PhD).

visit author page

biography

Darren K Maczka Virginia Tech Department of Engineering Education orcid.org/0000-0001-5966-5670

visit author page

Darren Maczka is a Ph.D. student in Engineering Education at Virginia Tech. His background is in control systems engineering and information systems design and he received his B.S. in Computer Systems Engineering from The University of Massachusetts at Amherst. He has several years of experience teaching and developing curricula in the department of Electrical and Computer Engineering at Virginia Tech.

visit author page

biography

Michelle Soledad Virginia Tech orcid.org/0000-0002-2491-6684

visit author page

Michelle Soledad is a doctoral student and graduate research assistant in the Department of Engineering Education at the Virginia Polytechnic Institute and State University. Her research interests include faculty motivation and development towards supporting student success. Ms. Soledad has bachelor's and master's degrees in Electrical Engineering from the Ateneo de Davao University in Davao City, Philippines, where she continues to be a faculty member of the Electrical Engineering Department. She also served as Department Chair and was a member of the University Research Council before pursuing doctoral studies at Virginia Tech. Prior to joining the University in 2008, Ms. Soledad was a Senior Team Lead for Accenture, where she worked on and managed systems maintenance and enhancement projects.

visit author page

author page

Kiran Kelsey Bagalkotkar

Download Paper | Permalink

Abstract

We believe that educating problem solvers is one of the most critical functions of higher education. In the problem-solving literature, problems are understood in part by the degree to which they are structured. Well-structured problems have clear goals which can be obtained through repeatable algorithms to get absolute correct solutions. In contrast, ill-structured problems are inherently vague, can have several viable solution paths, and solving the problem largely requires subjective arguments that frame and guide the attempted solutions.

The central aim of this paper is to explore the feasibility of using an educational computer game as a novel means of assessing problem-solving competency. In this case, the novelty of the approach compared to traditional methods is twofold: (1) the use of a computer game enables us to directly observe problem-solving process through action in the game and (2) because of the naturally immersive game environment, we hope to see motivation and persistence in the face of complex problems that might otherwise be difficult to achieve in a laboratory setting. As a feasibility study, we present methods and data that we suggest should guide further research but should not be used to make specific claims.

In particular, this project uses the educational game Contraption Maker as a problem-solving research tool. Contraption Maker is a commercial (free for educational use) game involving Rube Goldberg inspired puzzles using simple physics-based interactions among objects. The simple premise of the game is seen as an advantage because it minimizes the importance of domain specific knowledge. This project poses a series of increasingly difficult puzzles to participants. Video recordings of the computer screen allow study of the diverse ways in which participants solve puzzles and retrospective interviews of participants allow us to understand the problem solving strategies used by individuals while attempting solutions.

The paper explains the schemes used to analyze the data along with initial findings from the pilot study with university students. We hope this work will generate conversation about novel ways in which educational computer games can be used as informative research and assessment tools. In particular, because Contraption Maker is being used in some K-12 classrooms as a learning tool, this research could contribute to more robust ways of studying game-based learning. Finally, this work also serves as an important first step towards further utilizing computers in problem-solving research. As we better understand how to port problem-solving tasks in a digital environment like this, we can more fully utilize the capabilities of the technology to automate some data processing. One such example could involve using automatically collected mouse-over data in order to identify how the problem-solving process proceeded.

Citation
Format

Grohs, J. R., & Maczka, D. K., & Soledad, M., & Bagalkotkar, K. K. (2016, June), Exploring the Feasibility of an Educational Computer Game as a Novel Means of Assessing Problem Solving Competencies Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. 10.18260/p.26869

TY - CPAPER
AB - We believe that educating problem solvers is one of the most critical functions of higher education. In the problem-solving literature, problems are understood in part by the degree to which they are structured. Well-structured problems have clear goals which can be obtained through repeatable algorithms to get absolute correct solutions. In contrast, ill-structured problems are inherently vague, can have several viable solution paths, and solving the problem largely requires subjective arguments that frame and guide the attempted solutions.

The central aim of this paper is to explore the feasibility of using an educational computer game as a novel means of assessing problem-solving competency. In this case, the novelty of the approach compared to traditional methods is twofold: (1) the use of a computer game enables us to directly observe problem-solving process through action in the game and (2) because of the naturally immersive game environment, we hope to see motivation and persistence in the face of complex problems that might otherwise be difficult to achieve in a laboratory setting. As a feasibility study, we present methods and data that we suggest should guide further research but should not be used to make specific claims.

In particular, this project uses the educational game Contraption Maker as a problem-solving research tool. Contraption Maker is a commercial (free for educational use) game involving Rube Goldberg inspired puzzles using simple physics-based interactions among objects. The simple premise of the game is seen as an advantage because it minimizes the importance of domain specific knowledge. This project poses a series of increasingly difficult puzzles to participants. Video recordings of the computer screen allow study of the diverse ways in which participants solve puzzles and retrospective interviews of participants allow us to understand the problem solving strategies used by individuals while attempting solutions.

The paper explains the schemes used to analyze the data along with initial findings from the pilot study with university students. We hope this work will generate conversation about novel ways in which educational computer games can be used as informative research and assessment tools. In particular, because Contraption Maker is being used in some K-12 classrooms as a learning tool, this research could contribute to more robust ways of studying game-based learning. Finally, this work also serves as an important first step towards further utilizing computers in problem-solving research. As we better understand how to port problem-solving tasks in a digital environment like this, we can more fully utilize the capabilities of the technology to automate some data processing. One such example could involve using automatically collected mouse-over data in order to identify how the problem-solving process proceeded.

AU - Jacob R Grohs
AU - Darren K Maczka
AU - Michelle Soledad
AU - Kiran Kelsey Bagalkotkar
CY - New Orleans, Louisiana
DA - 2016/06/26
PB - ASEE Conferences
TI - Exploring the Feasibility of an Educational Computer Game as a Novel Means of Assessing Problem Solving Competencies
UR - https://peer.asee.org/26869
DO - 10.18260/p.26869
ER -