Asee peer logo

Framework for Evaluating Simulations: Analysis of a Student-Developed Interactive Computer Tool

Download Paper |


2015 ASEE Annual Conference & Exposition


Seattle, Washington

Publication Date

June 14, 2015

Start Date

June 14, 2015

End Date

June 17, 2015





Conference Session

Novel Teaching Methods in a Multidisciplinary Context

Tagged Division

Multidisciplinary Engineering

Page Count


Page Numbers

26.796.1 - 26.796.13



Permanent URL

Download Count


Request a correction

Paper Authors


Kelsey Joy Rodgers Purdue University, West Lafayette Orcid 16x16

visit author page

Kelsey Rodgers is a graduate student at Purdue University in the School of Engineering Education. Her research focus is investigating how engineers' understand, develop, and use mathematical models and simulations. Her research also focuses on feedback. She is currently conducting research in first-year engineering on the Network for Nanotechnology (NCN) Educational Research team. She previously conducted research with the Model-Eliciting Activities (MEAs) Educational Research team and a few fellow STEM education graduates for an obtained Discovery, Engagement, and Learning (DEAL) grant. Prior to attending Purdue University, she graduated from Arizona State University with her B.S.E in Engineering from the College of Technology and Innovation, where she worked on a team conducting research on how students learn LabVIEW through Disassemble, Analyze, Assemble (DAA) activities.

visit author page


Heidi A. Diefes-Dux Purdue University, West Lafayette Orcid 16x16

visit author page

Heidi A. Diefes-Dux is a Professor in the School of Engineering Education at Purdue University. She received her B.S. and M.S. in Food Science from Cornell University and her Ph.D. in Food Process Engineering from the Department of Agricultural and Biological Engineering at Purdue University. She is a member of Purdue’s Teaching Academy. Since 1999, she has been a faculty member within the First-Year Engineering Program, teaching and guiding the design of one of the required first-year engineering courses that engages students in open-ended problem solving and design. Her research focuses on the development, implementation, and assessment of modeling and design activities with authentic engineering contexts. She is currently a member of the educational team for the Network for Computational Nanotechnology (NCN).

visit author page


Yi Kong Purdue University, West Lafayette

visit author page

Yi Kong is a doctoral student in biology education and a graduate research assistant for the Network for Computational Nanotechnology (NCN) education research team at Purdue University. She received her M.S. in agriculture in Fishery Resources from Huazhong Agricultural University and B.S. in Biological Science from Shaanxi Normal University in China. Her research includes evaluating first-year engineering students’ communication of nanoscience concepts through project-based-learning activities.

visit author page


Krishna Madhavan Purdue University, West Lafayette

visit author page

Dr. Krishna Madhavan is an Assistant Professor in the School of Engineering Education at Purdue University. He is Co-PI and Education Director of the NSF-funded Network for Computational Nanotechnology ( which serves over 330,000 global researchers and learners annually). Dr. Madhavan was the Chair of the IEEE/ACM Supercomputing Education Program 2006. In January 2008, he was awarded the US National Science Foundation (NSF) CAREER award for work on learner-centric, adaptive cyber-tools and cyber-environments. He was one of 49 faculty members selected as the nation’s top engineering educators and researchers by the US National Academy of Engineering to the Frontiers in Engineering Education symposium. Dr. Madhavan leads a major NSF funded effort called Deep Insights Anytime, Anywhere (DIA2) that attempts to characterize the impact of NSF and other federal investments in the area of science, technology, engineering, and mathematics education using interactive knowledge mining and visual analytics for non-experts in data mining. DIA2 is currently deployed inside the NSF and is already starting to affect federal funding policy. Dr. Madhavan also served as Visiting Research Scientist at Microsoft Research, Internet Services Research Group. His research has been published in Nature Nanotechnology, IEEE Transactions on Computer Graphics and Applications, IEEE Transactions on Learning Technologies, and several other top peer-reviewed venues. Dr. Madhavan currently serves as PI or Co-PI on federal and industry funded projects totaling over $20M.

visit author page

Download Paper |


Framework of Basic Interactions to Computer Simulations: Analysis of Student Developed Interactive Computer ToolsComputer simulations make learning meaningful through interactive and authentic opportunitiesto observe, explore, and recreate real objects, phenomena, and processes that would otherwise beimpossible to investigate due to complexity, size-constraints, time-consumption, and/or danger.Simulations are crucial for the analysis and understanding of physical properties and products,especially at small scales like the nanoscale. According to the National Center for Learning andTeaching in Nanoscale Science and Engineering (NCLT) and the National Science TeachersAssociations (NSTA), the use of computer simulations in nanotechnology is one of the “bigideas” of nanotechnology education. Based on the lack of literature describing the progression ofstudent developed simulations in an open-ended learning environment and informal practionerreports of students’ struggles with understanding the nature of a simulation, this studyinvestigates student teams’ “simulations”. This study is driven by the following researchquestion: What types of interactions, mathematical models, and simulations are seen in teams’design projects?A required First-Year Engineering (FYE) course at a Midwestern U.S. university utilizes open-ended problem solving and scaffolding through feedback to encourage student learning. InSpring 2013, students were given a design project that challenged teams to develop an interactivetool (using MATLAB) to present nanotechnology concepts and applications to their peersthrough one or more simulations. The teams iteratively developed their design projects throughnine milestones and with continuous feedback from instructors. The final submissions of 30teams were analyzed. The theoretical framework for this study was grounded theory and thestrategies of inquiry were open coding and axial coding [3]. Coding categories were developedbased on the data and then slightly modified based on existing literature to establish moremeaningful language to describe different components of a simulation (i.e. glass vs. black boxmodel, types of variables – discrete vs. continuous).The resulting coding scheme consists of four progressive levels of interaction in a graphical-userinterface (GUI) building up to simulations: Level 1: Basic Level of Interaction (e.g. clicking topull up more information, selecting quiz answers), Level 2: Simple Input to Output through aBlack Box Equation (e.g. conversion calculator – input a size for a macro scale outputs the sizein nanometers), Level 3: Animation of a Simulation – user can only play simulation with defaultinput variables, and Level 4: Simulation – user can change input variables and interact withvisualizations that help communicate the concept (making it more of a glass box process). Theresults of this coding scheme applied to the 30 teams will be given in the full paper along withimplications for practice (e.g. curriculum development, scaffolding techniques, and assessmentmethods pertinent to teaching simulation building through open-ended problem solving learningenvironments).

Rodgers, K. J., & Diefes-Dux, H. A., & Kong, Y., & Madhavan, K. (2015, June), Framework for Evaluating Simulations: Analysis of a Student-Developed Interactive Computer Tool Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24133

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015