Assessing the Effectiveness of Shah's Innovation Metrics for Measuring Innovative Design within a Virtual Design Space
- Conference
- Location
-
Columbus, Ohio
- Publication Date
-
June 24, 2017
- Start Date
-
June 24, 2017
- End Date
-
June 28, 2017
- Conference Session
- Tagged Division
-
Entrepreneurship & Engineering Innovation
- Page Count
-
17
- DOI
-
10.18260/1-2--27624
- Permanent URL
-
https://peer.asee.org/27624
- Download Count
-
622
Paper Authors
Abigail Jane Kulhanek University of Pittsburgh
Abigail Kulhanek is an undergraduate student studying Chemical Engineering at the University of Pittsburgh.
Matthew Raymond Markovetz University of North Carolina, Chapel Hill
Matthew Markovetz is a postdoctoral researcher in the Cystic Fibrosis and Pulmonary Disease Research Center at the University of North Carolina at Chapel Hill. His interest in both engineering education and technical engineering research developed while studying Chemical and Biological Engineering at the University of Colorado at Boulder. Matthew’s research in education focuses on methods that assess and increase innovation in product design, and his laboratory research seeks to understand and treat the airway dehydration present in patients with Cystic Fibrosis through mathematical modeling, rheological analysis, and systems engineering principles.
Golnaz Arastoopour Irgens University of Wisconsin, Madison
Before becoming interested in education, Golnaz studied Mechanical Engineering at the University of Illinois at Urbana-Champaign with a minor in Spanish. While earning her Bachelor’s degree in engineering, she worked as a computer science instructor at Campus Middle School for Girls in Urbana, IL. Along with a team of undergraduates, she headlined a project to develop a unique computer science curriculum for middle school students. She then earned her M.A. in mathematics education at Columbia University. Afterwards, she taught in the Chicago Public School system at Orr Academy High School (an AUSL school) for two years. Currently, Golnaz is working with the Epistemic Games Research Group at the University of Wisconsin-Madison where she has led the efforts on engineering virtual internship simulations for high school and first year undergraduate students. Golnaz's current research is focused on how games and simulations increase student engagement in STEM fields, how players learn engineering design in real-world and virtual professional environments, and how to assess engineering design thinking.
Zachari Lucius Swiecki University of Wisconsin, Madison
Graduate student in educational psychology, learning sciences area.
Naomi C. Chesler University of Wisconsin, Madison
Naomi C. Chesler is Professor of Biomedical Engineering with an affiliate appointment in Educational Psychology. Her research interests include vascular biomechanics, hemodynamics and cardiac function as well as the factors that motivate students to pursue and persist in engineering careers, with a focus on women and under-represented minorities.
David Williamson Shaffer University of Wisconsin, Madison
David Williamson Shaffer is the Vilas Distinguished Professor of Learning Sciences at the University of Wisconsin, Madison, in the Department of Educational Psychology, and a Game Scientist at the Wisconsin Center for Education Research. His most recent book is How Computer Games Help Children Learn.
Cheryl A. Bodnar Rowan University
Cheryl A. Bodnar, Ph.D., CTDP is an Assistant Professor in the Department of Experiential Engineering Education at Rowan University. Dr. Bodnar’s research interests relate to the incorporation of active learning techniques in undergraduate classes as well as integration of innovation and entrepreneurship into the engineering curriculum. In particular, she is interested in the impact that these tools can have on student perception of the classroom environment, motivation and learning outcomes. She obtained her certification as a Training and Development Professional (CTDP) from the Canadian Society for Training and Development (CSTD) in 2010, providing her with a solid background in instructional design, facilitation and evaluation. She was selected to participate in the National Academy of Engineering (NAE) Frontiers of Engineering Education Symposium in 2013 and awarded the American Society for Engineering Education Educational Research Methods Faculty Apprentice Award in 2014.
Abstract
Epistemic games, such as the virtual engineering internship Nephrotex, allow students to explore creative ways to approaching engineering problems while providing a novel alternative to the direct transmission method of instruction. Within Nephrotex, students choose a polymer, manufacturing process, surfactant, and percentage of carbon nanotube to create a functioning kidney dialysis membrane prototype. The performance of the membrane is measured using cost, flux, blood cell reactivity, marketability, and reliability thresholds given by stakeholders within the fictitious company. Although Nephrotex has been shown to be a valuable educational tool for modeling the product design process, only limited work has been done to investigate whether it is capable of providing an environment that allows students to generate innovative designs.
The innovation assessment framework of Shah and colleagues employs four metrics of innovation – novelty, variety, quality, and quantity; novelty is further divided into a priori and a posteriori metrics. This work found that a priori and a posteriori novelty, variety, and quality were applicable metrics of innovation in the epistemic game environment of Nephrotex. Literature ranges for a priori and a posteriori novelty scores aligned with those found in this study.
Comparing prior work on Nephrotex that identified innovative student designs based on a proposed literature definition, it was found that the Shah metrics between the innovative and non-innovative groups showed little variation and no statistically significant differences. A t test and a Mann-Whitney U test showed no significant difference between innovative and non-innovative groups with regard to variety or novelty scores; however, these tests did show a significant difference between groups with regards to the quality score. The same results were found when calculating Cohen’s Effect Size – a priori novelty, a posteriori novelty, and variety had a small effect when comparisons were made between the innovative and non-innovative groups while quality had a large effect. The significant difference and large effect in regards to quality however, may be the result of the previous literature definition which employed quality as a measure to define innovation. Results from this study demonstrate that novelty is perhaps the most aligned innovation metric for an epistemic game environment and that both variety and quality can be helpful in understanding the designs generated within these contexts although they may need adjustment based on the application to a constrained design space.
Kulhanek, A. J., & Markovetz, M. R., & Arastoopour Irgens, G., & Swiecki, Z. L., & Chesler, N. C., & Shaffer, D. W., & Bodnar, C. A. (2017, June), Assessing the Effectiveness of Shah's Innovation Metrics for Measuring Innovative Design within a Virtual Design Space Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. 10.18260/1-2--27624
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2017 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015