Refining Concept Maps as Method to Assess Learning Outcomes Among Engineering Students

Sean Michael Ferguson; Rider W. Foley; John Kofi Eshirow; Catherine Claire Pollack

Download Paper | Permalink

Conference: 2018 ASEE Annual Conference & Exposition
Location: Salt Lake City, Utah
Publication Date: June 23, 2018
Start Date: June 23, 2018
End Date: July 27, 2018
Conference Session: Maps, Metaphors, Tweets, and Drafts
Tagged Division: Liberal Education/Engineering & Society
Page Count: 20
DOI: 10.18260/1-2--30924
Permanent URL: https://peer.asee.org/30924
Download Count: 576

Paper Authors

biography

Sean Michael Ferguson University of Virginia

visit author page

Sean Ferguson is a Lecturer in the Department of Engineering and Society at UVA since 2014. He specializes in sustainable technology, standards, and bioeconomy from a background in Science and Technology Studies from Rensselaer Polytechnic Institute, working on energy and environmental policy in New York State, and a former life in cellular biology.

visit author page

biography

Rider W. Foley University of Virginia

visit author page

Dr. Rider W. Foley is an assistant professor in the science, technology & society program in the Department of Engineering and Society at the University of Virginia. He is the principal investigator at University of Virginia on the ‘4C Project’ on Cultivating Cultures of Ethical STEM education with colleagues from Notre Dame, Xavier University and St. Mary’s College. He is also the co-leader of the ‘Nano and the City’ thematic research cluster for the Center for Nanotechnology in Society at Arizona State University. Rider is a Research Collaborator with the Sustainability Science Education program at the Biodesign Institute. His research focuses on wicked problems that arise at the intersection of society and technology. Rider holds a Ph.D. in Sustainability from Arizona State University, and a Master's degree in Environmental Management from Harvard University and a Bachelor’s degree in Environmental Science from University of New Hampshire. Before earning his doctorate, he has worked for a decade in consulting and emergency response for Triumvirate Environmental Inc.

visit author page

author page

John Kofi Eshirow Jr. University of Virginia

biography

Catherine Claire Pollack University of Virginia

visit author page

Catherine Pollack is a fourth year undergraduate Rodman Scholar at the University of Virginia pursuing a double major in biomedical engineering and applied statistics. Following graduation, she will attend the Geisel School of Medicine at Dartmouth College to pursue a Ph.D. in quantitative biomedical sciences beginning Fall 2018.

visit author page

Download Paper | Permalink

Abstract

Concept mapping (CM) activities are well understood as a formative, generative and summative means to assess learning and document knowledge acquisition in a variety of disciplines. Within Engineering Education (EE) and other STEM fields, CM has been deemed successful in assessing whether a given set of students have achieved learning goals associated with ‘perfect’ (or near perfect) representation of established engineering and scientific concepts. What is less understood is how to assess learning goals in which there is no ‘perfect’ answer or expert level example to the question. CMs have struggled in adapting to situations where students need to demonstrate the competencies in complex sociocultural phenomenon, systems thinking, or ethical reasoning. The challenges are clear for those educators, assessors and companies that want engineering students to have demonstrated the non-technical standards set forth by ABET EC2000, specifically C3 and C5, and the current debates among those in charge of accreditation. This paper offers a methodological application from past research and suggests refinements in the administration and scoring of concept maps after presenting the successes and challenges of this form of assessment. The goal is to understand knowledge acquisition related to conceptualizing the interconnections between technical, social, environmental, ethical and global dimensions of engineering. We build on a previously developed data set with an initial ‘pre-test’ concept mapping activity performed in August 2016 by 151 4th year undergraduate engineering students enrolled in a mandatory, multidisciplinary course sequence on the first day of class. A second CM ‘post-test’ was performed by 115 of the original students in early March 2017. Comparing the output of pre- and post-test demonstrates what categories of knowledge are acquired and how the acquired knowledge aligns with the ABET criteria. A second evaluation considers how instructor’s pedagogical approaches influence the outcomes. This analysis suggests that CM offers a compelling strategy to assess knowledge acquisition where no ‘perfect’ answer can be used for absolute measurement. This assessment method offers a bridge from assessment tools for individual knowledge representation and programmatic assessment.

Citation
Format

Ferguson, S. M., & Foley, R. W., & Eshirow, J. K., & Pollack, C. C. (2018, June), Refining Concept Maps as Method to Assess Learning Outcomes Among Engineering Students Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. 10.18260/1-2--30924

TY - CPAPER
AB - Concept mapping (CM) activities are well understood as a formative, generative and summative means to assess learning and document knowledge acquisition in a variety of disciplines. Within Engineering Education (EE) and other STEM fields, CM has been deemed successful in assessing whether a given set of students have achieved learning goals associated with ‘perfect’ (or near perfect) representation of established engineering and scientific concepts. What is less understood is how to assess learning goals in which there is no ‘perfect’ answer or expert level example to the question. CMs have struggled in adapting to situations where students need to demonstrate the competencies in complex sociocultural phenomenon, systems thinking, or ethical reasoning. The challenges are clear for those educators, assessors and companies that want engineering students to have demonstrated the non-technical standards set forth by ABET EC2000, specifically C3 and C5, and the current debates among those in charge of accreditation. This paper offers a methodological application from past research and suggests refinements in the administration and scoring of concept maps after presenting the successes and challenges of this form of assessment. The goal is to understand knowledge acquisition related to conceptualizing the interconnections between technical, social, environmental, ethical and global dimensions of engineering. We build on a previously developed data set with an initial ‘pre-test’ concept mapping activity performed in August 2016 by 151 4th year undergraduate engineering students enrolled in a mandatory, multidisciplinary course sequence on the first day of class. A second CM ‘post-test’ was performed by 115 of the original students in early March 2017. Comparing the output of pre- and post-test demonstrates what categories of knowledge are acquired and how the acquired knowledge aligns with the ABET criteria. A second evaluation considers how instructor’s pedagogical approaches influence the outcomes. This analysis suggests that CM offers a compelling strategy to assess knowledge acquisition where no ‘perfect’ answer can be used for absolute measurement. This assessment method offers a bridge from assessment tools for individual knowledge representation and programmatic assessment.
AU - Sean Michael Ferguson
AU - Rider W. Foley
AU - John Kofi Eshirow Jr.
AU - Catherine Claire Pollack
CY - Salt Lake City, Utah
DA - 2018/06/23
PB - ASEE Conferences
TI - Refining Concept Maps as Method to Assess Learning Outcomes Among Engineering Students
UR - https://peer.asee.org/30924
DO - 10.18260/1-2--30924
ER -