Assessment of Student Learning Outcomes and ABET Accreditation: A Pilot Study of Fourth-Year Engineering Students using Longitudinal Concept Maps

Sean Ferguson; Rider W. Foley

Download Paper | Permalink

Conference: 2017 ASEE Annual Conference & Exposition
Location: Columbus, Ohio
Publication Date: June 24, 2017
Start Date: June 24, 2017
End Date: June 28, 2017
Conference Session: Assessment and Liberal Education
Tagged Division: Liberal Education/Engineering & Society
Page Count: 18
DOI: 10.18260/1-2--27641
Permanent URL: https://peer.asee.org/27641
Download Count: 966

Request a correction

Paper Authors

biography

Sean Ferguson University of Virginia

visit author page

Sean Ferguson is a Lecturer in the Department of Engineering and Society at UVA since 2014. He specializes in sustainable technology and policy making from a background in Science and Technology Studies from Rensselaer Polytechnic Institute, working on energy and environmental policy in New York State, and a former life in cellular biology.

visit author page

biography

Rider W. Foley University of Virginia

visit author page

Dr. Rider W. Foley is an assistant professor in the science, technology & society program in the Department of Engineering and Society at the University of Virginia. He is the principal investigator at University of Virginia on the ‘4C Project’ on Cultivating Cultures of Ethical STEM education with colleagues from Notre Dame, Xavier University and St. Mary’s College. He is also the co-leader of the ‘Nano and the City’ thematic research cluster for the Center for Nanotechnology in Society at Arizona State University. Rider is a Research Collaborator with the Sustainability Science Education program at the Biodesign Institute. His research focuses on wicked problems that arise at the intersection of society and technology. Rider holds a Ph.D. in Sustainability from Arizona State University, and a Master's degree in Environmental Management from Harvard University and a Bachelor’s degree in Environmental Science from University of New Hampshire. Before earning his doctorate, he has worked for a decade in consulting and emergency response for Triumvirate Environmental Inc.

visit author page

Download Paper | Permalink

Abstract

The events surrounding Chernobyl, Three-mile Island, Challenger, and Bhopal prompted calls for a more rigorous call for engineering training. Professional societies representing engineers supported more innovative accreditation criteria to include ethics, global context, and understanding engineering problems as technical-social challenges with complex solutions. Pressure from global events and professional engineering societies contributed to ABET issuing the Engineering Criteria (EC) 2000. Those criteria signaled a shift to make non-technical criteria more explicit, diverse, and extend beyond communication skills. Engineering schools were tasked with developing assessment programs for fulfilling the new EC 2000 criteria and are again faced with a new set of criteria with the recently proposed amendments. Student portfolios are often a proxy for comprehensive learning or exams act as indicators of knowledge acquisition. However, there are few shared resources across institutions that are robust, malleable, and don’t require considerable effort by students, faculty and staff to create and codify. Engineering schools are challenged to attribute student outcomes for broader social context to specific curricular or course-design elements and then situate this attribution to overall progress for accreditation. This project introduces a pilot study using concept maps (cmaps) as an assessment technique that is useful for understanding learning over time, pedagogical influences between sections of a course, and mechanism to link to ABET criteria. Data was collected from 78 students completing three sequential maps out of a 590 person senior class conducted as a yearlong course in ‘Science, Technology and Society’ at the University of Virginia. One of two technical artifacts was used as a shared starting point on three occasions (August 2015, December 2015, and May 2016) with small amounts of class time allocated for the exercise. The cmaps were coded for 14 categories and analyzed with paired t-tests to identify key areas where students demonstrated significant learning outcomes changes over time. The findings illustrate how students articulate significantly greater technical knowledge and increased societal complexity in regards to a given material artifact. Pedagogical choices and course goals are matched to ABET criteria and supported by statistically significant correlations. Two focus groups offered a qualitative reflection upon the methodology and student outcomes. This method supports rigorous, but still flexible, means of linking ABET criteria for ethical and societal context to evaluation, assessment, student outcomes, and program objectives. This study suggests how concept mapping can be deployed and analyzed in a manner that offers compelling results between pre- and post-tests and offers a systematic means to assess ABET EC 2000 criteria by evaluators from across the country. We discuss limitations of timing, rule following, and interdisciplinary differences that warrant further systematic investigation.

Citation
Format

Ferguson, S., & Foley, R. W. (2017, June), Assessment of Student Learning Outcomes and ABET Accreditation: A Pilot Study of Fourth-Year Engineering Students using Longitudinal Concept Maps Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. 10.18260/1-2--27641

TY - CPAPER
AB - The events surrounding Chernobyl, Three-mile Island, Challenger, and Bhopal prompted calls for a more rigorous call for engineering training. Professional societies representing engineers supported more innovative accreditation criteria to include ethics, global context, and understanding engineering problems as technical-social challenges with complex solutions. Pressure from global events and professional engineering societies contributed to ABET issuing the Engineering Criteria (EC) 2000. Those criteria signaled a shift to make non-technical criteria more explicit, diverse, and extend beyond communication skills. Engineering schools were tasked with developing assessment programs for fulfilling the new EC 2000 criteria and are again faced with a new set of criteria with the recently proposed amendments. Student portfolios are often a proxy for comprehensive learning or exams act as indicators of knowledge acquisition. However, there are few shared resources across institutions that are robust, malleable, and don’t require considerable effort by students, faculty and staff to create and codify. Engineering schools are challenged to attribute student outcomes for broader social context to specific curricular or course-design elements and then situate this attribution to overall progress for accreditation.
This project introduces a pilot study using concept maps (cmaps) as an assessment technique that is useful for understanding learning over time, pedagogical influences between sections of a course, and mechanism to link to ABET criteria. Data was collected from 78 students completing three sequential maps out of a 590 person senior class conducted as a yearlong course in ‘Science, Technology and Society’ at the University of Virginia. One of two technical artifacts was used as a shared starting point on three occasions (August 2015, December 2015, and May 2016) with small amounts of class time allocated for the exercise. The cmaps were coded for 14 categories and analyzed with paired t-tests to identify key areas where students demonstrated significant learning outcomes changes over time. The findings illustrate how students articulate significantly greater technical knowledge and increased societal complexity in regards to a given material artifact. Pedagogical choices and course goals are matched to ABET criteria and supported by statistically significant correlations. Two focus groups offered a qualitative reflection upon the methodology and student outcomes. This method supports rigorous, but still flexible, means of linking ABET criteria for ethical and societal context to evaluation, assessment, student outcomes, and program objectives. This study suggests how concept mapping can be deployed and analyzed in a manner that offers compelling results between pre- and post-tests and offers a systematic means to assess ABET EC 2000 criteria by evaluators from across the country. We discuss limitations of timing, rule following, and interdisciplinary differences that warrant further systematic investigation.

AU - Sean Ferguson
AU - Rider W. Foley
CY - Columbus, Ohio
DA - 2017/06/24
PB - ASEE Conferences
TI - Assessment of Student Learning Outcomes and ABET Accreditation: A Pilot Study of Fourth-Year Engineering Students using Longitudinal Concept Maps
UR - https://peer.asee.org/27641
DO - 10.18260/1-2--27641
ER -