Asee peer logo

A Direct Method for Teaching and Measuring Engineering Professional Skills: A Validity Study

Download Paper |

Conference

2011 ASEE Annual Conference & Exposition

Location

Vancouver, BC

Publication Date

June 26, 2011

Start Date

June 26, 2011

End Date

June 29, 2011

ISSN

2153-5965

Conference Session

NSF Grantees Poster Session

Tagged Topic

NSF Grantees

Page Count

21

Page Numbers

22.38.1 - 22.38.21

Permanent URL

https://peer.asee.org/17320

Download Count

78

Request a correction

Paper Authors

biography

Ashley Ater Kranov Washington State University

visit author page

Dr. Ater Kranov is Director of Educational Innovation and Assessment for the College of Engineering and Architecture at Washington State University, USA. She is affiliated assistant professor in the School of Electrical Engineering and Computer Science where she co-teaches the 2-semester senior design capstone sequence.

Dr. Ater Kranov is a leader in university and community internationalization efforts, including developing and assessing global competencies in faculty, staff, and students. The paper describing her collaborative work with faculty in the WSU College of Engineering and Architecture, "A Direct Method for Teaching and Assessing the ABET Professional Skills in Engineering Programs", won the 2008 ASEE Best Conference Paper Award. She has served as evaluator on a number of multi-institutional, interdisciplinary NSF sponsored grants. She is principal investigator on a NSF Research and Evaluation on Education in Science and Engineering project called “A Direct Method for Teaching and Measuring Engineering Professional Skills: A Validity Study.”

visit author page

biography

Mo Zhang Washington State University

visit author page

Mo Zhang is a doctoral student major in educational psychology at Washington State University. Her research interests include applied statistics, educational measurement, design of experiments, sampling theories, and item response theory oriented mathematical models. She holds an M.A. in education from Washington State University.

visit author page

biography

Steven W. Beyerlein University of Idaho, Moscow

visit author page

Dr. Beyerlein is a professor of Mechanical Engineering and coordinator of the college-wide inter-disciplinary capstone design program at the University of Idaho where he has been on the faculty since 1987. He is involved in a number of research projects and initiatives related to design pedagogy, professional skills assessment, catalytic combustion, engine testing, and hybrid vehicle realization.

visit author page

biography

Jay McCormack University of Idaho

visit author page

Jay McCormack is an assistant professor in the mechanical engineering department at the University of Idaho where he is an instructor for the college's interdisciplinary capstone design course. Dr. McCormack received his PhD in mechanical engineering from Carnegie Mellon University in 2003.

visit author page

biography

Patrick D. Pedrow Washington State University

visit author page

Patrick D. Pedrow received the B.S. degree in electrical engineering from the University of Idaho, Moscow, in 1975, the M.Eng. degree in electric power engineering from Rensselaer Polytechnic Institute, Troy, NY, in 1976, the M.S. degree in physics from Marquette University, Milwaukee, WI, in 1981, and the Ph.D. degree in electrical engineering from Cornell University, Ithaca, NY, in 1985. From 1976 to 1981, he was with McGraw-Edison Company, where he conducted research and development on electric power circuit breakers. He is currently an Associate Professor with Washington State University, Pullman, where he is the Associate Director of the School of Electrical Engineering and Computer Science. His research interests are in plasma-assisted materials processing, including the deposition and evaluation of thin plasma-polymerized films. Dr. Pedrow is a member of IEEE, the American Physical Society, Tau Beta Pi and he is a Registered Professional Engineer in the State of Wisconsin. He has served on the Executive Committee of the IEEE Nuclear and Plasma Sciences Society Plasma Science and Applications Committee.

visit author page

biography

Edwin R. Schmeckpeper Norwich University

visit author page

Dr. Schmeckpeper is an Associate Professor at Norwich University’s David Crawford School of Engineering, the oldest private engineering school in the nation. Prior to coming to Norwich University he was an Associate Professor at the University of Idaho.

visit author page

Download Paper |

Abstract

A Direct Method for Teaching and Measuring Engineering Professional Skills: A Validity StudyProficiency in engineering professional skills (Table 1) is critical for success in themultidisciplinary, intercultural team interactions that characterize 21st century engineeringcareers. In 2006, the XXX University College of Engineering and Architecture partnered withthe Office of YYY to create an innovative, direct method to teach and measure the ABETprofessional skills simultaneously called the curricular debrief (CD) method. The CD method isan authentic performance task and the Engineering Professional Skills Rubric (EPS), an analyticrubric to measure the quality of the student performance. No direct method for teaching andmeasuring these skills simultaneously has existed in the literature prior to the CD method.Table 1. ABET Criterion 3 Professional Skills Student Learning Outcomes 3d Ability to Function on Multidisciplinary Teams 3f Understanding of Professional and Ethical Responsibility 3g Ability to Communicate Effectively 3h Understanding of the Impact of Engineering Solutions in Global, Economic, Environmental, and Cultural/Societal Contexts 3i Recognition of and Ability to Engage in Life-Long Learning 3j Knowledge of Contemporary IssuesThe four years of on-going college-wide, program-level research conducted at WSU to establishthe initial reliability and validity of this method has shown its potential for significant nationwideimpact. Yet, more research is needed to provide strong evidence that the CD method as aperformance task and the EPS Rubric as a measurement instrument are reliable and valid. Thisstudy’s primary research goal is to rigorously establish the reliability and validity of the CDmethod and the EPS Rubric. This project will directly contribute to fundamental research inengineering education on a problem of national importance and interest.Our research questions are: 1) To what extent does the CD method as a performance task equally elicit students’ consideration of engineering professional skills when implemented in different course types and at different points in a program’s curriculum? 2) Do EPS Rubric scores reliably provide information about students’ engineering professional skills proficiency levels? 3) What is the correlation coefficient between the EPS Rubric’s scores and scores from other established instruments that measure the same or similar skills?A descriptive collective multi-site case-study methodology will be used. This methodology willallow the investigators to understand and examine the contexts in which parallel performancetasks are implemented in three distinct sites and four distinct course-type settings. A completerandomized design will be used to sample students into control and experiment groups withineach course offering, where each course offering is a block of the analysis. There will be 70experimental teams and 66 control teams for a total of 136 teams (796 students).The studentteam will be the primary unit of analysis within a block.Pattern matching will be used to establish the performance assessment’s reliability and validity,complemented by these statistical analytic techniques: (a) a generalizability study; (b) ANOVA; ത(c) correlation coefficient; and (d) task difficulty using the formula p = ܺ/ܺ௠௔௫

Kranov, A. A., & Zhang, M., & Beyerlein, S. W., & McCormack, J., & Pedrow, P. D., & Schmeckpeper, E. R. (2011, June), A Direct Method for Teaching and Measuring Engineering Professional Skills: A Validity Study Paper presented at 2011 ASEE Annual Conference & Exposition, Vancouver, BC. https://peer.asee.org/17320

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2011 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015