Asee peer logo

Defining and Assessing Global Engineering Competency: Methodological Reflections

Download Paper |


2015 ASEE Annual Conference & Exposition


Seattle, Washington

Publication Date

June 14, 2015

Start Date

June 14, 2015

End Date

June 17, 2015





Conference Session

Discussions on Research Methodology: ERM Roundtable

Tagged Division

Educational Research and Methods

Page Count


Page Numbers

26.442.1 - 26.442.13



Permanent URL

Download Count


Request a correction

Paper Authors


Brent K Jesiek Purdue University, West Lafayette

visit author page

Dr. Brent K. Jesiek is Associate Professor in the Schools of Engineering Education and Electrical and Computer Engineering at Purdue University. He is also an Associate Director of Purdue’s Global Engineering Program, leads the Global Engineering Education Collaboratory (GEEC) research group, and is the recipient of an NSF CAREER award to study boundary-spanning roles and competencies among early career engineers. He holds a B.S. in Electrical Engineering from Michigan Tech and M.S. and Ph.D. degrees in Science and Technology Studies (STS) from Virginia Tech. Dr. Jesiek draws on expertise from engineering, computing, and the social sciences to advance understanding of geographic, disciplinary, and historical variations in engineering education and practice.

visit author page


Sang Eun Woo Purdue University

visit author page

Sang Eun Woo is an assistant professor in the Department of Psychological Sciences at Purdue University. She received her Ph.D. degree in industrial and organizational psychology from University of Illinois Urbana-Champaign. Her research interests include construct validation, psychological measurement (focusing on behavioral assessment), personality and individual differences, turnover and withdrawal, and alternative methodological approaches to organizational and psychological science.

visit author page


Qin Zhu Purdue University Orcid 16x16

visit author page

Qin Zhu is a Ph.D. student in the School of Engineering Education at Purdue University. His main research interests include global, comparative, and international engineering education, engineering education policy, and engineering ethics. He received his B.S. degree in materials science and engineering and his first Ph.D. degree in the philosophy of science and technology (engineering ethics) both from Dalian University of Technology, China.

visit author page

author page

Kavitha D Ramane Purdue University


Neha Choudhary Purdue University Programs

visit author page

Neha Choudhary is currently pursuing doctoral studies at Purdue University in Engineering Education. Her research interests include graduate studies, global engineering, and design for community services.

visit author page

Download Paper |


Defining and Assessing Global Engineering Competency: Methodological ReflectionsResearchers are facing manifold challenges as engineering education continues to emerge, grow,and evolve as a distinct field of scholarly activity. For instance, discussions about criteria forevaluating engineering education research have intensified, including through publishedcommentaries focused on concepts like “research quality,” “rigor,” and “systematic research,” aswell as accompanying shifts in the various criteria used to evaluate funding proposals and peerreviewed papers. The field’s topical foci are also something of a moving target given a long andepisodic history of efforts to reinvent the form and content of engineering curricula. As themethods and desired outcomes of engineering instruction change, so does the engineeringeducation research agenda. Further worth noting are rising pressures to relate research to practice,as reflected in mandates to identify the “broader impacts” associated with our scholarly work.This paper speaks to these challenges through the lens of our team’s recent experiences workingon an NSF-supported research project, one primary objective of which is to develop and validatea global competency assessment tool. This paper is organized in three major parts, each focusedon a different phase of the project and a variety of associated questions and challenges. The firstsection is focused on efforts to clarify our main domain of interest, including by developing aworking definition of “global engineering competency” that is grounded in both the extantliterature and our own qualitative data set. This part of the paper is particularly concerned withhow we meaningfully delineate, define, and study specific domains or facets of engineeringeducation and practice, including by working toward more nuanced interpretations of keyattributes and capabilities important for current and future engineers. The second section of thepaper describes our efforts to develop a large pool of assessment questions using a situationaljudgment test (SJT) format. Here we interrogate our rationale for picking this type of assessmenttool, procedures for developing the scenario-based test questions, and efforts to bolster validitythrough use of an underlying theoretical framework and systematic item development procedures.Third and finally, we discuss our ongoing efforts to validate our large pool of SJT questions,with an emphasis on data collection bottlenecks and questions about what counts as validation indifferent academic fields. In this paper we also explore how we are working to maximize theimpacts of our assessment questions by building an online survey and assessment system.The major goal of this paper is to make visible and critically reflect on key methodological issuesand decisions that are often absent or simplified in more conventional research papers. Ourcommentary will likely be of interest for engineering education researchers who wish to explorecontemporary methodological concerns that cut across both qualitative and quantitative researchtraditions. This paper may also be of interest to scholars involved with ongoing efforts to define,develop, and/or assess complex, multi-faceted learning outcomes such as global competency,innovation, teamwork, leadership, inter/trans/multi-disciplinarity, engineering design, etc.

Jesiek, B. K., & Woo, S. E., & Zhu, Q., & Ramane, K. D., & Choudhary, N. (2015, June), Defining and Assessing Global Engineering Competency: Methodological Reflections Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23781

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015