June 14, 2015
June 14, 2015
June 17, 2015
Educational Research and Methods
26.442.1 - 26.442.13
Defining and Assessing Global Engineering Competency: Methodological ReflectionsResearchers are facing manifold challenges as engineering education continues to emerge, grow,and evolve as a distinct field of scholarly activity. For instance, discussions about criteria forevaluating engineering education research have intensified, including through publishedcommentaries focused on concepts like “research quality,” “rigor,” and “systematic research,” aswell as accompanying shifts in the various criteria used to evaluate funding proposals and peerreviewed papers. The field’s topical foci are also something of a moving target given a long andepisodic history of efforts to reinvent the form and content of engineering curricula. As themethods and desired outcomes of engineering instruction change, so does the engineeringeducation research agenda. Further worth noting are rising pressures to relate research to practice,as reflected in mandates to identify the “broader impacts” associated with our scholarly work.This paper speaks to these challenges through the lens of our team’s recent experiences workingon an NSF-supported research project, one primary objective of which is to develop and validatea global competency assessment tool. This paper is organized in three major parts, each focusedon a different phase of the project and a variety of associated questions and challenges. The firstsection is focused on efforts to clarify our main domain of interest, including by developing aworking definition of “global engineering competency” that is grounded in both the extantliterature and our own qualitative data set. This part of the paper is particularly concerned withhow we meaningfully delineate, define, and study specific domains or facets of engineeringeducation and practice, including by working toward more nuanced interpretations of keyattributes and capabilities important for current and future engineers. The second section of thepaper describes our efforts to develop a large pool of assessment questions using a situationaljudgment test (SJT) format. Here we interrogate our rationale for picking this type of assessmenttool, procedures for developing the scenario-based test questions, and efforts to bolster validitythrough use of an underlying theoretical framework and systematic item development procedures.Third and finally, we discuss our ongoing efforts to validate our large pool of SJT questions,with an emphasis on data collection bottlenecks and questions about what counts as validation indifferent academic fields. In this paper we also explore how we are working to maximize theimpacts of our assessment questions by building an online survey and assessment system.The major goal of this paper is to make visible and critically reflect on key methodological issuesand decisions that are often absent or simplified in more conventional research papers. Ourcommentary will likely be of interest for engineering education researchers who wish to explorecontemporary methodological concerns that cut across both qualitative and quantitative researchtraditions. This paper may also be of interest to scholars involved with ongoing efforts to define,develop, and/or assess complex, multi-faceted learning outcomes such as global competency,innovation, teamwork, leadership, inter/trans/multi-disciplinarity, engineering design, etc.
Jesiek, B. K., & Woo, S. E., & Zhu, Q., & Ramane, K. D., & Choudhary, N. (2015, June), Defining and Assessing Global Engineering Competency: Methodological Reflections Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23781
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015