August 23, 2022
June 26, 2022
June 29, 2022
Peer review of scholarship is critical to the advancement of knowledge in a scholarly discipline. Despite this, scholars receive little or no training in effective and constructive peer review. The process of peer review has been routinely criticized in higher education for lack of quality reviews and reviewers, and reviews that are personal and not constructive. As a discipline, engineering education research (EER) benefits from diverse disciplinary backgrounds and perspectives of scholars, and as such relies on peer review of scholarship to generate, interpret, and translate knowledge. Supported by funding through the National Science Foundation, this project is developing, implementing, and assessing a project that conducts training in EER peer review for journal articles and grant proposals. This paper describes the Engineering Education Research Peer Review Training (EER PERT) project, which is designed to develop EER scholars’ peer review skills through mentored reviewing experiences. The overall goals of the EER PERT project are twofold: to establish, assess and evaluate a mentored reviewer program for 1) EER journal manuscripts and 2) EER grant proposals. In the first year of the project, two cohorts have participated in the mentored EER journal manuscript program through partnership with the Journal of Engineering Education (JEE), where triads (two mentees and one mentor) work collaboratively to review three manuscripts. Triads work with coaches informally to provide feedback on their experience and discuss manuscripts they are reviewing. Across the two cohorts, 42 mentees have collaborated with 18 mentors and 3 coaches, with some mentors and coaches participating in both cohorts. Project evaluation activities for the mentored manuscript review program included focus groups and exit surveys with both mentees and mentors in the first cohort. The main findings from the evaluation of the first cohort included the following themes: 1) the program provided valuable training and increased participant’s confidence in conducting EER scholarship, 2) the program fostered a sense of community and inclusion, particularly for those without EER backgrounds and from outside the US, and 3) program logistics could create barriers for participants, including working across time zones and tracking triad progress. Two main changes were implemented for the second cohort, including 1) a program dashboard for teams to track progress and organize update form submissions and 2) an optional monthly discussion meeting to supplement initial program orientation and triad meetings. The monthly discussions are intended to provide additional training in peer review topics and develop community among participants through networking opportunities. Planned discussion topics include updated JEE Author Guidelines and Review Criteria, inclusive language in scholarly publications, anti-racism in peer review, getting involved with the EER community, and roles and processes within the JEE Editorial Board. Program evaluation of the mentored manuscript review program is being leveraged to develop a mentored grant proposal review program wherein participants will work with former NSF program officers to review six grant proposals and participate in a mock review panel. This paper will present an overview of the journal manuscript and proposal mentored reviewer programs and findings from program evaluation.
Jensen, K., & Benson, L., & Watts, K., & Lichtenstein, G., & Ko, E., & Bates, R. (2022, August), Building a Community of Mentors in Engineering Education Research Through Peer Review Training Paper presented at 2022 ASEE Annual Conference & Exposition, Minneapolis, MN. https://peer.asee.org/42028
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2022 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015