Graduate Student Self and Adviser Ratings on Professional Competencies

Bret Austin Arnold; Alison J. Kerr; Bradley J. Brummel; Michael W. Keller

Download Paper | Permalink

Conference: 2018 ASEE Annual Conference & Exposition
Location: Salt Lake City, Utah
Publication Date: June 23, 2018
Start Date: June 23, 2018
End Date: July 27, 2018
Conference Session: Developing Teaching and Mentoring Skills
Tagged Division: Graduate Studies
Page Count: 13
DOI: 10.18260/1-2--30563
Permanent URL: https://peer.asee.org/30563
Download Count: 385

Request a correction

Paper Authors

biography

Bret Austin Arnold University of Tulsa

visit author page

Bret is a doctoral student of Industrial and Organizational Psychology at the University of Tulsa. His recent projects concern how personality shapes team-related behaviors and the degree to which antecedents of workplace burnout differ across cultures. Most recently, Bret has joined the University of Tulsa's cross-disciplinary STEM ProDev team. The team has recently designed and piloted a training program that develops the professional soft-skills of graduate engineering students.

visit author page

biography

Alison J. Kerr University of Tulsa

visit author page

Alison Kerr is a graduate student at The University of Tulsa. She is pursuing a doctoral degree in Industrial-Organizational Psychology. Her research interests include training development and evaluation as explored across a variety of academic disciplines and organizational settings. She is currently assisting on a number of training projects aimed at developing engineering students on relevant non-technical professional skills including ethical practice and presentation.

visit author page

biography

Bradley J. Brummel University of Tulsa

visit author page

Dr. Brummel is an Associate Professor of Industrial/Organizational Psychology at The University of Tulsa. He received his PhD from the University of Illinois at Urbana-Champaign. He conducts research on training and development with a specific focus on professional development, ethics, and coaching.

visit author page

biography

Michael W. Keller University of Tulsa

visit author page

Michael Keller is an associate professor of mechanical engineering at the university of tulsa. His research and teaching interests are in solid mechanics, both experimental and theoretical, and materials science.

visit author page

Download Paper | Permalink

Abstract

Professional societies, government agencies, and other organizations have issued numerous calls to strengthen the engineering work force. However, this declaration of need does not answer the question of how. One possible direction focuses on improving the field relevant, but non-technical skills that help engineers as professionals. The authors of this paper have recently completed the first round of a pilot program for a professional development program for graduate students in engineering fields. As a part of the program, students compare their own ratings on these competencies to those from their advisor and selected peers. The participants and advisors then discuss results of the initial assessment together and create a development plan based on those results.

As part of this program, nine engineering graduate students participated in an initial rating assessment of their current performance in nine identified areas of professional competency. The competencies are: Creativity, Planning, Problem Solving, Oral Communication, Written Communication, Conflict Management, Cultural Adaptability, and Teamwork. These competencies were selected by industry professionals, graduate students, and the departmental advisory board as relevant skills which are present among high performing, professional engineers. Ratings for each competency were on a 9-point, behaviorally anchored rating scale ranging from Growth Area to Area of Strength. The rating form also allowed for rater commentary and feedback on each competency to support and justify the ratings.

This paper presents the results of this initial assessment as a form of training needs analysis by illustrating where graduate students believe that they currently stand in their professional development in these areas. This data can inform advisors regarding individual student needs and where to direct development efforts. This paper seeks to answer two questions. The first is a question of current standing. The results of these assessments help indicate the non-technical competencies in which graduate students are most and least skilled. The second question concerns how competency ratings vary depending on who is evaluating the graduate student’s capabilities.

According to peer and supervisor ratings, students were most competent in Teamwork and Cultural Adaptability but received their lowest scores in Leadership and Creativity. The largest discrepancy between self-ratings and supervisor ratings was on the Teamwork competency. Here, peers rated themselves with scores more consistent with those provided by their peers, but advisors provided ratings that, on average, were considerably higher in comparison. This may indicate a failure of advisors to observe certain aspects of Teamwork performance.

The paper reports general student standing on the competencies and how this knowledge can guide developmental goals for students in the program. The rater discrepancies can provide developmental insights as well. For example, it may indicate that ratees demonstrate competence in some settings but less so in others. By setting goals with their advisors and leveraging professionally created development tools, students can build on their lower rated competencies. Additionally, exploring the areas of greatest differences underscores the importance of receiving feedback from multiple sources rather than advisors alone.

Citation
Format

Arnold, B. A., & Kerr, A. J., & Brummel, B. J., & Keller, M. W. (2018, June), Graduate Student Self and Adviser Ratings on Professional Competencies Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. 10.18260/1-2--30563

TY - CPAPER
AB - Professional societies, government agencies, and other organizations have issued numerous calls to strengthen the engineering work force. However, this declaration of need does not answer the question of how. One possible direction focuses on improving the field relevant, but non-technical skills that help engineers as professionals. The authors of this paper have recently completed the first round of a pilot program for a professional development program for graduate students in engineering fields. As a part of the program, students compare their own ratings on these competencies to those from their advisor and selected peers. The participants and advisors then discuss results of the initial assessment together and create a development plan based on those results.

As part of this program, nine engineering graduate students participated in an initial rating assessment of their current performance in nine identified areas of professional competency. The competencies are: Creativity, Planning, Problem Solving, Oral Communication, Written Communication, Conflict Management, Cultural Adaptability, and Teamwork. These competencies were selected by industry professionals, graduate students, and the departmental advisory board as relevant skills which are present among high performing, professional engineers. Ratings for each competency were on a 9-point, behaviorally anchored rating scale ranging from Growth Area to Area of Strength. The rating form also allowed for rater commentary and feedback on each competency to support and justify the ratings.

This paper presents the results of this initial assessment as a form of training needs analysis by illustrating where graduate students believe that they currently stand in their professional development in these areas. This data can inform advisors regarding individual student needs and where to direct development efforts. This paper seeks to answer two questions. The first is a question of current standing. The results of these assessments help indicate the non-technical competencies in which graduate students are most and least skilled. The second question concerns how competency ratings vary depending on who is evaluating the graduate student’s capabilities.

According to peer and supervisor ratings, students were most competent in Teamwork and Cultural Adaptability but received their lowest scores in Leadership and Creativity. The largest discrepancy between self-ratings and supervisor ratings was on the Teamwork competency. Here, peers rated themselves with scores more consistent with those provided by their peers, but advisors provided ratings that, on average, were considerably higher in comparison. This may indicate a failure of advisors to observe certain aspects of Teamwork performance.

The paper reports general student standing on the competencies and how this knowledge can guide developmental goals for students in the program. The rater discrepancies can provide developmental insights as well. For example, it may indicate that ratees demonstrate competence in some settings but less so in others. By setting goals with their advisors and leveraging professionally created development tools, students can build on their lower rated competencies. Additionally, exploring the areas of greatest differences underscores the importance of receiving feedback from multiple sources rather than advisors alone.

AU - Bret Austin Arnold
AU - Alison J. Kerr
AU - Bradley J. Brummel
AU - Michael W. Keller
CY - Salt Lake City, Utah
DA - 2018/06/23
PB - ASEE Conferences
TI - Graduate Student Self and Adviser Ratings on Professional Competencies
UR - https://peer.asee.org/30563
DO - 10.18260/1-2--30563
ER -