Evolving a Rubric for Use in Assessing Engineering Graduate Attributes in a Student Senior Research Thesis

Alan Chong; Lisa Romkey

Download Paper | Permalink

Conference: 2012 ASEE Annual Conference & Exposition
Location: San Antonio, Texas
Publication Date: June 10, 2012
Start Date: June 10, 2012
End Date: June 13, 2012
ISSN: 2153-5965
Conference Session: Research in Assessment
Tagged Division: Educational Research and Methods
Page Count: 24
Page Numbers: 25.594.1 - 25.594.24
DOI: 10.18260/1-2--21351
Permanent URL: https://peer.asee.org/21351
Download Count: 477

Request a correction

Paper Authors

biography

Alan Chong University of Toronto

visit author page

Alan Chong is a Senior Lecturer in the Engineering Communication program at the University of Toronto, housed in the Faculty of Applied Science and Engineering, where he teaches technical communication to undergraduate engineering students. He has spent the last five years working with engineering faculty to conduct research on and develop integrated courses in engineering design, research and communication, focusing on designing tools for better assessment and instruction, and improving students' critical thinking skills.

visit author page

biography

Lisa Romkey University of Toronto

visit author page

Lisa Romkey serves as Senior Lecturer, Curriculum, Teaching and Learning with the Division of Engineering Science at the University of Toronto. In this position, Romkey plays a central role in the evaluation, design and delivery of a dynamic and complex curriculum, while facilitating the development and implementation of various teaching and learning initiatives. Romkey is cross-appointed with the Department of Curriculum, Teaching, and Learning at OISE/UT, and teaches undergraduate courses in engineering and society, and graduate courses in engineering education. Romkey’s current doctoral research focuses on teaching practices in engineering and the integration of an STSE (science, technology, society, and the environment) approach. She has also conducted research on teacher education, the first year university experience, peer teaching, gender issues in science and engineering, and graduate attribute assessment.

visit author page

Download Paper | Permalink

Abstract

Evolving a Rubric for Use in Assessing Engineering Graduate Attributes in a Student Senior Research ThesisRecently, changes to the Canadian Engineering Accreditation requirements, following theexample set by the ABET, have called for the measurement of 12 graduate attributes in theengineering curriculum. Some attributes, such as “Knowledge Base,” lend themselves toquantitative measurement; others, such as “Investigation” and “Communication” are inherentlydifficult to measure quantitatively and comprehensively. For these attributes, a secondarychallenge involves identifying an appropriate experience in which to measure them.This paper describes the process of developing and utilizing a rubric for graduate attributesassessment in a large senior research thesis course in a multidisciplinary engineering program.The thesis is designed to provide students with opportunity to conduct, document, andexperience engineering related research as an undergraduate student. Every year, nearly 200students work with over 100 supervisors from across several academic departments at theuniversity. Given that thesis is a requirement for all senior students in the program, and that itserves as a capstone experience, capitalizing on students’ earlier work in engineeringcommunication and experimentation, it stands as an ideal place in which to measure students’skills in these areas at graduation.However, the variability of these projects presents significant challenges for common rubricdevelopment and by implication, our ability to retrieve accurate data on student performance inthese categories/attributes. This variability also brings unique challenges to the development of asingle rubric that is 1) flexible enough to apply to a variety of engineering thesis projects, 2)reflective of the learning objectives of the thesis course, and also 3) appropriate for use ingathering data about students’ graduate attributes.This paper describes the evolution of the rubric, from an initial model, designed to provideflexibility to a diverse group of projects and supervisors, to a revised model, intended to alsoserve the needs of the graduate attribute reporting, including the provision of more specific levelsof criteria to demonstrate how a cohort performs across the various indicators that comprise anattribute. The challenges found in mapping the graduate attributes to an existing rubric, and thetensions present in promoting the use of a rubric for both authentic student assessment in a largeand complex undergraduate learning experience, and graduate attributes mapping, will beexplored. Despite these tensions, the results provided by this process provide insight about therubric design, supervisors’ assessment strategies and the students’ strengths and weaknesseswithin the two graduate attributes, providing valuable information to feed back into thecurriculum and thesis experience. Finally, aggregate data gathered from the rubric that allows aquantitative analysis of graduate attributes will be presented, enabling a discussion of the data’sreliability and rubric’s limitations as an accreditation tool.

Citation
Format

Chong, A., & Romkey, L. (2012, June), Evolving a Rubric for Use in Assessing Engineering Graduate Attributes in a Student Senior Research Thesis Paper presented at 2012 ASEE Annual Conference & Exposition, San Antonio, Texas. 10.18260/1-2--21351

TY  - CPAPER
AB  -       Evolving a Rubric for Use in Assessing Engineering Graduate             Attributes in a Student Senior Research ThesisRecently, changes to the Canadian Engineering Accreditation requirements, following theexample set by the ABET, have called for the measurement of 12 graduate attributes in theengineering curriculum. Some attributes, such as “Knowledge Base,” lend themselves toquantitative measurement; others, such as “Investigation” and “Communication” are inherentlydifficult to measure quantitatively and comprehensively. For these attributes, a secondarychallenge involves identifying an appropriate experience in which to measure them.This paper describes the process of developing and utilizing a rubric for graduate attributesassessment in a large senior research thesis course in a multidisciplinary engineering program.The thesis is designed to provide students with opportunity to conduct, document, andexperience engineering related research as an undergraduate student. Every year, nearly 200students work with over 100 supervisors from across several academic departments at theuniversity. Given that thesis is a requirement for all senior students in the program, and that itserves as a capstone experience, capitalizing on students’ earlier work in engineeringcommunication and experimentation, it stands as an ideal place in which to measure students’skills in these areas at graduation.However, the variability of these projects presents significant challenges for common rubricdevelopment and by implication, our ability to retrieve accurate data on student performance inthese categories/attributes. This variability also brings unique challenges to the development of asingle rubric that is 1) flexible enough to apply to a variety of engineering thesis projects, 2)reflective of the learning objectives of the thesis course, and also 3) appropriate for use ingathering data about students’ graduate attributes.This paper describes the evolution of the rubric, from an initial model, designed to provideflexibility to a diverse group of projects and supervisors, to a revised model, intended to alsoserve the needs of the graduate attribute reporting, including the provision of more specific levelsof criteria to demonstrate how a cohort performs across the various indicators that comprise anattribute. The challenges found in mapping the graduate attributes to an existing rubric, and thetensions present in promoting the use of a rubric for both authentic student assessment in a largeand complex undergraduate learning experience, and graduate attributes mapping, will beexplored. Despite these tensions, the results provided by this process provide insight about therubric design, supervisors’ assessment strategies and the students’ strengths and weaknesseswithin the two graduate attributes, providing valuable information to feed back into thecurriculum and thesis experience. Finally, aggregate data gathered from the rubric that allows aquantitative analysis of graduate attributes will be presented, enabling a discussion of the data’sreliability and rubric’s limitations as an accreditation tool.
AU  - Alan Chong
AU  - Lisa Romkey
CY  - San Antonio, Texas
DA  - 2012/06/10
PB  - ASEE Conferences
TI  - Evolving a Rubric for Use in Assessing Engineering Graduate Attributes in a Student Senior Research Thesis
UR  - https://peer.asee.org/21351
DO  - 10.18260/1-2--21351
ER  -