Successful Use of Performance Indicators to Assess Student Outcomes

Daniel K. Jones; Mohammed Abdallah

Download Paper | Permalink

Conference: 2016 ASEE Annual Conference & Exposition
Location: New Orleans, Louisiana
Publication Date: June 26, 2016
Start Date: June 26, 2016
End Date: June 29, 2016
ISBN: 978-0-692-68565-5
ISSN: 2153-5965
Conference Session: ETAC/ABET-Related Issues
Tagged Division: Engineering Technology
Page Count: 9
DOI: 10.18260/p.25961
Permanent URL: https://peer.asee.org/25961
Download Count: 2017

Request a correction

Paper Authors

biography

Daniel K. Jones P.E. SUNY Polytechnic Institute

visit author page

Dr. Jones is Associate Professor of Mechanical Engineering Technology and Chair of the Engineering Technology Department at SUNYIT in Utica, NY. He coordinates the ETAC/ABET accrediations processes. His technical interests include machine design, mechancial measurements, vibrations, instrumentation, and assistive technology for people with physical disabilities.

visit author page

author page

Mohammed Abdallah SUNY Polytechnic Institute

Download Paper | Permalink

Abstract

Well-written Student Outcomes (SOs) are a vital part of a successful improvement process. However, the SOs are relatively broad statements on what students are expected to know. Performance Indicators (PIs) provide more specific actions that may be used for direct measurement of SOs, and they are useful tools for assessing the degree to which students successfully achieve subsets of each SO. During a recent reaccreditation by ETAC/ABET, several engineering technology programs demonstrated successful use of PIs for outcomes assessment and improvement processes.

Rubrics have been developed as tools to provide direct measurement of student performance in each of the SOs. The rubrics were designed to be used primarily in upper-level courses that were well-aligned with the student outcomes. Instructors selected student work representative of a particular SO in their course. The selected work depended on the type of course, and it typically included items such as oral presentations, written lab reports, or problem solutions from exams, quizzes, or homework assignments. It was most effective to complete rubric scores for student work while grading or as soon as possible afterward.

The precise wording of each PI was central to the successful use of the rubrics. Each rubric was limited to one page with three to five concise PIs that captured the vital aspects of the SO. Proper selection of the verbs in each PI was a very important aspect of defining the expectation of students. Each PI was evaluated with performance levels on a scale of one to four: 1 – Not acceptable, 2 – Below standards, 3 – Meets standards, 4- Exemplary. This simplified scale helped to maintain consistency among instructors, and it forced a decision between acceptable (meets standards) and unacceptable (below standards) performance. Each performance level contained a brief, thorough description of the expectations, clarifying the differences between the levels. The intent was to provide enough detail to distinguish between levels, while giving flexibility for use in evaluating student work in different projects and courses.

The total number of students, as well as the percentage of students, scoring 4, 3, 2, and 1 was used to evaluate aggregate performance of the group. Data from students not passing a course was not included; since they needed to retake the course, assessment data was collected when they passed. An initial benchmark was to have 70% of students scoring 3 or 4, indicating that at least 70% of the students met or exceeded acceptable standards. If less than 70% of students scored 3 or 4, overall student performance was below the benchmark, indicating potential for improvement in that particular PI. After obtaining baseline data from an initial evaluation, the 70% benchmark could then be changed, if appropriate. As the assessment process evolved, different SOs had different benchmarks to reflect the level of difficulty in the specific assessment tool.

Citation
Format

Jones, D. K., & Abdallah, M. (2016, June), Successful Use of Performance Indicators to Assess Student Outcomes Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. 10.18260/p.25961

TY - CPAPER
AB - Well-written Student Outcomes (SOs) are a vital part of a successful improvement process. However, the SOs are relatively broad statements on what students are expected to know. Performance Indicators (PIs) provide more specific actions that may be used for direct measurement of SOs, and they are useful tools for assessing the degree to which students successfully achieve subsets of each SO. During a recent reaccreditation by ETAC/ABET, several engineering technology programs demonstrated successful use of PIs for outcomes assessment and improvement processes.

Rubrics have been developed as tools to provide direct measurement of student performance in each of the SOs. The rubrics were designed to be used primarily in upper-level courses that were well-aligned with the student outcomes. Instructors selected student work representative of a particular SO in their course. The selected work depended on the type of course, and it typically included items such as oral presentations, written lab reports, or problem solutions from exams, quizzes, or homework assignments. It was most effective to complete rubric scores for student work while grading or as soon as possible afterward.

The precise wording of each PI was central to the successful use of the rubrics. Each rubric was limited to one page with three to five concise PIs that captured the vital aspects of the SO. Proper selection of the verbs in each PI was a very important aspect of defining the expectation of students. Each PI was evaluated with performance levels on a scale of one to four: 1 – Not acceptable, 2 – Below standards, 3 – Meets standards, 4- Exemplary.
This simplified scale helped to maintain consistency among instructors, and it forced a decision between acceptable (meets standards) and unacceptable (below standards) performance. Each performance level contained a brief, thorough description of the expectations, clarifying the differences between the levels. The intent was to provide enough detail to distinguish between levels, while giving flexibility for use in evaluating student work in different projects and courses.

The total number of students, as well as the percentage of students, scoring 4, 3, 2, and 1 was used to evaluate aggregate performance of the group. Data from students not passing a course was not included; since they needed to retake the course, assessment data was collected when they passed. An initial benchmark was to have 70% of students scoring 3 or 4, indicating that at least 70% of the students met or exceeded acceptable standards. If less than 70% of students scored 3 or 4, overall student performance was below the benchmark, indicating potential for improvement in that particular PI. After obtaining baseline data from an initial evaluation, the 70% benchmark could then be changed, if appropriate. As the assessment process evolved, different SOs had different benchmarks to reflect the level of difficulty in the specific assessment tool.

AU - Daniel K. Jones P.E.
AU - Mohammed Abdallah
CY - New Orleans, Louisiana
DA - 2016/06/26
PB - ASEE Conferences
TI - Successful Use of Performance Indicators to Assess Student Outcomes
UR - https://peer.asee.org/25961
DO - 10.18260/p.25961
ER -