Course Outcome Assessment: Is Using the Average Good Enough?

Phil Dacunto; Andrew Joseph Ng

Download Paper | Permalink

Conference: 2020 ASEE Virtual Annual Conference Content Access
Location: Virtual On line
Publication Date: June 22, 2020
Start Date: June 22, 2020
End Date: June 26, 2021
Conference Session: Innovative Approaches to Improving Student Learning
Tagged Division: Environmental Engineering
Page Count: 27
DOI: 10.18260/1-2--34340
Permanent URL: https://peer.asee.org/34340
Download Count: 535

Paper Authors

biography

Phil Dacunto U.S. Military Academy

visit author page

COL Phil Dacunto is an Associate Professor of Environmental Engineering at the United States Military Academy at West Point, NY. He earned a Ph.D. in the field of environmental engineering at Stanford University in 2013.

visit author page

biography

Andrew Joseph Ng U.S. Military Academy

visit author page

Andrew Ng is a Captain in the United States Army and an Instructor in the Department of Geography and Environmental Engineering at the United States Military Academy. He is a 2010 graduate of the United States Military Academy with a B.S. in Environmental Engineering with honors and a 2019 graduate from the University of California, Berkeley with an M.S. in Civil and Environmental Engineering. He teaches Environmental Engineering for Community Development, Environmental Engineering Technologies, and Environmental Biological Systems.

visit author page

Download Paper | Permalink

Abstract

Assessment of environmental engineering course outcomes is critical for measuring student achievement, evaluating course design, and ultimately assessing programs for ABET accreditation. Often such assessment relies upon the interpretation of results from direct, embedded indicators (graded events such as homework assignments or exams). The simplest approach uses the arithmetic mean of a particular graded event (or set of graded events) to assess the outcome: for example, an average of greater than 80% indicates successful achievement of the outcome. However, such an approach assumes a reasonably normal distribution of student grades, and thus may not be adequately descriptive in some cases. For example, one case where the use of the mean fails is a bimodal distribution of results: if half the students score 100% and half score 60% on a graded event, the resulting mean, 80% (which is generally considered successful) fails to capture the fact that half the students failed. Stoker, Blair, and Sobiesk (2014) proposed a “binning” approach, which accounts for the proportions of students who achieve different levels of success on each assignment; for example, if greater than 5% fail, we assess the outcome to be not successfully achieved, regardless of the overall average. Such an approach, however, is more complicated and time-intensive for faculty to employ. This study investigates the impact of course size (as measured by number of students) on assessment results. Specifically, we present a comparison of course outcome assessment results using both the simpler “mean” and more complicated “binning” approaches for three large (more than 160 students) and two small (less than 35 students) environmental engineering and science courses. The results indicate that the small courses had a greater drop in outcome achievement than the large ones when using the “binning” vs the “mean” approach; however, the "binning" approach could still be useful in these courses with minor modification. In addition, we present some adjustments to the method proposed by Stoker et al. (2014) to make overall outcome assessment more efficient. This study will inform programs as to the conditions under which the more time-intensive “binning” approach is worthwhile, and will enable them to implement the approach more effectively if they so choose.

Ref: Stoker, Geoff; Blair, Jean; and Edward Sobiesk. “Meaningful Assessment.” Proceedings of the 15th Annual Conference on Information Technology Education. Atlanta, GA: 15-18 October, 2014.

Citation
Format

Dacunto, P., & Ng, A. J. (2020, June), Course Outcome Assessment: Is Using the Average Good Enough? Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--34340

TY - CPAPER
AB - Assessment of environmental engineering course outcomes is critical for measuring student achievement, evaluating course design, and ultimately assessing programs for ABET accreditation. Often such assessment relies upon the interpretation of results from direct, embedded indicators (graded events such as homework assignments or exams). The simplest approach uses the arithmetic mean of a particular graded event (or set of graded events) to assess the outcome: for example, an average of greater than 80% indicates successful achievement of the outcome. However, such an approach assumes a reasonably normal distribution of student grades, and thus may not be adequately descriptive in some cases. For example, one case where the use of the mean fails is a bimodal distribution of results: if half the students score 100% and half score 60% on a graded event, the resulting mean, 80% (which is generally considered successful) fails to capture the fact that half the students failed. Stoker, Blair, and Sobiesk (2014) proposed a “binning” approach, which accounts for the proportions of students who achieve different levels of success on each assignment; for example, if greater than 5% fail, we assess the outcome to be not successfully achieved, regardless of the overall average. Such an approach, however, is more complicated and time-intensive for faculty to employ. This study investigates the impact of course size (as measured by number of students) on assessment results. Specifically, we present a comparison of course outcome assessment results using both the simpler “mean” and more complicated “binning” approaches for three large (more than 160 students) and two small (less than 35 students) environmental engineering and science courses. The results indicate that the small courses had a greater drop in outcome achievement than the large ones when using the “binning” vs the “mean” approach; however, the &quot;binning&quot; approach could still be useful in these courses with minor modification. In addition, we present some adjustments to the method proposed by Stoker et al. (2014) to make overall outcome assessment more efficient. This study will inform programs as to the conditions under which the more time-intensive “binning” approach is worthwhile, and will enable them to implement the approach more effectively if they so choose.

Ref: Stoker, Geoff; Blair, Jean; and Edward Sobiesk. “Meaningful Assessment.” Proceedings of the 15th Annual Conference on Information Technology Education. Atlanta, GA: 15-18 October, 2014.

AU - Phil Dacunto
AU - Andrew Joseph Ng
CY - Virtual On line
DA - 2020/06/22
PB - ASEE Conferences
TI - Course Outcome Assessment: Is Using the Average Good Enough?
UR - https://peer.asee.org/34340
DO - 10.18260/1-2--34340
ER -