The Development of an Online Grading System for Distributed Grading in a Large First-year Project-based Design Course

Mary Kathryn Thompson; Beunguk Ahn

Download Paper | Permalink

Conference: 2012 ASEE Annual Conference & Exposition
Location: San Antonio, Texas
Publication Date: June 10, 2012
Start Date: June 10, 2012
End Date: June 13, 2012
ISSN: 2153-5965
Conference Session: Design Tools and Methodology I
Tagged Division: Design in Engineering Education
Page Count: 14
Page Numbers: 25.11.1 - 25.11.14
DOI: 10.18260/1-2--20767
Permanent URL: https://peer.asee.org/20767
Download Count: 843

Request a correction

Paper Authors

biography

Mary Kathryn Thompson Korea Advanced Institute of Science and Technology

visit author page

Mary Kathryn Thompson is an Associate Professor in the Department of Civil and Environmental Engineering at the Korea Advanced Institute of Science and Technology (KAIST). She is engaged in design research at the boundary, both within engineering and beyond, and is actively working on the development and application of formal design theories to civil, environmental, and urban engineering. She is also interested in non-traditional applications of the finite element method, especially for surface phenomena such as fluid sealing, thermal and electrical contact resistance, and friction and wear. Thompson is the Director of the KAIST Freshman Design Program, which earned her both the KAIST Grand Prize for Creative Teaching and the Republic of Korea Ministry of Education, Science and Technology Award for Innovation in Engineering Education in 2009. She also received the Grand Prize at the 2009 International Conference on Axiomatic Design Theory for her paper on the synthesis of formal design theories for traffic intersections. Thompson earned her B.S. (2002), M.S. (2004), and Ph.D. (2007) from the Massachusetts Institute of Technology, Department of Mechanical Engineering.

visit author page

biography

Beunguk Ahn Korea Advanced Institute of Science and Technology

visit author page

Beunguk Ahn is an undergraduate student in the Department of Computer Science at the Korea Advanced Institute of Science and Technology. He is engaged in computer science research related to web content analysis, databases, and data mining. He is also interested in software engineering that integrates values from the humanities and social sciences with computer science. From 2008 to 2011, Ahn served as a teaching assistant and consultant for the KAIST Freshman Design Course. During this time, he helped to set up and run the university’s Moodle e-learning system and developed custom capabilities for the freshman design course. He received an award for enhancing education at KAIST from the university in 2010 and a special award for dedicated service to the KAIST Freshman Design Course in 2011 in recognition for this work.

visit author page

Download Paper | Permalink

Abstract

The Development of an Online Grading System for Distributed Grading in a Large First Year Project-Based Design CourseEnsuring fairness and consistency in grading between sections is a challenge for large project-based design courses. To address this challenge, a distributed grading system was developed fora mandatory first year design subject in a leading technical university. This system requirescourse faculty members and teaching assistants to serve on grading juries to assess the work ofstudents in other sections of the course. Each member of the grading jury evaluates the students'work independently from the other graders using provided grading rubrics. After grading iscomplete, statistical outliers are removed and the remaining scores are averaged. Studentsreceive this averaged grade as their score.The course has between 500 and 600 students per semester organized into roughly 100 teams.Each grading jury has 5 to 6 members and each member serves on six different juries (one ofeach of the mid-term and final deliverables) to reduce the students’ exposure to any one grader.Thus, each semester approximately 3600 sets of scores must be collected, analyzed, averaged,and returned in order to produce the final student grades.This paper will describe an online grading system that was developed to form the grading juriesand collect, process, and return the grades produced by this grading method. In particular, it willdiscuss the design requirements and features of the online grading system from student, faculty,and course administrator perspectives, the implementation of the grading system, the ways inwhich course faculty and staff members interact with the system, and faculty and staff reactionsto the system based on end-of-semester survey results. It will be shown that the online gradingsystem has a number of advantages over analog grading methods, including scalability, real-timefeedback on the status of grading, the reduced potential for human error in compiling grades, theability for jury members to grade remotely and to revise their grades after submission, the abilityfor course administrators to easily review grading results and remove statistical outliers from thescore set, the ability to return both provisional and final grades to the course faculty, staff, andstudents in a timely manner, and the ability to archive and export grading data for future use. Thesystem has also increased the overall quality of the grades produced by the course and hasreceived good feedback from course faculty and staff members who appreciate the opportunity tochoose the projects that they will help to grade and the convenience of the online system.

Citation
Format

Thompson, M. K., & Ahn, B. (2012, June), The Development of an Online Grading System for Distributed Grading in a Large First-year Project-based Design Course Paper presented at 2012 ASEE Annual Conference & Exposition, San Antonio, Texas. 10.18260/1-2--20767

TY  - CPAPER
AB  -          The Development of an Online Grading System for Distributed          Grading in a Large First Year Project-Based Design CourseEnsuring fairness and consistency in grading between sections is a challenge for large project-based design courses. To address this challenge, a distributed grading system was developed fora mandatory first year design subject in a leading technical university. This system requirescourse faculty members and teaching assistants to serve on grading juries to assess the work ofstudents in other sections of the course. Each member of the grading jury evaluates the students&#39;work independently from the other graders using provided grading rubrics. After grading iscomplete, statistical outliers are removed and the remaining scores are averaged. Studentsreceive this averaged grade as their score.The course has between 500 and 600 students per semester organized into roughly 100 teams.Each grading jury has 5 to 6 members and each member serves on six different juries (one ofeach of the mid-term and final deliverables) to reduce the students’ exposure to any one grader.Thus, each semester approximately 3600 sets of scores must be collected, analyzed, averaged,and returned in order to produce the final student grades.This paper will describe an online grading system that was developed to form the grading juriesand collect, process, and return the grades produced by this grading method. In particular, it willdiscuss the design requirements and features of the online grading system from student, faculty,and course administrator perspectives, the implementation of the grading system, the ways inwhich course faculty and staff members interact with the system, and faculty and staff reactionsto the system based on end-of-semester survey results. It will be shown that the online gradingsystem has a number of advantages over analog grading methods, including scalability, real-timefeedback on the status of grading, the reduced potential for human error in compiling grades, theability for jury members to grade remotely and to revise their grades after submission, the abilityfor course administrators to easily review grading results and remove statistical outliers from thescore set, the ability to return both provisional and final grades to the course faculty, staff, andstudents in a timely manner, and the ability to archive and export grading data for future use. Thesystem has also increased the overall quality of the grades produced by the course and hasreceived good feedback from course faculty and staff members who appreciate the opportunity tochoose the projects that they will help to grade and the convenience of the online system.
AU  - Mary Kathryn Thompson
AU  - Beunguk Ahn
CY  - San Antonio, Texas
DA  - 2012/06/10
PB  - ASEE Conferences
TI  -  The Development of an Online Grading System for Distributed Grading in a Large First-year Project-based Design Course
UR  - https://peer.asee.org/20767
DO  - 10.18260/1-2--20767
ER  -