Lindsay, E., & Sabet Jahromi, M. N. (2023, June), The development of an artificial intelligence classifier to automate assessment in large class settings: Preliminary results  Paper presented at 2023 ASEE Annual Conference & Exposition, Baltimore , Maryland. 10.18260/1-2--44085

\bibitem{asee_peer_44085}
  Lindsay, E., \& Sabet Jahromi, M. N. (2023, June), \emph{The development of an artificial intelligence classifier to automate assessment in large class settings: Preliminary results}
  Paper presented at 2023 ASEE Annual Conference \& Exposition, Baltimore , Maryland.
  10.18260/1-2--44085

Euan Lindsay and Mohammad Naser Sabet Jahromi.  "The development of an artificial intelligence classifier to automate assessment in large class settings: Preliminary results".  2023 ASEE Annual Conference & Exposition, Baltimore , Maryland, 2023, June.  ASEE Conferences, 2023.  https://peer.asee.org/44085 Internet.  04 Dec, 2024

\bibitem{asee_peer_44085}
  Euan Lindsay and Mohammad Naser Sabet Jahromi.
  "The development of an artificial intelligence classifier to automate assessment in large class settings: Preliminary results".
  \emph{2023 ASEE Annual Conference \& Exposition, Baltimore , Maryland, 2023, June}.
  ASEE Conferences, 2023.
  https://peer.asee.org/44085 Internet.  04 Dec, 2024

@INPROCEEDINGS{asee_peer_44085
author = "Euan Lindsay and Mohammad Naser Sabet Jahromi"
title = "The development of an artificial intelligence classifier to automate assessment in large class settings: Preliminary results"
booktitle = "2023 ASEE Annual Conference \& Exposition"
year = "2023"
month = "June"
address = "Baltimore , Maryland"
publisher = "ASEE Conferences"
note = {https://peer.asee.org/44085}
number = {10.18260/1-2--44085}
}

TY  - CPAPER
AB  - This evidence based practice paper presents preliminary results in using an artificial intelligence classifier to mark student assignments in a large class setting.  The assessment task consists of an approximately 2000 word reflective essay that is produced under examination conditions and submitted electronically.  The marking is a simple pass/fail determination, and no explicit feedback beyond the pass/fail grade is provided to the students.  Each year around 1500 students complete this assignment, which places a significant and time-constrained marking load upon the teaching faculty.
This paper presents a Natural Language Process (NLP) framework/tool for developing a machine learning based binary classifier for automated assessment of these assignments.  The classifier allocates each assignment a score representing the probability that the assignment would receive a passing grade from a human marker.  The effectiveness and performance of the classifier is measured by investigating the accuracy of those predictions.
Several iterations and statistical analyses were carried out to determine operational thresholds that balance the risks of false positives and false negatives with the required quantity of human marking to assess the assignment.
The resulting classifier was able to provide accuracy levels that are potentially feasible in an operational context, and the potential for significant overall reductions in the human marking load for this assignment.

AU  - Euan Lindsay
AU  - Mohammad Naser Sabet Jahromi
CY  - Baltimore , Maryland
DA  - 2023/06/25
PB  - ASEE Conferences
TI  - The development of an artificial intelligence classifier to automate assessment in large class settings: Preliminary results
UR  - https://peer.asee.org/44085
DO  - 10.18260/1-2--44085
ER  -