AI-Based Concept Inventories: Using Cognitive Diagnostic Computer Adaptive Testing in LASSO for Classroom Assessment

Jason Morphew; Amirreza Mehrabi; Ben Van Dusen; Jayson Nissen

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Engineering Physics and Physics Division (EP2D) Technical Session 1
Tagged Division: Engineering Physics and Physics Division (EP2D)
Permanent URL: https://peer.asee.org/46534

Request a correction

Paper Authors

biography

Jason Morphew Purdue University orcid.org/0000-0001-5971-214X

visit author page

Jason W. Morphew is an Assistant Professor in the School of Engineering Education at Purdue University. He earned a B.S. in Science Education from the University of Nebraska and spent 11 years teaching math and science at the middle school, high school, and community college level. He earned a M.A. in Educational Psychology from Wichita State and a Ph.D. from the University of Illinois Urbana-Champaign.

visit author page

biography

Amirreza Mehrabi Purdue Engineering Education

visit author page

I am Amirreza Mehrabi, a Ph.D. student in Engineering Education at Purdue University, West Lafayette. Now I am working in computer adaptive testing (CAT) enhancement with AI and analyzing big data with machine learning (ML) under Prof. J. W. Morphew at the ENE department. My master's was in engineering education at UNESCO chair on Engineering Education at the University of Tehran. I pursue Human adaptation to technology and modeling human behavior(with machine learning and cognitive research). My background is in Industrial Engineering (B.Sc. at the Sharif University of Technology and "Gold medal" of Industrial Engineering Olympiad (Iran-2021- the highest-level prize in Iran)). Now I am working as a researcher in the Erasmus project, which is funded by European Unions (1M $_European Union & 7 Iranian Universities) which focus on TEL and students as well as professors' adoption of technology(modern Education technology). Moreover, I cooperated with Dr. Taheri to write the "R application in Engineering statistics" (an attachment of his new book "Engineering probability and statistics.")

visit author page

author page

Ben Van Dusen Iowa State University of Science and Technology

author page

Jayson Nissen

Download Paper | Permalink

Abstract

Research-based concept inventories, such as the Force Concept Inventory, the Force and Motion Conceptual Evaluation, or the Energy and Momentum Conceptual Survey have played a pivotal role in designing and evaluating instruction in introductory physics courses. While these concept inventories have helped identify inequity in course instruction and have led to improved pedagogical methods, concerns about their ability to provide actionable feedback for formative assessment remain. In addition, using the same questions for both a pre- and a post- assessment leaves the potential for students learning the correct answers without undergoing conceptual change. To address these issues, we designed a Computerized Adaptive Testing (CAT) platform that utilizes Cognitive Diagnostic Modeling (CDM) for delivering concept inventories through the Learning About STEM Student Outcomes (LASSO) platform. CAT is a modern approach to educational technology that can transform classroom assessment and self-assessment strategies. CAT selects questions using item difficulty and item discrimination to estimate student ability. By selecting questions at an appropriate difficulty for each student, CAT can assess student conceptual understanding with greater accuracy. The addition of CDM allows for an assessment of skill mastery across concept and provides actionable information to instructors to provide individualized instruction even within large-enrollment introductory physics courses. LASSO serves as a centralized platform enabling classes nationwide to access a diverse array of assessment contents and questions aligning with established educational standards, promoting frequent assessment. In this presentation, we describe the design of Mechanics Cognitive Diagnostic (MCD) that uses CAT and CDM and is delivered within the LASSO platform. We also present the results from simulation studies that assessed the ability of the MCD to assess conceptual understanding while maintaining test security and decreasing test length. In addition, we present the results from analyses of student data to identify the underlying skills within conceptual inventories. The amalgamation of CAT with cognitive diagnosis models within the LASSO platform empowers educators to gauge student mastery levels and confidently navigate the subsequent stages of the teaching process.

Citation
Format

Morphew, J., & Mehrabi, A., & Van Dusen, B., & Nissen, J. (2024, June), AI-Based Concept Inventories: Using Cognitive Diagnostic Computer Adaptive Testing in LASSO for Classroom Assessment Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/46534

TY  - CPAPER
AB  - Research-based concept inventories, such as the Force Concept Inventory, the Force and Motion Conceptual Evaluation, or the Energy and Momentum Conceptual Survey have played a pivotal role in designing and evaluating instruction in introductory physics courses. While these concept inventories have helped identify inequity in course instruction and have led to improved pedagogical methods, concerns about their ability to provide actionable feedback for formative assessment remain. In addition, using the same questions for both a pre- and a post- assessment leaves the potential for students learning the correct answers without undergoing conceptual change. To address these issues, we designed a Computerized Adaptive Testing (CAT) platform that utilizes Cognitive Diagnostic Modeling (CDM) for delivering concept inventories through the Learning About STEM Student Outcomes (LASSO) platform. CAT is a modern approach to educational technology that can transform classroom assessment and self-assessment strategies. CAT selects questions using item difficulty and item discrimination to estimate student ability. By selecting questions at an appropriate difficulty for each student, CAT can assess student conceptual understanding with greater accuracy. The addition of CDM allows for an assessment of skill mastery across concept and provides actionable information to instructors to provide individualized instruction even within large-enrollment introductory physics courses. LASSO serves as a centralized platform enabling classes nationwide to access a diverse array of assessment contents and questions aligning with established educational standards, promoting frequent assessment. In this presentation, we describe the design of Mechanics Cognitive Diagnostic (MCD) that uses CAT and CDM and is delivered within the LASSO platform. We also present the results from simulation studies that assessed the ability of the MCD to assess conceptual understanding while maintaining test security and decreasing test length. In addition, we present the results from analyses of student data to identify the underlying skills within conceptual inventories. The amalgamation of CAT with cognitive diagnosis models within the LASSO platform empowers educators to gauge student mastery levels and confidently navigate the subsequent stages of the teaching process. 
AU  - Jason Morphew
AU  - Amirreza Mehrabi
AU  - Ben Van Dusen
AU  - Jayson Nissen
CY  - Portland, Oregon
DA  - 2024/06/23
PB  - ASEE Conferences
TI  - AI-Based Concept Inventories: Using Cognitive Diagnostic Computer Adaptive Testing in LASSO for Classroom Assessment
UR  - https://peer.asee.org/46534
ER  -