BOARD # 470: Work in Progress: RIEF - An Observational Study of Student Question-Prompted Discussion to Identify Student’s Knowledge Level

Sarah M Johnston; Thien Ngoc Y Ta; Ryan James Milcarek; Samantha Ruth Brunhaver; Karl A Smith; Gary Lichtenstein

Download Paper | Permalink

Conference: 2025 ASEE Annual Conference & Exposition
Location: Montreal, Quebec, Canada
Publication Date: June 22, 2025
Start Date: June 22, 2025
End Date: August 15, 2025
Conference Session: NSF Grantees Poster Session II
Tagged Topic: NSF Grantees Poster Session
Page Count: 6
Permanent URL: https://peer.asee.org/55853

She is a doctoral student of Engineering Education Systems and Design at a U.S. university at the Southwest. She has been working as a research associate for a project of the Kern Family Foundation at this university. She has taught for a technical colleg

visit author page

biography

Ryan James Milcarek Arizona State University

visit author page

Ryan Milcarek obtained his B.S., M.S. and Ph.D. in the Mechanical & Aerospace Engineering Department at Syracuse University. He also obtained a M.S. in Energy Systems Engineering, Certificate of Advanced Study in Sustainable Enterprise and Certificate in

visit author page

biography

Samantha Ruth Brunhaver Arizona State University, Polytechnic Campus orcid.org/0000-0001-8607-5959

visit author page

Samantha Brunhaver is an Assistant Professor of Engineering in the Fulton Schools of Engineering Polytechnic School. Dr. Brunhaver recently joined Arizona State after completing her M.S. and Ph.D. in Mechanical Engineering at Stanford University. She also

visit author page

biography

Karl A Smith University of Minnesota - Twin Cities

visit author page

Emeritus Professor of Civil, Environmental, and Geo- Engineering, Morse-Alumni Distinguished University Teaching Professor, Faculty Member, Technological Leadership Institute at the University of Minnesota; and Cooperative Learning Professor of Engineerin

visit author page

biography

Gary Lichtenstein Arizona State University

visit author page

Gary Lichtenstein, Ed.D., is founder and principal of Quality Evaluation Designs, a firm specializing in education research and program evaluation.

visit author page

Download Paper | Permalink

Abstract

This work in progress identifies student levels of knowledge achieved during a Peer-Led Study Group (PLSG) intervention within a thermodynamics course at the institution. Following a study of student cognitive level of processing, this study continues an observational study using Bloom’s Revised Taxonomy as a methodological basis [1]. Specifically, this work aims to answer 1) What levels of knowledge are observed within question-prompted student discussions in PLSGs? and 2) To what extent do these knowledge levels observed impact student pass-rates and/or final grades?

Using a recently developed observational protocol, student question-prompted discussions were organized using Bloom’s Revised Taxonomy as a basis [1]. The Revised Bloom’s Taxonomy model is a tool for identifying student expected applications and understanding of course content through the knowledge and cognitive levels, respectively [2, 3]. Specifically, the knowledge dimension is organized into factual, conceptual, procedural, and metacognitive levels; these correspond to recalling basic information, connecting concepts, solving using course techniques and methodologies, and self-awareness of cognition, respectively [2]. Student-prompted discussions were recorded weekly during 20-minute observation periods occurring within 50-minute recitations over a 13-week course. The eight selected groups were observed for an average of 10 weeks. All groups within the course were organized primarily by Beginning of Term (BoT) GPA, taking demographics such as race, ethnicity, and gender differences into account as a secondary factor. In addition, First-Time Full-Time (FTFT) and Transfer (TRN) student admission statuses were used as a point of group organization where applicable. The eight observed groups were selected based on the available student population enrolled in the course and the demographics of each group. In the cognitive processing level research, students question-prompted discussions were categorized based on 1-3 associated verbs [1]. These verbs corresponded to the remember, understand, apply, analyze, evaluate, and create levels of the Revised Bloom’s Taxonomy model [2].

Results from the study show a range of cognitive processing was observed. The most common level observed was the factual level of the knowledge dimension, with the “state” verb as the most utilized. However, there are several instances of student question-prompted discussions reaching as high as the procedural level of the knowledge dimension. Of the 1,300 question-prompted discussions recorded, the most prevalent cognitive levels observed were remember and understand, which made up ~38% and ~15% of discussions, respectively. Of the remember level, the factual knowledge level makes up a majority of the data, with procedural knowledge frequently being discussed from the understand level.

Citation
Format

Johnston, S. M., & Ta, T. N. Y., & Milcarek, R. J., & Brunhaver, S. R., & Smith, K. A., & Lichtenstein, G. (2025, June), BOARD # 470: Work in Progress: RIEF - An Observational Study of Student Question-Prompted Discussion to Identify Student’s Knowledge Level Paper presented at 2025 ASEE Annual Conference & Exposition , Montreal, Quebec, Canada . https://peer.asee.org/55853

TY - CPAPER
AB - This work in progress identifies student levels of knowledge achieved during a Peer-Led Study Group (PLSG) intervention within a thermodynamics course at the institution. Following a study of student cognitive level of processing, this study continues an observational study using Bloom’s Revised Taxonomy as a methodological basis [1]. Specifically, this work aims to answer 1) What levels of knowledge are observed within question-prompted student discussions in PLSGs? and 2) To what extent do these knowledge levels observed impact student pass-rates and/or final grades?

Using a recently developed observational protocol, student question-prompted discussions were organized using Bloom’s Revised Taxonomy as a basis [1]. The Revised Bloom’s Taxonomy model is a tool for identifying student expected applications and understanding of course content through the knowledge and cognitive levels, respectively [2, 3]. Specifically, the knowledge dimension is organized into factual, conceptual, procedural, and metacognitive levels; these correspond to recalling basic information, connecting concepts, solving using course techniques and methodologies, and self-awareness of cognition, respectively [2]. Student-prompted discussions were recorded weekly during 20-minute observation periods occurring within 50-minute recitations over a 13-week course. The eight selected groups were observed for an average of 10 weeks. All groups within the course were organized primarily by Beginning of Term (BoT) GPA, taking demographics such as race, ethnicity, and gender differences into account as a secondary factor. In addition, First-Time Full-Time (FTFT) and Transfer (TRN) student admission statuses were used as a point of group organization where applicable. The eight observed groups were selected based on the available student population enrolled in the course and the demographics of each group. In the cognitive processing level research, students question-prompted discussions were categorized based on 1-3 associated verbs [1]. These verbs corresponded to the remember, understand, apply, analyze, evaluate, and create levels of the Revised Bloom’s Taxonomy model [2].

Results from the study show a range of cognitive processing was observed. The most common level observed was the factual level of the knowledge dimension, with the “state” verb as the most utilized. However, there are several instances of student question-prompted discussions reaching as high as the procedural level of the knowledge dimension. Of the 1,300 question-prompted discussions recorded, the most prevalent cognitive levels observed were remember and understand, which made up ~38% and ~15% of discussions, respectively. Of the remember level, the factual knowledge level makes up a majority of the data, with procedural knowledge frequently being discussed from the understand level.

AU - Sarah M Johnston
AU - Thien Ngoc Y Ta
AU - Ryan James Milcarek
AU - Samantha Ruth Brunhaver
AU - Karl A Smith
AU - Gary Lichtenstein
CY - Montreal, Quebec, Canada
DA - 2025/06/22
PB - ASEE Conferences
TI - BOARD # 470: Work in Progress: RIEF - An Observational Study of Student Question-Prompted Discussion to Identify Student’s Knowledge Level
UR - https://peer.asee.org/55853
ER -