Board 428: Work in Progress: An Open Educational Resource to Improve Architectural Engineering Students Conceptual Knowledge When Writing-to-Learn: Investigation 1

Ryan Solnosky; Roy B. Clariana

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: June 26, 2024
Conference Session: NSF Grantees Poster Session
Tagged Topic: NSF Grantees Poster Session
Page Count: 14
DOI: 10.18260/1-2--47018
Permanent URL: https://peer.asee.org/47018
Download Count: 82

Paper Authors

biography

Ryan Solnosky P.E. Pennsylvania State University orcid.org/0000-0002-5214-2102

visit author page

Ryan Solnosky is an Associate Teaching Professor in the Department of Architectural Engineering at The Pennsylvania State University at University Park. Dr. Solnosky has taught courses for Architectural Engineering, Civil Engineering, and Pre-Major Freshmen. He is the recipient of several teaching awards both within Penn State and Nationally. Ryan’s research centers on technology for teaching, capstones, and active learning in design classes.

visit author page

biography

Roy B. Clariana Pennsylvania State University

visit author page

Roy B. Clariana is a professor of Learning Design and Technology in the College of Education, Pennsylvania State University, US. His research areas include measures and models of knowledge structure, natural language processing of texts, automated writing evaluation, and writing to learn.

visit author page

Download Paper | Permalink

Abstract

This investigation is the first of four investigations funded by the NSF (DUE award 2215807) to develop and then field test on open educational browser-based writing-to-learn tool called GIKS. The underlying theory is that writing-to-learn with immediate formative feedback presented as concept networks is engaging and effective for learning concepts covered in lectures. This work was studied in a second year architectural engineering course focusing on building materials, processes and modeling. Participants (n=84) completed a lesson (readings, lecture, and labs) then followed by writing prompts centered on the following topics: Building with Concrete and Wood Construction (3 weeks later). Participants were assigned to one of two counterbalanced groups, group A used GIKS software to write a 300-word summary of the first lesson but did not write in the second lesson, while group B did not write in the first lesson but used GIKS in the second lesson, so that each group served as a control treatment for the other group. All students completed a concept structure survey at the end of each lesson that contained 20 key concepts from that lesson, the two concept structure surveys’ data were transformed into concept networks and then these networks were compared to an expert network benchmark referent, as well as to networks of the textbook chapter and the PowerPoint slides of the related lesson. Then a week after the second lesson students completed the standing end-of-module multiple-choice posttest that included items from these lesson as well as from other lessons in the module. Results to date highlight that for both lessons, the group using GIKS scored higher on the concept structure survey (more like the expert network) BUT lower on the multiple-choice test, the difference was significant for the Building with Concrete lesson (p < .05) but not for the Wood Construction lesson. This interaction has been reported before by Ntshalintshali & Clariana (2020), that improving conceptual knowledge sometimes decreases memory of lesson details. Descriptive analysis of the group-average networks derived from the concept structure surveys for Building with Concrete show that the group-averaged network of those using GIKS compared to the control was more like the expert network (54% vs. 36%), the network of the textbook Chapter (32% vs. 29%), the network of the PowerPoint (PP) (46% vs. 41%), and especially like peers in the other group (67%). For Wood Construction the difference between the groups was less, the group-averaged network of those using GIKS compared to the control was more like the expert (40% vs. 39%), like the light-framed construction PP (28% vs. 24%), and especially like peers in the other group (72%). These findings show that writing-to-learn with GIKS with immediate network feedback improves conceptual knowledge as expected but at the cost of details. Peers conceptual structure of the lesson materials were very similar (peer-peer mental model convergence) and were more like others than like the expert, or the book chapters, or the PowerPoint slides; in addition, the PowerPoint slides appear to influence conceptual structure more than the textbook chapters. Investigation 2 will consider writing-to-learn with or without immediate network feedback in order to isolate the effects of immediate network feedback.

Citation
Format

Solnosky, R., & Clariana, R. B. (2024, June), Board 428: Work in Progress: An Open Educational Resource to Improve Architectural Engineering Students Conceptual Knowledge When Writing-to-Learn: Investigation 1 Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. 10.18260/1-2--47018

TY - CPAPER
AB - This investigation is the first of four investigations funded by the NSF (DUE award 2215807) to develop and then field test on open educational browser-based writing-to-learn tool called GIKS. The underlying theory is that writing-to-learn with immediate formative feedback presented as concept networks is engaging and effective for learning concepts covered in lectures. This work was studied in a second year architectural engineering course focusing on building materials, processes and modeling. Participants (n=84) completed a lesson (readings, lecture, and labs) then followed by writing prompts centered on the following topics: Building with Concrete and Wood Construction (3 weeks later). Participants were assigned to one of two counterbalanced groups, group A used GIKS software to write a 300-word summary of the first lesson but did not write in the second lesson, while group B did not write in the first lesson but used GIKS in the second lesson, so that each group served as a control treatment for the other group. All students completed a concept structure survey at the end of each lesson that contained 20 key concepts from that lesson, the two concept structure surveys’ data were transformed into concept networks and then these networks were compared to an expert network benchmark referent, as well as to networks of the textbook chapter and the PowerPoint slides of the related lesson. Then a week after the second lesson students completed the standing end-of-module multiple-choice posttest that included items from these lesson as well as from other lessons in the module.
Results to date highlight that for both lessons, the group using GIKS scored higher on the concept structure survey (more like the expert network) BUT lower on the multiple-choice test, the difference was significant for the Building with Concrete lesson (p &amp;lt; .05) but not for the Wood Construction lesson. This interaction has been reported before by Ntshalintshali &amp;amp; Clariana (2020), that improving conceptual knowledge sometimes decreases memory of lesson details. Descriptive analysis of the group-average networks derived from the concept structure surveys for Building with Concrete show that the group-averaged network of those using GIKS compared to the control was more like the expert network (54% vs. 36%), the network of the textbook Chapter (32% vs. 29%), the network of the PowerPoint (PP) (46% vs. 41%), and especially like peers in the other group (67%). For Wood Construction the difference between the groups was less, the group-averaged network of those using GIKS compared to the control was more like the expert (40% vs. 39%), like the light-framed construction PP (28% vs. 24%), and especially like peers in the other group (72%). These findings show that writing-to-learn with GIKS with immediate network feedback improves conceptual knowledge as expected but at the cost of details. Peers conceptual structure of the lesson materials were very similar (peer-peer mental model convergence) and were more like others than like the expert, or the book chapters, or the PowerPoint slides; in addition, the PowerPoint slides appear to influence conceptual structure more than the textbook chapters. Investigation 2 will consider writing-to-learn with or without immediate network feedback in order to isolate the effects of immediate network feedback.

AU - Ryan Solnosky P.E.
AU - Roy B. Clariana
CY - Portland, Oregon
DA - 2024/06/23
PB - ASEE Conferences
TI - Board 428: Work in Progress: An Open Educational Resource to Improve Architectural Engineering Students Conceptual Knowledge When Writing-to-Learn: Investigation 1
UR - https://peer.asee.org/47018
DO - 10.18260/1-2--47018
ER -