Computer Tutor Versus Solving Problems by Hand: A Comparison in Statistics

Colin Engebretsen; Paul S. Steif

Download Paper | Permalink

Conference: 2015 ASEE Annual Conference & Exposition
Location: Seattle, Washington
Publication Date: June 14, 2015
Start Date: June 14, 2015
End Date: June 17, 2015
ISBN: 978-0-692-50180-1
ISSN: 2153-5965
Conference Session: Computer Tutors, Simulation, and Videos
Tagged Division: Mechanics
Page Count: 10
Page Numbers: 26.384.1 - 26.384.10
DOI: 10.18260/p.23723
Permanent URL: https://peer.asee.org/23723
Download Count: 440

Request a correction

Paper Authors

biography

Colin Engebretsen U.S. Air Force Academy

visit author page

Captain Colin C. Engebretsen is an assistant professor of engineering mechanics at the US Air Force Academy. He is the Course Director for EM 220 Fundamentals of Mechanics and has also instructed EM 330 Mechanics of Deformable Bodies. He is the academic advisor for 13 cadets and works as the Assistant Advisor-in-Charge for the department of engineering mechanics.

Captain Engebretsen was commissioned in 2008 through the ROTC program at the University of North Dakota, obtaining a Bachelor's Degree in Mechanical Engineering. He obtained a master’s degree in Aeronautical Engineering from the Air Force Institute of Technology while researching hysteretic damping in ceramic coated titanium. Additionally He has worked as a structural engineer on the KC-135 Stratotanker and Chief Engineer for Aircraft Battle Damage Repair Engineering at the Oklahoma City Air Logistics Center.

visit author page

biography

Paul S. Steif Carnegie Mellon University

visit author page

Paul S. Steif is a Professor of Mechanical Engineering at Carnegie Mellon University. He received a Sc.B. in engineering from Brown University (1979) and M.S. (1980) and Ph.D. (1982) degrees from Harvard University in applied mechanics. He has been active as a teacher and researcher in the field of engineering education and mechanics. His research has focused on student learning of mechanics concepts and developing new course materials and classroom approaches. Drawing upon methods of cognitive and learning sciences, he has led the development and psychometric validation of the Statics Concept Inventory – a test of statics conceptual knowledge. He is the co-author of Open Learning Initiative (OLI) Engineering Statics, and he is the author of a textbook Mechanics of Materials, published by Pearson.

visit author page

Download Paper | Permalink

Abstract

Computer tutor versus solving problems by hand: a comparison in statics While computer-based instruction can be used to automate the process of gradinghomework, sometimes its goal is to provide formative feedback to students whilelearning. Formative feedback is often straightforward to provide when students areperforming simple tasks, such as answering multiple-choice questions. In the case ofproblems that are more complex, with many correct pathways to solution, the task ofenabling the computer to follow student work and provide feedback is much morecomplex.Recently, a computer-based tutoring system was devised that focused on truss problemsin statics. This Truss Tutor has a graphical user interface that enables students to solvesuch problems via method of joints or method of sections in a relatively free form way.Furthermore, by intervening at judiciously chosen points that prevent new parts of asolution from building upon errors, Truss Tutor can give feedback to students regardlessof the solution path they take. This is made possible by a cognitive model for solvingtrusses, in particular a set of algorithms for recognizing, for any current correct state,whether any forward step is correct. When an error is detected, feedback is offered thatenables the user to correct that error and continue with the solution. Students in classes ata number of institutions have completed homework assignments using truss Tutor. Byapplying analysis tools used in intelligent tutoring systems, it has been shown that, formany of the sub-skills necessary for solving truss problems, students commit fewer errorswith practice. So, while students using Truss Tutor improve in their use of the tutor itself,it is unknown how using the tutor compares with completing homework problems in thetraditional manner with paper and pencil.In this paper, we describe a study conducted to compare performance on examinations ofstudents who used Truss Tutor to complete their homework assignment with studentswho solved truss problems by hand. The study took place at an institution in which over600 were enrolled in statics in the current semester. 600 students take statics at the sametime. There are approximately 25 students in each section that are run independently,with some instructors teaching multiple sections. All students in statics take a commonexamination. To provide a sound comparison between computer tutor and pencil andpaper, instructors who taught two or more sections were recruited to participate in thestudy. Each such instructor had one section that used Truss Tutor and one section thatsolved the same truss problems for homework by hand. Sections were also balanced withrespect to time of day and whether the experimental or the control group received lecturefirst. The results of this study, comprising nearly 250 students, currently ongoing, will bereported upon. Development of a cognitive tutor for learning truss analysis Statics poses many conceptual challenges to students and offers exposure to realisticsystems and to a style of analysis important throughout engineering: subsystem isolation.The ability to solve problems is a principal goal of statics; yet students traditionallyreceive the least contact with instructors as they practice solving problems. With thecomputer, one may be able to construct environments that offer instruction and feedbackto students while learning, without the presence of an instructor. But there is a trade off:between how much freedom students have to create solutions to problems and the abilityof the computer to ascertain, judge, and give feedback on what students have done. Onthe one hand, one could give students a blank piece of paper (or a computer tablet) andask them to conduct their analysis; but it would be quite challenging to interpret whatstudents are drawing and respond to it. On the other hand, we could ask students a seriesof multiple-choice questions, with carefully chosen answers; while the responses areinterpretable, they give no indication if students could independently solve a full problemon their own. We refer to this as the latitude-interpretation trade-off.The analysis of trusses via method of joints and method of sections is a topic that is ripefor effective computer-assisted problem solving: students can have reasonably broadlatitude to solve truss problems, while the computer can track students’ work in detail andprovide feedback. Truss analysis presents this opportunity because the forms of solutionsare well structured and the common range of student errors can be identified.In this paper, we describe the development of a cognitive truss tutor. The tutor is deemeda cognitive tutor in the sense that there is an underlying cognitive model for the set ofskills or knowledge components needed to solve trusses, as well as the common incorrectactions typical of novice learners. We show how the errors typically committed bystudents in solving truss problems are also allowed by the tutor. We explain how thetutor’s design imposes modest constraints on user actions relative to fully free paper-and-pencil solving, but still enable full interpretation of student work. Students have used thetutor in place of written homework in regular statics courses, and data on this usage hasbeen collected. Results from initial analysis suggest that students commit fewer errors asthey use the tutor.

Citation
Format

Engebretsen, C., & Steif, P. S. (2015, June), Computer Tutor Versus Solving Problems by Hand: A Comparison in Statistics Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23723

TY - CPAPER
AB - Computer tutor versus solving problems by hand: a comparison in statics While computer-based instruction can be used to automate the process of gradinghomework, sometimes its goal is to provide formative feedback to students whilelearning. Formative feedback is often straightforward to provide when students areperforming simple tasks, such as answering multiple-choice questions. In the case ofproblems that are more complex, with many correct pathways to solution, the task ofenabling the computer to follow student work and provide feedback is much morecomplex.Recently, a computer-based tutoring system was devised that focused on truss problemsin statics. This Truss Tutor has a graphical user interface that enables students to solvesuch problems via method of joints or method of sections in a relatively free form way.Furthermore, by intervening at judiciously chosen points that prevent new parts of asolution from building upon errors, Truss Tutor can give feedback to students regardlessof the solution path they take. This is made possible by a cognitive model for solvingtrusses, in particular a set of algorithms for recognizing, for any current correct state,whether any forward step is correct. When an error is detected, feedback is offered thatenables the user to correct that error and continue with the solution. Students in classes ata number of institutions have completed homework assignments using truss Tutor. Byapplying analysis tools used in intelligent tutoring systems, it has been shown that, formany of the sub-skills necessary for solving truss problems, students commit fewer errorswith practice. So, while students using Truss Tutor improve in their use of the tutor itself,it is unknown how using the tutor compares with completing homework problems in thetraditional manner with paper and pencil.In this paper, we describe a study conducted to compare performance on examinations ofstudents who used Truss Tutor to complete their homework assignment with studentswho solved truss problems by hand. The study took place at an institution in which over600 were enrolled in statics in the current semester. 600 students take statics at the sametime. There are approximately 25 students in each section that are run independently,with some instructors teaching multiple sections. All students in statics take a commonexamination. To provide a sound comparison between computer tutor and pencil andpaper, instructors who taught two or more sections were recruited to participate in thestudy. Each such instructor had one section that used Truss Tutor and one section thatsolved the same truss problems for homework by hand. Sections were also balanced withrespect to time of day and whether the experimental or the control group received lecturefirst. The results of this study, comprising nearly 250 students, currently ongoing, will bereported upon. Development of a cognitive tutor for learning truss analysis Statics poses many conceptual challenges to students and offers exposure to realisticsystems and to a style of analysis important throughout engineering: subsystem isolation.The ability to solve problems is a principal goal of statics; yet students traditionallyreceive the least contact with instructors as they practice solving problems. With thecomputer, one may be able to construct environments that offer instruction and feedbackto students while learning, without the presence of an instructor. But there is a trade off:between how much freedom students have to create solutions to problems and the abilityof the computer to ascertain, judge, and give feedback on what students have done. Onthe one hand, one could give students a blank piece of paper (or a computer tablet) andask them to conduct their analysis; but it would be quite challenging to interpret whatstudents are drawing and respond to it. On the other hand, we could ask students a seriesof multiple-choice questions, with carefully chosen answers; while the responses areinterpretable, they give no indication if students could independently solve a full problemon their own. We refer to this as the latitude-interpretation trade-off.The analysis of trusses via method of joints and method of sections is a topic that is ripefor effective computer-assisted problem solving: students can have reasonably broadlatitude to solve truss problems, while the computer can track students’ work in detail andprovide feedback. Truss analysis presents this opportunity because the forms of solutionsare well structured and the common range of student errors can be identified.In this paper, we describe the development of a cognitive truss tutor. The tutor is deemeda cognitive tutor in the sense that there is an underlying cognitive model for the set ofskills or knowledge components needed to solve trusses, as well as the common incorrectactions typical of novice learners. We show how the errors typically committed bystudents in solving truss problems are also allowed by the tutor. We explain how thetutor’s design imposes modest constraints on user actions relative to fully free paper-and-pencil solving, but still enable full interpretation of student work. Students have used thetutor in place of written homework in regular statics courses, and data on this usage hasbeen collected. Results from initial analysis suggest that students commit fewer errors asthey use the tutor.
AU - Colin Engebretsen
AU - Paul S. Steif
CY - Seattle, Washington
DA - 2015/06/14
PB - ASEE Conferences
TI - Computer Tutor Versus Solving Problems by Hand: A Comparison in Statistics
UR - https://peer.asee.org/23723
DO - 10.18260/p.23723
ER -