The Potential for Computer Tutors to Assist Students Learning to Solve Complex Problems

Paul S. Steif; Luoting Fu; Levent Burak Kara

Download Paper | Permalink

Conference: 2014 ASEE Annual Conference & Exposition
Location: Indianapolis, Indiana
Publication Date: June 15, 2014
Start Date: June 15, 2014
End Date: June 18, 2014
ISSN: 2153-5965
Conference Session: Innovative Use of Technology and the Internet in Engineering Education
Tagged Division: Educational Research and Methods
Page Count: 20
Page Numbers: 24.1239.1 - 24.1239.20
DOI: 10.18260/1-2--23172
Permanent URL: https://peer.asee.org/23172
Download Count: 495

Paper Authors

biography

Paul S. Steif Carnegie Mellon University

visit author page

Paul S. Steif is a Professor of Mechanical Engineering at Carnegie Mellon University. He received a Sc.B. in engineering from Brown University (1979) and M.S. (1980) and Ph.D. (1982) degrees from Harvard University in applied mechanics. He has been active as a teacher and researcher in the field of engineering education and mechanics. His research has focused on student learning of mechanics concepts and developing new course materials and classroom approaches. Drawing upon methods of cognitive and learning sciences, he has led the development and psychometric validation of the Statics Concept Inventory – a test of statics conceptual knowledge. He is the co-author of Open Learning Initiative (OLI) Engineering Statics, and he is the author of a new textbook Mechanics of Materials, published by Pearson.

visit author page

biography

Luoting Fu

visit author page

Luoting Fu has a BS in aerospace engineering from Shanghai Jiao Tong University, and an MS and PhD in mechanical engineering from Carnegie Mellon University. This work was performed when he was with the Visual Design and Engineering Lab in the Department of Mechanical Engineering at Carnegie Mellon University.

visit author page

author page

Levent Burak Kara Carnegie Mellon University

Download Paper | Permalink

Abstract

The Potential for Computer Tutors to Assist Students Learning to Solve Complex Problems Engineering education includes significant attention to problem solving skills, withstudents gradually confronting problems of increasing complexity. Even within anysingle fundamental engineering science course, which addresses a limited set of concepts,students must learn to solve problems that require coordinating and organizing multipleparts. To solve, the student needs to decompose a problem into inter-related sub-problems, define variables of different types, carry out analyses of sub-problems, andfinally combine and interpret the results. Problems typically have multiple pathways tothe correct answers; students should be granted considerable latitude in constructing apathway.In general, formative assessment, that is feedback to students enabling them to revise andrefine their understanding and actions, can significantly promote learning. Formativeassessment can be provided through in-class activities, and also through computer-basedinstruction outside of the classroom. This paper addresses the issue of providingformative feedback for students confronting complex problems that involve significantlatitude in decomposition and construction of solutions. Traditionally, students solvecomplex problems as part of written homework assignments that are hand graded. In suchcircumstances, offering effective formative assessment is exceptionally challenging,requiring careful attention to solution details and rapid, rather than weeklong, turnaround.Furthermore, since later work builds upon earlier work, grading of a completed solutionoften involves judging off-path steps that may be irrelevant to the intended learning ormay build upon prior incorrect work.The research questions this paper seeks to answer are: (1) Under what circumstances isautomated, formative assessment on complex problem solving possible, (2) What metricsallow us to judge whether the feedback indeed promotes learning, and (3) On what basiscan we target ongoing improvements to the formative assessment offered?We address these questions in the context of a test case: a tutor to help students in staticslearning to solve truss problems. Trusses exemplify complex problems: students selectmultiple portions of the truss, draw free body diagrams, write down appropriateequilibrium equations for each diagram, organize the solving of equations, and interpretresults physically in terms of the original truss. Mastery requires clarity, systematicorganization, as well as conceptual and mathematical competence. Building on a catalogof typical errors, a computer interface was created where correct steps and typical errorsin solving truss problems can be executed with wide latitude to pursue solution paths.The tutor reflects careful trade-offs between granting latitude to the solver and retainingability to interpret work. The student can solve unimpeded until errors are made that caninterfere with future solving steps; feedback is then offered which enables students tocorrect their errors. Through task analysis, steps hypothesized to involve the samecomponents of knowledge have been grouped, and data is collected on the fly of attemptsto apply the different knowledge components. Statistical models are used to determinewhether errors in using different knowledge components decrease in frequency withpractice. The determined learning rates give insights into whether feedback is effectiveand can inform future improvements in the tutor.

Citation
Format

Steif, P. S., & Fu, L., & Kara, L. B. (2014, June), The Potential for Computer Tutors to Assist Students Learning to Solve Complex Problems Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. 10.18260/1-2--23172

TY  - CPAPER
AB  -        The Potential for Computer Tutors to Assist Students Learning                            to Solve Complex Problems  Engineering education includes significant attention to problem solving skills, withstudents gradually confronting problems of increasing complexity. Even within anysingle fundamental engineering science course, which addresses a limited set of concepts,students must learn to solve problems that require coordinating and organizing multipleparts. To solve, the student needs to decompose a problem into inter-related sub-problems, define variables of different types, carry out analyses of sub-problems, andfinally combine and interpret the results. Problems typically have multiple pathways tothe correct answers; students should be granted considerable latitude in constructing apathway.In general, formative assessment, that is feedback to students enabling them to revise andrefine their understanding and actions, can significantly promote learning. Formativeassessment can be provided through in-class activities, and also through computer-basedinstruction outside of the classroom. This paper addresses the issue of providingformative feedback for students confronting complex problems that involve significantlatitude in decomposition and construction of solutions. Traditionally, students solvecomplex problems as part of written homework assignments that are hand graded. In suchcircumstances, offering effective formative assessment is exceptionally challenging,requiring careful attention to solution details and rapid, rather than weeklong, turnaround.Furthermore, since later work builds upon earlier work, grading of a completed solutionoften involves judging off-path steps that may be irrelevant to the intended learning ormay build upon prior incorrect work.The research questions this paper seeks to answer are: (1) Under what circumstances isautomated, formative assessment on complex problem solving possible, (2) What metricsallow us to judge whether the feedback indeed promotes learning, and (3) On what basiscan we target ongoing improvements to the formative assessment offered?We address these questions in the context of a test case: a tutor to help students in staticslearning to solve truss problems. Trusses exemplify complex problems: students selectmultiple portions of the truss, draw free body diagrams, write down appropriateequilibrium equations for each diagram, organize the solving of equations, and interpretresults physically in terms of the original truss. Mastery requires clarity, systematicorganization, as well as conceptual and mathematical competence. Building on a catalogof typical errors, a computer interface was created where correct steps and typical errorsin solving truss problems can be executed with wide latitude to pursue solution paths.The tutor reflects careful trade-offs between granting latitude to the solver and retainingability to interpret work. The student can solve unimpeded until errors are made that caninterfere with future solving steps; feedback is then offered which enables students tocorrect their errors. Through task analysis, steps hypothesized to involve the samecomponents of knowledge have been grouped, and data is collected on the fly of attemptsto apply the different knowledge components. Statistical models are used to determinewhether errors in using different knowledge components decrease in frequency withpractice. The determined learning rates give insights into whether feedback is effectiveand can inform future improvements in the tutor.
AU  - Paul S. Steif
AU  - Luoting Fu
AU  - Levent Burak Kara
CY  - Indianapolis, Indiana
DA  - 2014/06/15
PB  - ASEE Conferences
TI  - The Potential for Computer Tutors to Assist Students Learning to Solve Complex Problems
UR  - https://peer.asee.org/23172
DO  - 10.18260/1-2--23172
ER  -