June 23, 2013
June 23, 2013
June 26, 2013
Educational Research and Methods
23.987.1 - 23.987.12
Process Analysis as a Feedback Tool for Development of Engineering Problem Solving SkillsOne of the directives set forth by the National Academy of Engineers urges educationalresearchers to develop means of advancing personalized learning, or tailoring instruction to meetstudents’ individual needs. When solving problems, there are a vast number of ways thatstudents can err, all of which require different instructional interventions to amend. Identifyingindividual instructional needs requires a different form of assessment than traditional outcomes-based grading systems. It requires a more thorough analysis of the cognitive processes that leadto outcomes and discrepancies that occur throughout the completion of learning activities thatdiffer from the intended process.The purpose of this research was to establish an evidence-based method of assessing students’problem solving processes that can be applied to problems from a variety of contexts andformats. Data was collected from first year engineering students enrolled in an introductorycourse focused on building foundational skills required by all engineering majors. Processanalysis was applied to identify cognitive tasks used in successful solutions completed by firstyear students, the most prominent deficiencies in problem solving processes, and common errorsmade in students’ problem solving attempts. These tasks and errors were mapped to a problem-solving cycle identified by Pretz et al. (2003) with six categories of skills (Define the problem,Represent the problem, Organize knowledge about the problem, Allocate resources for solvingthe problem, Monitor progress toward the goals, and Evaluate the solution) and one category forsolution accuracy.Based on the findings of this research, an assessment tool was developed that enables theevaluation of the problem solving process, pinpointing underutilization of key tasks that havebeen linked to successful problem solutions as well as identifying errors committed in eachsegment of the process. This seven-category instrument ranks student performances based onlevel of proficiency exhibited for each skill (high/medium/low) and identifies the presence orabsence of errors. These values are tallied for an overall score that reflects level of problem-solving proficiency.The major advantage to this assessment tool over traditional grading methods is that it can beapplied as a personalized feedback system to inform students of their level of proficiency as wellas pinpoint skill deficiencies that need attention. This information could then be used byinstructors to direct students to resources and/or to design instructional interventions forovercoming these deficiencies. Additionally, researchers can use this assessment tool toinvestigate the effectiveness of various pedagogical interventions in terms of improving problemsolving performance and pinpointing what skill(s) are most affected. The assessment tool iscurrently being evaluated in terms of reliability and validity for use with different problemsdesigned for first year engineering students, specifically testing its use with problem statementsthat have modified structures and that address new contexts.
Grigg, S. J., & Van Dyken, J., & Benson, L., & Morkos, B. (2013, June), Process Analysis as a Feedback Tool for Development of Engineering Problem Solving Skills Paper presented at 2013 ASEE Annual Conference & Exposition, Atlanta, Georgia. https://peer.asee.org/22372
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2013 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015