Open-Book Problem-Solving in Engineering: An Exploratory Study

David J. Therriault; Christine S. Lee; Elliot P. Douglas; Mirka Koro-Ljungberg; Nathan McNeill

Download Paper | Permalink

Conference: 2011 ASEE Annual Conference & Exposition
Location: Vancouver, BC
Publication Date: June 26, 2011
Start Date: June 26, 2011
End Date: June 29, 2011
ISSN: 2153-5965
Conference Session: Assessing Student Learning
Tagged Division: Educational Research and Methods
Page Count: 13
Page Numbers: 22.1123.1 - 22.1123.13
DOI: 10.18260/1-2--18495
Permanent URL: https://peer.asee.org/18495
Download Count: 831

Paper Authors

biography

David J. Therriault University of Florida

visit author page

Dr. Therriault, an Assistant Professor joined the College of Education at University of Florida in 2004. He received his undergraduate degree in psychology from the University of New Hampshire and his M. A. and Ph.D. in cognitive psychology from the University of Illinois at Chicago. Dr. Therriault’s primary research interests include the representation of text in memory, comprehending time and space in language, the link between attention and intelligence, the use of perceptual symbols in language, and educational issues related to these topics.

visit author page

author page

Christine S. Lee University of Florida

biography

Elliot P. Douglas University of Florida

visit author page

Dr. Elliot P. Douglas is Associate Chair, Associate Professor, and Distinguished Teaching Scholar in the Department of Materials Science and Engineering at the University of Florida. His research activities are in the areas of active learning, problem solving, critical thinking, and use of qualitative methodologies in engineering education. Specifically, he has published and presented work on the use of guided inquiry as an active learning technique for engineering; how critical thinking is used in practice by students; and how different epistemological stances are enacted in engineering education research. He has been involved in faculty development activities since 1998, through the ExCEEd Teaching Workshops of the American Society of Civil Engineers, the Essential Teaching Seminars of the American Society of Mechanical Engineers, and the US National Science Foundation-sponsored SUCCEED Coalition. He has received several awards for his work, including the Presidential Early Career Award for Scientists and Engineers, the Ralph Teetor Education Award from the Society of Automotive Engineers, being named a University of Florida Distinguished Teaching Scholar, and being named the University of Florida Teacher of the Year for 2003-04. He is a member of the American Society for Engineering Education, the American Educational Research Association, and the American Chemical Society. He is a Past Chair of the Polymeric Materials: Science and Engineering Division of the American Chemical Society and is currently Editor-in-Chief of Polymer Reviews.

visit author page

biography

Mirka Koro-Ljungberg University of Florida

visit author page

Mirka Koro-Ljungberg is an Associate professor of qualitative research methodology at the University of Florida. She received her doctorate from the University of Helsinki, Finland. Prior to joining the faculty at the University of Florida she conducted research a visiting scholar at the University of Georgia. Her research focuses on the conceptual and theoretical aspects of qualitative research and participant-driven methodologies.

visit author page

author page

Nathan McNeill University of Florida

Download Paper | Permalink

Abstract

Open-book problem-solving in engineeringAbstractThe bulk of educational research exploring open-book examination demonstrates thatthe open-book format reduces student anxiety and promotes higher-level learning (i.e.,reduces reliance upon rote memorization and prompts students to focus onunderstanding concepts and principles). Previous studies examining open-bookassessment provide evidence that students exhibit better performance on open-bookexams compared to closed-book exams. In addition, many university faculty find itadvantageous to employ exams using an open-book format, especially in engineering.However, within engineering we know little about how students approach open-booktesting, particularly with regard to how they spend their time on different tasks andhow this division of time may affect performance.The study in this paper examined the testing behavior of 8 senior materials science andengineering students at a large public university in the southeastern US. Studentscompleted four complex engineering problems during individual laboratory sessionswhile engaged in a think-aloud procedure (i.e. verbally explaining their thoughtprocesses as they worked through the problems). The problems were designed to varyin terms of their closed or open-endedness and the number of decision points involvedin their solution. Students’ think-aloud protocols were categorized to determine theamount of time spent on each of five exam behaviors: reading from the textbook,writing, calculating, reading the test question, talking/reflecting, or checking one’swork. Problem solutions were separately graded using a previously created rubric. Thetime spent on various behavior categories were then examined with respect to problemgrade. Reading represented the bulk of students’ time on the problems (37% onaverage). In contrast, students spent less than 5% of their time checking their work forerrors, 4% using a calculator, and 20% writing/working out their answer on paper.Interestingly, there was a significant negative correlation between reading the textbookand students’ grades. The more time that students spent with the text, the more poorlythey performed. This correlation was strongest for students who had the lowestproblem scores, but was still evident for students with the highest problem scores.Students with the lowest scores tended to search the text for information or an exampleproblem as a guide to how to approach the problem, while students with the highestscores tended to use the text to verify information. This data suggests that for oursample the textbook may have served as a distraction, with associated negative resultsduring problem solving. Our results may have been affected by the context; theproblem-solving sessions were not part of any particular class and the students did nothave an opportunity to prepare ahead of time. However, the results point to theimportance of training students in effective means of using resources during open-bookexams so as to avoid distractive behaviors. This training would serve not only toimprove exam performance, but to educate students in effective use of resources forprofessional practice where open-book problem-solving is the norm.

Citation
Format

Therriault, D. J., & Lee, C. S., & Douglas, E. P., & Koro-Ljungberg, M., & McNeill, N. (2011, June), Open-Book Problem-Solving in Engineering: An Exploratory Study Paper presented at 2011 ASEE Annual Conference & Exposition, Vancouver, BC. 10.18260/1-2--18495

TY - CPAPER
AB - Open-book problem-solving in engineeringAbstractThe bulk of educational research exploring open-book examination demonstrates thatthe open-book format reduces student anxiety and promotes higher-level learning (i.e.,reduces reliance upon rote memorization and prompts students to focus onunderstanding concepts and principles). Previous studies examining open-bookassessment provide evidence that students exhibit better performance on open-bookexams compared to closed-book exams. In addition, many university faculty find itadvantageous to employ exams using an open-book format, especially in engineering.However, within engineering we know little about how students approach open-booktesting, particularly with regard to how they spend their time on different tasks andhow this division of time may affect performance.The study in this paper examined the testing behavior of 8 senior materials science andengineering students at a large public university in the southeastern US. Studentscompleted four complex engineering problems during individual laboratory sessionswhile engaged in a think-aloud procedure (i.e. verbally explaining their thoughtprocesses as they worked through the problems). The problems were designed to varyin terms of their closed or open-endedness and the number of decision points involvedin their solution. Students’ think-aloud protocols were categorized to determine theamount of time spent on each of five exam behaviors: reading from the textbook,writing, calculating, reading the test question, talking/reflecting, or checking one’swork. Problem solutions were separately graded using a previously created rubric. Thetime spent on various behavior categories were then examined with respect to problemgrade. Reading represented the bulk of students’ time on the problems (37% onaverage). In contrast, students spent less than 5% of their time checking their work forerrors, 4% using a calculator, and 20% writing/working out their answer on paper.Interestingly, there was a significant negative correlation between reading the textbookand students’ grades. The more time that students spent with the text, the more poorlythey performed. This correlation was strongest for students who had the lowestproblem scores, but was still evident for students with the highest problem scores.Students with the lowest scores tended to search the text for information or an exampleproblem as a guide to how to approach the problem, while students with the highestscores tended to use the text to verify information. This data suggests that for oursample the textbook may have served as a distraction, with associated negative resultsduring problem solving. Our results may have been affected by the context; theproblem-solving sessions were not part of any particular class and the students did nothave an opportunity to prepare ahead of time. However, the results point to theimportance of training students in effective means of using resources during open-bookexams so as to avoid distractive behaviors. This training would serve not only toimprove exam performance, but to educate students in effective use of resources forprofessional practice where open-book problem-solving is the norm.
AU - David J. Therriault
AU - Christine S. Lee
AU - Elliot P. Douglas
AU - Mirka Koro-Ljungberg
AU - Nathan McNeill
CY - Vancouver, BC
DA - 2011/06/26
PB - ASEE Conferences
TI - Open-Book Problem-Solving in Engineering: An Exploratory Study
UR - https://peer.asee.org/18495
DO - 10.18260/1-2--18495
ER -