Asee peer logo

Using Digital Workbooks to Collect Design Process Data

Download Paper |


2013 ASEE Annual Conference & Exposition


Atlanta, Georgia

Publication Date

June 23, 2013

Start Date

June 23, 2013

End Date

June 26, 2013



Conference Session

Design Methodology and Evaluation 1

Tagged Division

Design in Engineering Education

Page Count


Page Numbers

23.1310.1 - 23.1310.9



Permanent URL

Download Count


Request a correction

Paper Authors


Adam R Carberry Arizona State University Orcid 16x16

visit author page

Adam R. Carberry, Ph.D., is an Assistant Professor at Arizona State University in the College of Technology & Innovation’s Department of Engineering. He earned a B.S. in Materials Science Engineering from Alfred University, and received his M.S. and Ph.D., both from Tufts University, in Chemistry and Engineering Education respectively. Dr. Carberry was previously an employee of the Tufts’ Center for Engineering Education & Outreach and manager of the Student Teacher Outreach Mentorship Program (STOMP).

visit author page


Morgan M Hynes Arizona State University

visit author page

Morgan Hynes is a Research Faculty Associate at Arizona State University
conducting research on the impact of product archaeology dissection activities
on students’ knowledge and abilities to engineer in broader contexts. Before
joining ASU, Hynes was a Research Assistant Professor in the Education
Department and Education Research Program Director at the Center of
Engineering Education and Outreach at Tufts University. Hynes received his
B.S. in Mechanical Engineering in 2001 and his Ph.D. in Engineering Education
in 2009 (both degrees at Tufts University).  In his current positions, Hynes
serves as PI and Co-PI on a number of funded research projects investigating
engineering education in the K-12 and college settings. He is particularly
interested in how students and teachers engage in and reflect upon the
engineering design process. His research includes investigating how teachers
conceptualize and teach and how students engage in engineering through in-
depth case study analysis.

visit author page


Ethan E Danahy Tufts University

visit author page

Ethan Danahy is a Research Assistant Professor in the Department Computer Science at Tufts University outside of Boston MA, having received the B.S. and M.S. degrees in Computer Science in 2000 and 2002, respectively, and a Ph.D. degree in Electrical Engineering in 2007, all from Tufts. Additionally, he acts as the Engineering Research Program Director at the Center for Engineering Education and Outreach (CEEO), where he manages educational technology development projects while researching innovative and interactive techniques for assisting teachers with performing engineering education and communicating robotics concepts to students spanning the K-12 through university age range.

visit author page

Download Paper |


Using Digital Workbooks to Collect Design Process DataAbstractRoboBooks™ is a software initiative started in 2007 by an engineering education research andoutreach center. The goal of the project was to produce an interactive electronic workbook thatbrings together many different technologies into one location. The initial target populations wereteachers and students in an effort to provide a customizable curricular tool that would make iteasier for teachers to engage students in challenging design-based projects. Within theenvironment, students are able to follow along teacher-created curriculum, input responses in theform of text, pictures, and audio, and connect to external robotics hardware and sensors.The customizable environment of RoboBooks made this new learning tool an intriguingalternative for research data collection. The interactive electronic workbook was subsequentlyused in a series of studies investigating student engineering design processes. The followingpaper describes the process by which the RoboBooks learning environment was modified andthen used to collect research data of students engaged in the problem solving process.Researchers at the at the center were conducting studies investigating the engineering designprocesses of undergraduate engineering students as they solved open-ended design challenges.Previous research suggested using verbal protocol analysis, a form of think-aloud, to study thecognitive processes of the research participants. The research design included a hands-on taskwith the objective of designing an assistive device. Pilot studies were conducted withengineering experts where they were presented with the design challenge and 15 stacks of indexcards containing a variety of information (e.g. talk to a possible client, review important math orscience concepts, view available materials, plan/draw/sketch, and build a prototype) that theywere allowed to peruse as they solved the task. After conducting a number of these tedious pilotinterviews, the research team decided they could collect a larger corpus of data by streamliningthe data collection protocol. The solution was to develop a digital version of the task usingRoboBooks.The basic structure of RoboBooks allowed the research team to quickly build the design task.Participants would enter the digital environment and immediately be presented with the designchallenge. The intuitive environment made it easy for participants to navigate the challengewithout any great instructional detail. The stacks of index cards were easily converted to a seriesof radio buttons (Figure 1). Information could be cycled through with forward and backwardbuttons that noted which page of how many possible pages they were on. The picture/videocapability of RoboBooks was utilized to allow the participant to record and document theirprototype without the assistance of a researcher. This feature was also used to replace somewritten text on the index cards with videos.Additional features were added to RoboBooks that enabled the researchers to collect data onwhen and for how long the participants accessed information. Participants were also prompted torate “how useful” a particular set of information was to the task. The final product was a stand-alone research tool that allowed the researchers to conduct multiple experiments simultaneouslyfor the purpose of widespread use. Usability testing showed the new digital version of the designtask to be well received by research participants.Figure 1: RoboBooks design task interface.

Carberry, A. R., & Hynes, M. M., & Danahy, E. E. (2013, June), Using Digital Workbooks to Collect Design Process Data Paper presented at 2013 ASEE Annual Conference & Exposition, Atlanta, Georgia. 10.18260/1-2--22695

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2013 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015