Asee peer logo

Reflection in Time: Using Data Visualization to Identify Student Reflection Modes in Design

Download Paper |


2020 ASEE Virtual Annual Conference Content Access


Virtual On line

Publication Date

June 22, 2020

Start Date

June 22, 2020

End Date

June 26, 2021

Conference Session

Design Mental Frameworks

Tagged Division

Design in Engineering Education

Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Corey T. Schimpf Concord Consortium Orcid 16x16

visit author page

Corey Schimpf is a Learning Analytics Scientist at the Concord Consortium with interest in design research and learning, learning analytics, research methods and underrepresentation in engineering. A major strand of his work focuses on developing and analyzing learning analytics that model students’ design practices or strategies through fine-grained computer-logged data from open-ended technology-centered science and engineering projects. A closely related strand focuses on leveraging innovative technology to support and engage students in engineering design in K-12 settings and beyond.

visit author page


Molly H. Goldstein University of Illinois at Urbana-Champaign Orcid 16x16

visit author page

Molly H. Goldstein is an engineering design educator and researcher at University of Illinois, Urbana-Champaign. She previously worked as an environmental engineer specializing in air quality influencing her focus in engineering design with environmental concerns. Her research interests include how students approach decision making in an engineering design context. She obtained her BS in General Engineering (Systems & Design) and MS in Systems and Entrepreneurial Engineering from the University of Illinois and PhD in Engineering Education from Purdue University.

visit author page

author page

Charles Xie

Download Paper |


In design, reflection is a central practice that helps designers evaluate past strategies, synthesis knowledge gained and plan future actions. However, the design process is fluid as distinct design stages may happen in a different order and repeat or cycle in a sequence unique to the design context and designers involved. Reflection is a key part of the this process, and similarly can happen in different patterns. While past work in design reflection has investigated areas such as reflection content, relationships between reflection and design outcomes and methods to incorporate meaningful reflection in design education, less work has looked at how reflection is distributed over the design process. We seek to address this gap by proposing to analyze designers’ reflection process through a visual analytics approach to generate time plots of students’ reflection activities in context with other key associated design activities. Said time plots will aim to visualize several features of students’ reflection process, such as the timing and duration of a reflection episode, the intensity of the episode as measured by the size of reflection notes and transitions between reflection and other design activities. Identifying common reflection modes student designers employ, such as long intense episodes clustered near the beginning, will help uncover more and less effective reflection strategies and can serve as insight for educators to support students to reflect during design.

Data for this paper comes from 51 high school students from a New England public school who participated in the Net-Zero Energy design project. In the Net-Zero Energy project students are challenged to design a home that conserves or generates enough energy to compensate for all of the home’s energy consumption, while remaining under a set budget and meeting several aesthetic criteria. Students completed the challenge in Energy3D, a computer-aided-design (CAD) platform that has a simulation engine for estimating energy production and consumption, an integrated design journal and the ability to automatically log students design and journal activities. Students were asked to document and reflect on their progress throughout the design challenge but given flexibility to decide when and how to use the journal.

From this data we will produce time plots for each student depicting their design modeling, analysis and reflection processes on a multiplot visualization. The first round of analysis of the time plots and their features will seek to discover the variety of ways students reflect over their time designing. A secondary analysis will aim to cluster students’ time plots into unique reflection modes (i.e. patterns) in how and when students reflect. After identifying common reflection modes, two raters will independently apply these categories to students’ time plots and inter-rater reliability will be calculated to ensure the categories are robust. The emergent reflection modes hold promise to advance our understanding of students’ varying reflection processes and may also provide a foundation for developing new scaffolds to promote student reflection at critical junctures in design, particularly for novice designers.

Schimpf, C. T., & Goldstein, M. H., & Xie, C. (2020, June), Reflection in Time: Using Data Visualization to Identify Student Reflection Modes in Design Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--35132

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2020 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015