Reflection in Time: Using Data Visualization to Identify Student Reflection Modes in Design

Corey T. Schimpf; Molly H. Goldstein; Charles Xie

Download Paper | Permalink

Conference: 2020 ASEE Virtual Annual Conference Content Access
Location: Virtual On line
Publication Date: June 22, 2020
Start Date: June 22, 2020
End Date: June 26, 2021
Conference Session: Design Mental Frameworks
Tagged Division: Design in Engineering Education
Page Count: 17
DOI: 10.18260/1-2--35132
Permanent URL: https://peer.asee.org/35132
Download Count: 579

Request a correction

Paper Authors

biography

Corey T. Schimpf Concord Consortium orcid.org/0000-0003-2706-3282

visit author page

Corey Schimpf is a Learning Analytics Scientist at the Concord Consortium with interest in design research and learning, learning analytics, research methods and underrepresentation in engineering. A major strand of his work focuses on developing and analyzing learning analytics that model students’ design practices or strategies through fine-grained computer-logged data from open-ended technology-centered science and engineering projects. A closely related strand focuses on leveraging innovative technology to support and engage students in engineering design in K-12 settings and beyond.

visit author page

biography

Molly H. Goldstein University of Illinois at Urbana-Champaign orcid.org/0000-0002-2382-4745

visit author page

Molly H. Goldstein is an engineering design educator and researcher at University of Illinois, Urbana-Champaign. She previously worked as an environmental engineer specializing in air quality influencing her focus in engineering design with environmental concerns. Her research interests include how students approach decision making in an engineering design context. She obtained her BS in General Engineering (Systems & Design) and MS in Systems and Entrepreneurial Engineering from the University of Illinois and PhD in Engineering Education from Purdue University.

visit author page

author page

Charles Xie

Download Paper | Permalink

Abstract

In design, reflection is a central practice that helps designers evaluate past strategies, synthesis knowledge gained and plan future actions. However, the design process is fluid as distinct design stages may happen in a different order and repeat or cycle in a sequence unique to the design context and designers involved. Reflection is a key part of the this process, and similarly can happen in different patterns. While past work in design reflection has investigated areas such as reflection content, relationships between reflection and design outcomes and methods to incorporate meaningful reflection in design education, less work has looked at how reflection is distributed over the design process. We seek to address this gap by proposing to analyze designers’ reflection process through a visual analytics approach to generate time plots of students’ reflection activities in context with other key associated design activities. Said time plots will aim to visualize several features of students’ reflection process, such as the timing and duration of a reflection episode, the intensity of the episode as measured by the size of reflection notes and transitions between reflection and other design activities. Identifying common reflection modes student designers employ, such as long intense episodes clustered near the beginning, will help uncover more and less effective reflection strategies and can serve as insight for educators to support students to reflect during design.

Data for this paper comes from 51 high school students from a New England public school who participated in the Net-Zero Energy design project. In the Net-Zero Energy project students are challenged to design a home that conserves or generates enough energy to compensate for all of the home’s energy consumption, while remaining under a set budget and meeting several aesthetic criteria. Students completed the challenge in Energy3D, a computer-aided-design (CAD) platform that has a simulation engine for estimating energy production and consumption, an integrated design journal and the ability to automatically log students design and journal activities. Students were asked to document and reflect on their progress throughout the design challenge but given flexibility to decide when and how to use the journal.

From this data we will produce time plots for each student depicting their design modeling, analysis and reflection processes on a multiplot visualization. The first round of analysis of the time plots and their features will seek to discover the variety of ways students reflect over their time designing. A secondary analysis will aim to cluster students’ time plots into unique reflection modes (i.e. patterns) in how and when students reflect. After identifying common reflection modes, two raters will independently apply these categories to students’ time plots and inter-rater reliability will be calculated to ensure the categories are robust. The emergent reflection modes hold promise to advance our understanding of students’ varying reflection processes and may also provide a foundation for developing new scaffolds to promote student reflection at critical junctures in design, particularly for novice designers.

Citation
Format

Schimpf, C. T., & Goldstein, M. H., & Xie, C. (2020, June), Reflection in Time: Using Data Visualization to Identify Student Reflection Modes in Design Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--35132

TY - CPAPER
AB - In design, reflection is a central practice that helps designers evaluate past strategies, synthesis knowledge gained and plan future actions. However, the design process is fluid as distinct design stages may happen in a different order and repeat or cycle in a sequence unique to the design context and designers involved. Reflection is a key part of the this process, and similarly can happen in different patterns. While past work in design reflection has investigated areas such as reflection content, relationships between reflection and design outcomes and methods to incorporate meaningful reflection in design education, less work has looked at how reflection is distributed over the design process. We seek to address this gap by proposing to analyze designers’ reflection process through a visual analytics approach to generate time plots of students’ reflection activities in context with other key associated design activities. Said time plots will aim to visualize several features of students’ reflection process, such as the timing and duration of a reflection episode, the intensity of the episode as measured by the size of reflection notes and transitions between reflection and other design activities. Identifying common reflection modes student designers employ, such as long intense episodes clustered near the beginning, will help uncover more and less effective reflection strategies and can serve as insight for educators to support students to reflect during design.

Data for this paper comes from 51 high school students from a New England public school who participated in the Net-Zero Energy design project. In the Net-Zero Energy project students are challenged to design a home that conserves or generates enough energy to compensate for all of the home’s energy consumption, while remaining under a set budget and meeting several aesthetic criteria. Students completed the challenge in Energy3D, a computer-aided-design (CAD) platform that has a simulation engine for estimating energy production and consumption, an integrated design journal and the ability to automatically log students design and journal activities. Students were asked to document and reflect on their progress throughout the design challenge but given flexibility to decide when and how to use the journal.

From this data we will produce time plots for each student depicting their design modeling, analysis and reflection processes on a multiplot visualization. The first round of analysis of the time plots and their features will seek to discover the variety of ways students reflect over their time designing. A secondary analysis will aim to cluster students’ time plots into unique reflection modes (i.e. patterns) in how and when students reflect. After identifying common reflection modes, two raters will independently apply these categories to students’ time plots and inter-rater reliability will be calculated to ensure the categories are robust. The emergent reflection modes hold promise to advance our understanding of students’ varying reflection processes and may also provide a foundation for developing new scaffolds to promote student reflection at critical junctures in design, particularly for novice designers.

AU - Corey T. Schimpf
AU - Molly H. Goldstein
AU - Charles Xie
CY - Virtual On line
DA - 2020/06/22
PB - ASEE Conferences
TI - Reflection in Time: Using Data Visualization to Identify Student Reflection Modes in Design
UR - https://peer.asee.org/35132
DO - 10.18260/1-2--35132
ER -