Assessing Elementary Students’ Engineering Design Thinking with an “Evaluate-And-Improve” Task (Fundamental)

Nicole Alexandra Batrouny; Kristen B. Wendell; Chelsea Joy Andrews; Tejaswini S. Dalvi; Christine M. Kelly

Download Paper | Permalink

Conference: 2021 ASEE Virtual Annual Conference Content Access
Location: Virtual Conference
Publication Date: July 26, 2021
Start Date: July 26, 2021
End Date: July 19, 2022
Conference Session: Pre-College Engineering Education Division Technical Session 15
Tagged Division: Pre-College Engineering Education
Tagged Topic: Diversity
Page Count: 15
DOI: 10.18260/1-2--36706
Permanent URL: https://peer.asee.org/36706
Download Count: 495

Request a correction

Paper Authors

biography

Nicole Alexandra Batrouny Tufts University

visit author page

Nicole Batrouny is a PhD candidate in Mechanical Engineering at Tufts University. Her engineering education research interests include upper elementary engineering education, integrated science and engineering, collaboration in engineering, and decision making in engineering. For her Master's thesis, she uncovered talk moves used by 4th grade students that fostered collaborative, disciplinary decision-making during an engineering design outreach program. For her dissertation, she intends to explore the ways in which team mental models function in teams of novice engineers and how novice engineers can be trained to collaborate more effectively on diverse teams.

visit author page

biography

Kristen B. Wendell Tufts University

visit author page

Kristen Wendell is Associate Professor of Mechanical Engineering and Adjunct Associate Professor of Education at Tufts University. Her research efforts at at the Center for Engineering Education and Outreach focus on supporting discourse and design practices during K-12, teacher education, and college-level engineering learning experiences, and increasing access to engineering in the elementary school experience. https://engineering.tufts.edu/me/people/faculty/kristen-bethke-wendell

visit author page

biography

Chelsea Joy Andrews Tufts University orcid.org/0000-0002-9017-595X

visit author page

Chelsea Andrews is a post-doctoral researcher at Tufts University in Engineering Education. She received a B.S. from Texas A&M University in ocean engineering, an S.M. from MIT in civil and environmental engineering, and a PhD from Tufts University in Engineering Education. Her current research includes investigating children's engagement in engineering design through in-depth case study analysis.

visit author page

author page

Tejaswini S. Dalvi University of Massachusetts, Boston

author page

Christine M. Kelly

Download Paper | Permalink

Abstract

Although curricular resources for elementary engineering design continue to grow, it remains challenging to identify assessment tools that focus on students’ reasoning within engineering design and that are feasible for classroom use and accessible for emerging writers. Many existing tools measure students’ knowledge of engineering design process steps or their ideas about the nature of engineering and technology. In our elementary engineering curriculum development work, we needed a tool that provides richer information about students’ design thinking – that is, about the quality of their ideas for scoping and solving a design problem and the nature of their justifications for those possible solutions. In collaboration with third-grade teachers, we developed an open-response task that asks students to evaluate and improve on the first iteration of a design solution. We refined this assessment task in the context of a three-year, grant-funded curriculum development project with two school districts. In this paper, we present the assessment task and an exploratory analysis of the pre- and post-unit responses collected from one classroom implementation using the final version of the task. The goal of this study is not to evaluate our curriculum, but instead to address the question To what extent does a written “evaluate-and-improve” task reveal a range of design thinking across different third-grade students and capture changes in their design thinking after an engineering learning experience?

The evaluate-and-improve assessment task presents a diagram of an unbalanced, lopsided cart for transporting classroom plants and asks students to identify problems with the cart design, propose changes to improve it, and justify those proposed changes. To explore how the task enables characterization of student design thinking, we conducted close analysis of the pre- and post-unit responses by 18 third-grade students. They complete the task before and after participating in an integrated science and engineering unit on playground design. Students took the assessment independently but worked in teams of three on the unit. We used open coding followed by a constant comparative approach to identify themes in the students’ proposed cart design changes and justifications. We then tabulated the frequency of proposed changes to the cart and the types of justification given, and we organized these results by time (pre or post-unit) and by student team. Across their pre- and post-unit responses, the 18 students proposed 14 different changes and provided seven different justifications to support those changes. Students included more justifications in their post-unit explanations, especially when they did not include any justifications in their pre-unit responses. We also noticed trends in the data based on student team, which may indicate an effect of collaboration on their design thinking. We were encouraged by the wide range of both proposed changes and justifications and by the ability of the task to capture across-team differences in reasoning. Our exploratory study of this evaluate and improve task suggests that it gives third-grade students an opportunity to demonstrate their ability to scope problems, propose design iterations, and justify those changes.

Citation
Format

Batrouny, N. A., & Wendell, K. B., & Andrews, C. J., & Dalvi, T. S., & Kelly, C. M. (2021, July), Assessing Elementary Students’ Engineering Design Thinking with an “Evaluate-And-Improve” Task (Fundamental) Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36706

TY - CPAPER
AB - Although curricular resources for elementary engineering design continue to grow, it remains challenging to identify assessment tools that focus on students’ reasoning within engineering design and that are feasible for classroom use and accessible for emerging writers. Many existing tools measure students’ knowledge of engineering design process steps or their ideas about the nature of engineering and technology. In our elementary engineering curriculum development work, we needed a tool that provides richer information about students’ design thinking – that is, about the quality of their ideas for scoping and solving a design problem and the nature of their justifications for those possible solutions. In collaboration with third-grade teachers, we developed an open-response task that asks students to evaluate and improve on the first iteration of a design solution. We refined this assessment task in the context of a three-year, grant-funded curriculum development project with two school districts. In this paper, we present the assessment task and an exploratory analysis of the pre- and post-unit responses collected from one classroom implementation using the final version of the task. The goal of this study is not to evaluate our curriculum, but instead to address the question To what extent does a written “evaluate-and-improve” task reveal a range of design thinking across different third-grade students and capture changes in their design thinking after an engineering learning experience?

The evaluate-and-improve assessment task presents a diagram of an unbalanced, lopsided cart for transporting classroom plants and asks students to identify problems with the cart design, propose changes to improve it, and justify those proposed changes. To explore how the task enables characterization of student design thinking, we conducted close analysis of the pre- and post-unit responses by 18 third-grade students. They complete the task before and after participating in an integrated science and engineering unit on playground design. Students took the assessment independently but worked in teams of three on the unit. We used open coding followed by a constant comparative approach to identify themes in the students’ proposed cart design changes and justifications. We then tabulated the frequency of proposed changes to the cart and the types of justification given, and we organized these results by time (pre or post-unit) and by student team. Across their pre- and post-unit responses, the 18 students proposed 14 different changes and provided seven different justifications to support those changes. Students included more justifications in their post-unit explanations, especially when they did not include any justifications in their pre-unit responses. We also noticed trends in the data based on student team, which may indicate an effect of collaboration on their design thinking. We were encouraged by the wide range of both proposed changes and justifications and by the ability of the task to capture across-team differences in reasoning. Our exploratory study of this evaluate and improve task suggests that it gives third-grade students an opportunity to demonstrate their ability to scope problems, propose design iterations, and justify those changes.
AU - Nicole Alexandra Batrouny
AU - Kristen B. Wendell
AU - Chelsea Joy Andrews
AU - Tejaswini S. Dalvi
AU - Christine M. Kelly
CY - Virtual Conference
DA - 2021/07/26
PB - ASEE Conferences
TI - Assessing Elementary Students’ Engineering Design Thinking with an “Evaluate-And-Improve” Task (Fundamental)
UR - https://peer.asee.org/36706
DO - 10.18260/1-2--36706
ER -