Authentic Engineering Design Assessment

Joanna Ambrosio; M. David Burghardt; Deborah Hecht

Download Paper | Permalink

Conference: 2021 ASEE Virtual Annual Conference Content Access
Location: Virtual Conference
Publication Date: July 26, 2021
Start Date: July 26, 2021
End Date: July 19, 2022
Conference Session: Design in Multidisciplinary Learning Environment
Tagged Division: Multidisciplinary Engineering
Tagged Topic: Diversity
Page Count: 16
DOI: 10.18260/1-2--36735
Permanent URL: https://peer.asee.org/36735
Download Count: 515

Dr. M. David Burghardt, professor of Engineering, founder and co-director of the Center for STEM Research, has been the principal or co-principle investigator on 13 NSF projects primarily dealing with engineering in STEM.

visit author page

biography

Deborah Hecht Center for Advanced Studyin Education

visit author page

As Director of the Center for Advanced Study in Education, at the CUNY Graduate Center I am involved in a wide range of educational evaluations of funded and local projects. I also mentor graduate students interested in careers in evaluation.

visit author page

Download Paper | Permalink

Abstract

Educators looking for authentic ways to assess learning might benefit from thinking as informal educators. In informal learning environments standard assessment measures, like exams, are not possible for many reasons. The Wise Guys and Gals project (WGG), an NSF funded Advancing Informal STEM Learning project, created an authentic assessment approach. This paper examines how engaging youth as activity designers can provide valuable assessment data.

The WGG project created blended engineering design challenges. Through the WISEngineering online learning environment youth are presented with a design challenge. They are guided through knowledge and skills (KSBS) that provide content knowledge important for successful designs and asked to evaluate their design solution according to presented criteria. They then engage in guided reflection about the experience. As an informal learning activity, delivered through Boys and Girls Clubs, the project team was very aware if the assessment resembled a school “test” many youths will simply not respond. Therefore, the embedded activity questions were designed to promote engagement rather than differentiate among youth. However, the project needed some way to assess learning.

The solution was use of a non-disruptive assessment that engaged youth in creating their own design challenge. Called the Shark Tank design challenge, this activity (assessment) reversed the roles for youth who were challenged to create a design challenge that could be implemented by their peers with all the elements of a WGG activity—a challenge statement with embedded specifications and constraints, evaluation criteria, knowledge and skill builders, and material cost limitations. The youth created a video pitch (the Shark Tank aspect) that demonstrates their design’s worthiness. The WGG project created video assessment criteria (shared with the youth developing their activity) and the video analysis using the criteria is the basis of this paper.

This approach has been used not only during out-of-school time activities but also in a college classroom. It is a simple, yet highly informative approach that can provide insights about both youth or student learning as well as the context of how learning occurs. To test its usability we used the scoring matrix to examine 39 video pitches. The results indicate an explicit understanding of an engineering design challenge and an implicit understanding of specifications and constraints.

Citation
Format

Ambrosio, J., & Burghardt, M. D., & Hecht, D. (2021, July), Authentic Engineering Design Assessment Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36735

TY - CPAPER
AB - Educators looking for authentic ways to assess learning might benefit from thinking as informal educators. In informal learning environments standard assessment measures, like exams, are not possible for many reasons. The Wise Guys and Gals project (WGG), an NSF funded Advancing Informal STEM Learning project, created an authentic assessment approach. This paper examines how engaging youth as activity designers can provide valuable assessment data.

The WGG project created blended engineering design challenges. Through the WISEngineering online learning environment youth are presented with a design challenge. They are guided through knowledge and skills (KSBS) that provide content knowledge important for successful designs and asked to evaluate their design solution according to presented criteria. They then engage in guided reflection about the experience. As an informal learning activity, delivered through Boys and Girls Clubs, the project team was very aware if the assessment resembled a school “test” many youths will simply not respond. Therefore, the embedded activity questions were designed to promote engagement rather than differentiate among youth. However, the project needed some way to assess learning.

The solution was use of a non-disruptive assessment that engaged youth in creating their own design challenge. Called the Shark Tank design challenge, this activity (assessment) reversed the roles for youth who were challenged to create a design challenge that could be implemented by their peers with all the elements of a WGG activity—a challenge statement with embedded specifications and constraints, evaluation criteria, knowledge and skill builders, and material cost limitations. The youth created a video pitch (the Shark Tank aspect) that demonstrates their design’s worthiness. The WGG project created video assessment criteria (shared with the youth developing their activity) and the video analysis using the criteria is the basis of this paper.

This approach has been used not only during out-of-school time activities but also in a college classroom. It is a simple, yet highly informative approach that can provide insights about both youth or student learning as well as the context of how learning occurs. To test its usability we used the scoring matrix to examine 39 video pitches. The results indicate an explicit understanding of an engineering design challenge and an implicit understanding of specifications and constraints.

AU - Joanna Ambrosio
AU - M. David Burghardt
AU - Deborah Hecht
CY - Virtual Conference
DA - 2021/07/26
PB - ASEE Conferences
TI - Authentic Engineering Design Assessment
UR - https://peer.asee.org/36735
DO - 10.18260/1-2--36735
ER -