A Content Analysis of How Engineering is Assessed in Published Curricula

Kerrie A Douglas; Tamara J. Moore; Hillary E. Merzdorf; Tingxuan Li; Amanda C. Johnston

Download Paper | Permalink

Conference: 2017 ASEE Annual Conference & Exposition
Location: Columbus, Ohio
Publication Date: June 24, 2017
Start Date: June 24, 2017
End Date: June 28, 2017
Conference Session: Pre-College: Fundamental Research in Engineering Education (1)
Tagged Division: Pre-College Engineering Education Division
Page Count: 17
DOI: 10.18260/1-2--27459
Permanent URL: https://peer.asee.org/27459
Download Count: 651

Paper Authors

biography

Kerrie A Douglas Purdue University, West Lafayette (College of Engineering) orcid.org/0000-0002-2693-5272

visit author page

Dr. Douglas is an Assistant Professor in the Purdue School of Engineering Education. Her research is focused on methods of assessment and evaluation unique to engineering learning contexts.

visit author page

biography

Tamara J. Moore Purdue University, West Lafayette (College of Engineering) orcid.org/0000-0002-7956-4479

visit author page

Tamara J. Moore, Ph.D., is an Associate Professor in the School of Engineering Education and Director of STEM Integration in the INSPIRE Institute at Purdue University. Dr. Moore’s research is centered on the integration of STEM concepts in K-12 and postsecondary classrooms in order to help students make connections among the STEM disciplines and achieve deep understanding. Her work focuses on defining STEM integration and investigating its power for student learning. Tamara Moore received an NSF Early CAREER award in 2010 and a Presidential Early Career Award for Scientists and Engineers (PECASE) in 2012.

visit author page

author page

Hillary E. Merzdorf Purdue University

author page

Tingxuan Li Purdue University

author page

Amanda C. Johnston Purdue University, West Lafayette (College of Engineering) orcid.org/0000-0001-5737-7798

Download Paper | Permalink

Abstract

A Content Analysis of How Engineering is Assessed in Published Curricula (Fundamental)

The purpose of this proposal is to present research findings concerning how and what about engineering is commonly assessed in well-known engineering or integrated STEM published curriculum. Two of the major shifts brought about by Next Generation Science Standards (NGSS) are an increased emphasis in students’ capabilities to perform higher-level reasoning skills and integrate content understanding into science practices. At the same time, NGSS has made engineering integration into science education a priority, and it is an exciting time of reform as schools are exploring curriculum resources and teachers are being trained in engineering design. When engineering is a part of science instruction, there must also be corresponding measurement of student learning, yet many teachers who are new to engineering are also unfamiliar with the process of assessing design practices. In addition, teachers must grapple with how to assess higher order skills, including how students use science to make design decisions. Evidence of higher levels of learning beyond memorizing content or identifying facts is the goal of engineering assessment, and NGSS provides general patterns of thought and behavior that students may exhibit at each grade level related to defining problems, developing solutions, and optimizing results which demonstrate learning. However, these guidelines are very broad and do not clearly recommend ways to go about assessing these practices in the classroom. It is imperative that teachers are provided with the means to properly assess student learning of both content and engineering practices.

As part of a larger goal of developing an integrated STEM curriculum for grades 4 – 8, this content analysis addresses the need for questioning how engineering is commonly assessed in elementary engineering or integrated STEM curricula. By examining current STEM assessments with two frameworks, this study investigates the following research questions: (1) What aspects of engineering are being assessed in common engineering or integrated STEM curricular units? (2) What level of cognitive demand is being referenced by these assessments? Using a purposeful sampling strategy, the authors reviewed 15 engineering curricula units published by 3 different publishing companies. To address the research questions, assessment tasks were coded based on the Task Analysis Guide in Science (TAGS) framework and alignment to the engineering process of design (POD). Preliminary results show that the majority of integrated STEM assessment items are testing students on memorized engineering practice, and very few are referencing higher levels of cognitive demand. These results indicate that the more complex thinking that students use during engineering classroom learning is being overlooked by subsequent assessment. In addition, a large proportion of assessment items are dedicated to learning background information about the problem and planning solution ideas, compared with other steps in the design process. The full paper will discuss conclusions and implications of the study.

Citation
Format

Douglas, K. A., & Moore, T. J., & Merzdorf, H. E., & Li, T., & Johnston, A. C. (2017, June), A Content Analysis of How Engineering is Assessed in Published Curricula Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. 10.18260/1-2--27459

TY - CPAPER
AB - A Content Analysis of How Engineering is Assessed in Published Curricula
(Fundamental)

The purpose of this proposal is to present research findings concerning how and what about engineering is commonly assessed in well-known engineering or integrated STEM published curriculum. Two of the major shifts brought about by Next Generation Science Standards (NGSS) are an increased emphasis in students’ capabilities to perform higher-level reasoning skills and integrate content understanding into science practices. At the same time, NGSS has made engineering integration into science education a priority, and it is an exciting time of reform as schools are exploring curriculum resources and teachers are being trained in engineering design. When engineering is a part of science instruction, there must also be corresponding measurement of student learning, yet many teachers who are new to engineering are also unfamiliar with the process of assessing design practices. In addition, teachers must grapple with how to assess higher order skills, including how students use science to make design decisions. Evidence of higher levels of learning beyond memorizing content or identifying facts is the goal of engineering assessment, and NGSS provides general patterns of thought and behavior that students may exhibit at each grade level related to defining problems, developing solutions, and optimizing results which demonstrate learning. However, these guidelines are very broad and do not clearly recommend ways to go about assessing these practices in the classroom. It is imperative that teachers are provided with the means to properly assess student learning of both content and engineering practices.

As part of a larger goal of developing an integrated STEM curriculum for grades 4 – 8, this content analysis addresses the need for questioning how engineering is commonly assessed in elementary engineering or integrated STEM curricula. By examining current STEM assessments with two frameworks, this study investigates the following research questions: (1) What aspects of engineering are being assessed in common engineering or integrated STEM curricular units? (2) What level of cognitive demand is being referenced by these assessments? Using a purposeful sampling strategy, the authors reviewed 15 engineering curricula units published by 3 different publishing companies. To address the research questions, assessment tasks were coded based on the Task Analysis Guide in Science (TAGS) framework and alignment to the engineering process of design (POD).
Preliminary results show that the majority of integrated STEM assessment items are testing students on memorized engineering practice, and very few are referencing higher levels of cognitive demand. These results indicate that the more complex thinking that students use during engineering classroom learning is being overlooked by subsequent assessment. In addition, a large proportion of assessment items are dedicated to learning background information about the problem and planning solution ideas, compared with other steps in the design process. The full paper will discuss conclusions and implications of the study.

AU - Kerrie A Douglas
AU - Tamara J. Moore
AU - Hillary E. Merzdorf
AU - Tingxuan Li
AU - Amanda C. Johnston
CY - Columbus, Ohio
DA - 2017/06/24
PB - ASEE Conferences
TI - A Content Analysis of How Engineering is Assessed in Published Curricula
UR - https://peer.asee.org/27459
DO - 10.18260/1-2--27459
ER -