Asee peer logo

Performance Assessment in Elementary Engineering: Evaluating Student (RTP)

Download Paper |

Conference

2016 ASEE Annual Conference & Exposition

Location

New Orleans, Louisiana

Publication Date

June 26, 2016

Start Date

June 26, 2016

End Date

August 28, 2016

ISBN

978-0-692-68565-5

ISSN

2153-5965

Conference Session

K-12 & Pre-College Engineering Division: Fundamental & Research-to-Practice: K-12 Engineering Resources: Best Practices in Curriculum Design (Part 2)

Tagged Division

Pre-College Engineering Education Division

Tagged Topic

Diversity

Page Count

16

DOI

10.18260/p.25884

Permanent URL

https://peer.asee.org/25884

Download Count

95

Request a correction

Paper Authors

biography

Cathy P. Lachapelle Museum of Science

visit author page

Cathy Lachapelle leads the EiE team responsible for assessment and evaluation of our curricula. This includes the design and field-testing of assessment instruments and research on how children use EiE materials. Cathy is particularly interested in how collaborative interaction and scaffolded experiences with disciplinary practices help children learn science, math, and engineering. Her work on other STEM education research projects includes the national Women's Experiences in College Engineering (WECE) study. Cathy received her S.B. in cognitive science from the Massachusetts Institute of Technology and her Ph.D. in educational psychology from Stanford University.

visit author page

biography

Christine M. Cunningham Museum of Science

visit author page

Dr. Christine Cunningham is an educational researcher who works to make engineering and science more relevant, accessible, and understandable, especially for underserved and underrepresented populations. A vice president at the Museum of Science, Boston since 2003, she founded and directs Engineering is Elementary™, a groundbreaking project that integrates engineering concepts into elementary curriculum and teacher professional development. As of September 2014, EiE has served 6.2 million children nationwide and 71,000 educators. Cunningham has previously served as director of engineering education research at the Tufts University Center for Engineering Educational Outreach, where her work focused on integrating engineering with science, technology, and math in professional development for K-12 teachers. She also directed the Women’s Experiences in College Engineering (WECE) project, the first national, longitudinal, large-scale study of the factors that support young women pursuing engineering degrees. Cunningham is a Fellow of the American Society for Engineering Education and was awarded the 2014 International Society for Design and Development in Education Prize. She holds B.A. and M.A. degrees in biology from Yale and a Ph.D. in Science Education from Cornell University.

visit author page

Download Paper |

Abstract

With the new emphasis on engineering practices and engineering design (NGSS Lead States, 2013), teachers of and researchers studying K-12 engineering need to find ways to measure students’ developing engineering skills. To efficiently measure student learning of engineering practices, there is need for a tool to capture student performances in a way that readily affords evaluation and scalability. The problem we pursue in this paper is how to accomplish this measurement. Can we evaluate individual elementary student skills with a quick pencil and paper design task? What aspects of performance does the measure capture? How do these aspects of performance match up with the intended measurement goals? Can the instrument be efficiently and reliably coded using a rubric, so that researchers and teachers can make use of it? To address this need, we developed a performance assessment. Performance assessments are a form of contextual assessment where students engage in tasks within a context that affords the use of practices of interest to the assessor (Klassen, 2006). Students are presented with three quick design challenges to choose from. Each is presented as a written thought experiment with follow-up questions. During development of the instrument, as part of our process of gathering evidence for validity, we conducted think-aloud protocols with a dozen students in the target age range (8-11) who were learning engineering, to inform design and ensure that students were interpreting the instrument as intended. To characterize quality of performance on the written assessment, we developed a rubric that focuses solely on aspects of the NGSS Engineering DCI and Practices that we expected to see, in particular (a) generating multiple possible solutions, (b) evaluating a potential solution against criteria and constraints, (c) planning an investigation, and (d) communicating information (a design plan). Using the rubric, we individually scored 1531 written assessments from 276 grades 3-5 classrooms. Participating classrooms hailed from schools in 3 states, from a variety of urban / suburban / rural contexts, with a variety of racial / ethnic demographics. In this paper, we present the instrument and coding rubric. We calculate inter-rater reliability for coders and present descriptive statistics for student scores to demonstrate the utility of the instrument for distinguishing a range of performances. To build a case for validity for use of the assessment to measure student learning of practices, we compare video of 30 students working on design challenges in their student groups, collected from 10 of the participating classrooms, to the same students’ performance on the assessment. This also informs the use and limits of utility of the written performance assessment for measuring elementary students’ engineering skills and understanding-in-use. Finally, we describe the time needed to score the assessments, and discuss its utility for larger-scale research studies.

Klassen, S. (2006). Contextual assessment in science education: Background, issues, and policy. Science Education, 90(5), 820–851. NGSS Lead States. (2013). Next Generation Science Standards: For States, By States. Washington, DC: The National Academies Press.

Lachapelle, C. P., & Cunningham, C. M. (2016, June), Performance Assessment in Elementary Engineering: Evaluating Student (RTP) Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. 10.18260/p.25884

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2016 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015