June 20, 2010
June 20, 2010
June 23, 2010
K-12 & Pre-College Engineering
15.199.1 - 15.199.13
Assessing Elementary Students’ Understanding of Engineering and Technology Concepts
Engineering is Elementary’s newest large-scale assessments are much improved over early attempts, thanks to innovation and improvement in the development process. Because engineering is so sensitive to specifics of a situation, and because multiple solutions are nearly always possible, targeting engineering “knowledge” and “know-how” is often best done with rubrics in the classroom. For large-scale assessment, the development of assessment questions must mirror the development of learning objectives for the curriculum. It is vital to develop questions assessing each learning objective from a number of points of view, in a variety of ways, especially in early stages of assessment development when it is unclear how students will interpret questions. Cognitive interviews with students allow for testing of content validity of questions. The inclusion of questions of a variety of difficulties, with some repetition of content assessed, improves statistical normality and reliability. Avoiding “teaching” questions that give away answers to other questions on the test is a particular difficulty when assessing engineering. This paper addresses content questions; the issue of assessing skills and process knowledge is left for future research.
Engineering is Elementary (EiE) is a research-based curriculum development project focused on creating curriculum units that cover topics in engineering and technology as a supplement to core science instruction. Each EiE curriculum unit is designed to build on and reinforce one science topic through the exploration and development of a related technology. EiE has been committed from the project’s inception in 2003 to assessing students’ knowledge about engineering and technology, and measuring the impact of EiE on student knowledge and attitudes. EiE is also committed to measuring the impact of EiE on student skills and procedural knowledge; however, this effort is only in the most nascent stages of development because of the challenges associated with such assessment.
This paper describes EiE’s assessment development process, as well as lessons learned by EiE over six years of design and improvement of engineering curriculum assessments. EiE has been a pioneer in this area: few instruments designed for elementary school students exist which measure student understanding of engineering and technology concepts. Over six years, EiE has developed and refined a number of instruments for measuring statistically significant change in the engineering understanding of large samples of students. These instruments have been multiple measure assessments (pre-post). Some were designed to assess understanding of basic engineering and technology content (such as “What is Engineering?”), while others have focused on measuring STEM learning resulting from student interaction with specific EiE units.
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2010 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015