Atlanta, Georgia
June 23, 2013
June 23, 2013
June 26, 2013
2153-5965
Educational Research and Methods
26
23.1094.1 - 23.1094.26
10.18260/1-2--22479
https://peer.asee.org/22479
3189
Malinda Faber is a Research Associate on the Evaluation Team at the Friday Institute for Educational Innovation at North Carolina State University. Malinda works on multiple research teams, including evaluations of The Golden LEAF Foundations’ STEM Initiative, North Carolina’s Race to the Top STEM Initiative, and the North Carolina Department of Public Instruction’s IMPACT-V Model for capacity-building in instructional technology.
Alana Unfried is a Graduate Research Assistant at the Friday Institute for Educational Innovation at North Carolina State University. She works on the Data Analytics team for the MISO Project (Maximizing the Impact of STEM Outreach through Data-Driven Decision Making), funded by the National Science Foundation. Alana’s responsibilities include the development of statistically sound evaluation instruments for teachers and students involved in these campus outreach programs. She also analyzes survey results and related data to understand the collective impact of these pre-college outreach programs. Alana is also a full-time Ph.D. student in the Department of Statistics at North Carolina State University.
Dr. Wiebe is a Professor in the Department of STEM Education at NC State University and Senior Research Fellow at the Friday Institute for Educational Innovation. A focus of his research and outreach work has been the integration of multimedia and multimodal teaching and learning approaches in STEM instruction. He has also worked on research and evaluation of technology integration in instructional settings in both secondary and post-secondary education. Dr. Wiebe has been a member of ASEE since 1989.
As the Director of Evaluation Programs at the Friday Institute for Educational Innovation at NC State University, Dr. Corn serves as PI of several large, statewide evaluation and research studies of innovations in K-12 schools and districts, including leading the evaluation of initiatives funded under North Carolina's Race to the Top grant. Her research interests focus on leadership, professional development, teaching and learning, infrastructure, and evaluation for technology-enhanced innovations in public school settings.
Tracey Louise Collins is the Project Coordinator for the MISO Project. Responsibilities include implementing activities of the project, coordinating efforts among K-12 science, technology, engineering, and mathematics (STEM) outreach programs, and working closely with university enrollment management and data management professionals at the Friday Institute at NC State University. She works closely with both large and small NC State outreach groups offering K-12 outreach to teachers and students. She also assists with planning, implementing, managing, and reporting of project activities which include survey development, coordination of data collection, interfacing with data managers, coordination of meetings and workshops for outreach providers to gather feedback, identify best practices, and disseminate findings to stakeholders. In addition, she assists with annual report writing and conference presentations. She has been a member of the American Society of Engineering Educators since 2011.
Survey Instruments for Measuring Student Attitudes toward STEMThe national economy is in need of engineers and allied skilled workers in science, technology,and mathematics (STEM) fields, and this need is only expected to grow over time. As part of on-going efforts to understand impacts of K-12 STEM education and workforce developmentprograms, surveys focusing on student attitudes towards STEM were developed by researchers atNorth Carolina State University. The Upper Elementary School and Middle/High School StudentAttitudes toward STEM (S-STEM) Surveys use a 5-point Likert-style response scale to measurestudents’ confidence and interest in mathematics, science, engineering and technology, and 21stcentury skills. The surveys use a 4-point interest-level scale to measure students’ interest invarious STEM career areas, including engineering. This paper describes the process fordeveloping these two valid, reliable survey instruments.The surveys’ STEM constructs were adapted from a survey created by evaluators of a program atthe engineering schools of Northeastern University, Tufts University, Worcester PolytechnicInstitute, and Boston University – the program was designed to increase female middle schoolstudents’ interest in engineering. The North Carolina Student Learning Conditions Surveyprovided the basis for the items measuring students’ confidence in their 21st century skills.Finally, the list of STEM subject career areas was derived from multiple national sources,including the National Academy for Engineering.A pilot version of the middle/high school survey was administered to students, and results wereused to conduct the first round of factor analysis and subject matter expert review. Results led tothe dropping of some items and the re-wording of others, especially for the purposes of reducinggender bias and refining measurement of technical and engineering skills. An upper elementaryversion of the survey was created from this initial, revised version. Each item was rewritten to areading level appropriate for fourth and fifth grade students, and other improvements were madebased upon results from cognitive interviews with fifth grade students and feedback fromsubject-matter experts.The revised middle/high school version of the S-STEM Survey was administered toapproximately 9,000 middle and high school students, with the upper elementary versionadministered to approximately 900 fourth and fifth graders. Using this response data, furthervalidity and reliability tests were conducted. Factor analysis results showed strong, clearconstructs with high reliability after dropping just a few items. Results from another round ofsubject-matter expert reviews demonstrated that both surveys were of appropriate length and atappropriate reading-levels. Differential item functioning analyses showed that 6th-12th graderscomprehended the survey in similar ways and that female and male students differed slightly intheir comprehension of the relationships between mathematics, science, and engineering andtechnology.The process described in this paper demonstrates that the Upper Elementary and Middle/HighSchool S-STEM Surveys are valid and reliable instruments. These surveys can serve as usefultools for measuring impacts of K-12 engineering and STEM education programs on studentattitudes, supporting critical efforts to improve the quality and quantity of engineering educationand workforce development.
Faber, M., & Unfried, A., & Wiebe, E. N., & Corn, J., & Townsend, L. W., & Collins, T. L. (2013, June), Student Attitudes toward STEM: The Development of Upper Elementary School and Middle/High School Student Surveys Paper presented at 2013 ASEE Annual Conference & Exposition, Atlanta, Georgia. 10.18260/1-2--22479
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2013 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015