Moore, T. J., & Guzey, S. S., & Holly , J. (2015, June), An Assessment Tool to Evaluate Student Learning of Engineering (Fundamental)  Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23516

\bibitem{asee_peer_23516}
  Moore, T. J., \& Guzey, S. S., \& Holly , J. (2015, June), \emph{An Assessment Tool to Evaluate Student Learning of Engineering (Fundamental)}
  Paper presented at 2015 ASEE Annual Conference \& Exposition, Seattle, Washington.
  10.18260/p.23516

Tamara J Moore, Siddika  Selcen Guzey, and James Holly  Jr..  "An Assessment Tool to Evaluate Student Learning of Engineering (Fundamental)".  2015 ASEE Annual Conference & Exposition, Seattle, Washington, 2015, June.  ASEE Conferences, 2015.  https://peer.asee.org/23516 Internet.  22 Jun, 2024

\bibitem{asee_peer_23516}
  Tamara J Moore, Siddika  Selcen Guzey, and James Holly  Jr..
  "An Assessment Tool to Evaluate Student Learning of Engineering (Fundamental)".
  \emph{2015 ASEE Annual Conference \& Exposition, Seattle, Washington, 2015, June}.
  ASEE Conferences, 2015.
  https://peer.asee.org/23516 Internet.  22 Jun, 2024

@INPROCEEDINGS{asee_peer_23516
author = "Tamara J Moore, Siddika  Selcen Guzey, and James Holly  Jr."
title = "An Assessment Tool to Evaluate Student Learning of Engineering (Fundamental)"
booktitle = "2015 ASEE Annual Conference \& Exposition"
year = "2015"
month = "June"
address = "Seattle, Washington"
publisher = "ASEE Conferences"
note = {https://peer.asee.org/23516}
number = {10.18260/p.23516}
}

TY  - CPAPER
AB  -                 An Assessment Tool to Evaluate Student Learning of                              Engineering (Fundamental)While STEM subjects have traditionally been taught separately in K-12 schools the newinitiatives share a focus on integrated approaches to teaching STEM. For example, the recentlyreleased Next Generation Science Standards (NGSS) addressed the need for explicit integrationof science with engineering. Science teachers are expected to teach intersecting concepts andcore disciplinary science using scientific and engineering practices. One challenge that manyscience teachers face is the lack of measurement instruments for student learning required in theNGSS. Measuring the student understanding of engineering, connections between science andengineering, and engineering practices requires the development of new assessments. Thepurpose of this study was to develop, scale, and validate an assessment in engineering. The useof item response theory to assess item functioning was a focus of the study. The work is part ofa larger project focused on increasing student learning in STEM-related areas in Grades 4 – 8through an engineering design-based, integrated approach to STEM instruction and assessment.The test construction process began with an assessment development team of academicresearchers with collective expertise in engineering, science, and mathematics. Researcherswrote 21 multiple-choice items and content and face validity of the items were established byexpert review. Next the assessment was piloted in two waves with the goal of identifying finalversions of each assessment via item analyses and scaling students’ responses to produce anestimate of their proficiency. In the first wave the assessment was administered to a small groupof about 10-20 middle school students. The purpose was to obtain and analyze preliminary itemresponse data as well as obtain feedback on characteristics of the assessments and the testingenvironment that affected performance (e.g., readability, clarity of items). Based on informationprovided by wave 1 piloting the assessments were modified by the assessment developmentteam.        In the second wave of piloting the revised assessments were administered toapproximately 150 students. Data from wave two were used to conduct extensive item analysesof the assessments. We initially performed a factor analysis of the data for each assessment formiddle school students. The goal of these analyses was to assess the likelihood that itemsrepresented a single underlying construct. The results suggested that a single (major) factorunderlies the assessment, a finding that was robust to different methods of factoring (principalaxis, maximum likelihood) and factor rotation (varimax, oblique). These results suggests that thegoal of constructing items for the assessment that reflected a single factor was generally met, andalso helped to justify the use of item response theory (IRT) to generate proficiency scores forstudents. Next the Rasch IRT model was fitted to the data for each item on each assessment toestimate that item’s ability to contribute to estimates of student proficiency. The Rasch modeladequately fits the data of all 21 items.       In this presentation, we will present the development and reliability of the engineeringassessment and share the assessment tool with the participants.
AU  - Tamara J Moore
AU  - Siddika  Selcen Guzey
AU  - James Holly  Jr.
CY  - Seattle, Washington
DA  - 2015/06/14
PB  - ASEE Conferences
TI  - An Assessment Tool to Evaluate Student Learning of Engineering (Fundamental)
UR  - https://peer.asee.org/23516
DO  - 10.18260/p.23516
ER  -