June 14, 2015
June 14, 2015
June 17, 2015
Educational Research and Methods
26.1657.1 - 26.1657.17
Title: Using an Instrument Blueprint to Support the Rigorous Development ofNew Surveys and Assessments in Engineering EducationAbstract:Many sound methods exist for creating the items or questions that make up educational surveysand assessments. These methods include the use of content experts, reviews of existinginstruments, and lists of behaviors and descriptors commonly associated with the construct(s) wewish to assess. Unfortunately, however, item creation sometimes becomes overly dependentupon a researcher’s personal attitudes about the construct(s) being tested, or on “borrowing”items from other instruments that may or may not be sound measures of the construct(s) ofinterest. These risks are particularly likely for new researchers in engineering education, whomay have little experience with best practices in social science research.One way to support best practices in the development of new surveys and assessments is to usean instrument blueprint to guide the creation of items, as well as the collection of validityevidence. This paper outlines a process for instrument blueprint creation and content validationto help support best practices in educational assessment. Based on Messick’s unified theory ofvalidity, the instrument blueprint includes a process for item construction that incorporatesmultiple resources, including: (1) the views of content experts; (2) research from the relevantdomain of interest; (3) reviews of existing instruments; and (4) the expertise of the researchteam.This paper uses the development of a new instrument to measure engineering innovativeness asan illustrative example of the blueprinting process. Our new instrument will assess 20characteristics of innovative engineers as identified by in-depth studies of expert engineeringinnovators in previous research. This work highlights the positive impact of using a systematicprocess for item construction to transform current methods of assessment in engineeringeducation.
Menold, J., & Jablokow, K. W., & Purzer, S., & Ferguson, D. M., & Ohland, M. W. (2015, June), Using an Instrument Blueprint to Support the Rigorous Development of New Surveys and Assessments in Engineering Education Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24993
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015