Asee peer logo

Board 157: Designing Engineers Student Survey: Instrument Development and Preliminary Psychometric Data

Download Paper |


2018 ASEE Annual Conference & Exposition


Salt Lake City, Utah

Publication Date

June 23, 2018

Start Date

June 23, 2018

End Date

July 27, 2018

Conference Session

NSF Grantees Poster Session

Tagged Topics

Diversity and NSF Grantees Poster Session

Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Tobin N. Walton North Carolina A&T State University

visit author page

My research is focused on developing interdisciplinary theoretical frameworks and methodological designs capable of modeling the social and psychological drivers of behavior, decision-making, and information processing across multiple domains (e.g., health, education, the workplace).

visit author page


Robin Guill Liles North Carolina A&T State University

visit author page

Robin Guill Liles is a professor in the Department of Counseling at North Carolina A&T State University. She is also Co-PI for the IUSE/PFE:RED: A Revolution in Engineering Education Motivated by Needs and Design and the Associate Director for Assessment for the ERC for Revolutionizing Metallic Biomaterials.

visit author page

Download Paper |


This paper reports on preliminary results of research activities associated with a recently awarded (July, 2017), RED grant was awarded to one university to revolutionize engineering undergraduate curricula experiences. The purpose of this specific RED project is to infuse design teaching and learning throughout general and specialized engineering curricula and to study the effects of these changes upon enrollment and student success (particularly among women and underrepresented minorities). Pragmatism is at the heart of these curricula changes whereby it is hypothesized that as students are increasingly able to identify “needs” and see design capability as a “solution” to real world problems, they will be more likely to value their engineering education, experience greater self-efficacy as engineers, and ultimately internalize strong engineering identities. In addition to curricula changes, other changes to the teaching and learning experience will include: (a) annual summer design instructor workshops and labs; (b) full-time, on-campus design and testing summer research experience; and (c) off-campus “forward internships” (again, during summer) with industry partners.

Programmatic efficacy will be measured in a manner consistent with accredited engineering programs (i.e., ABET Standards a-k), as well as by the university’s institutional effectiveness protocols guided by the Southern Association of Colleges and Schools (SACS) accreditation standards. Enrollment, retention, graduation, and post-graduation data will also be collected. Beyond these conventional measures, four additional outcome measures have been identified to evaluate programmatic efficacy, specifically:

1. How well program graduates understand the design process and its application to solve needs. 2. How highly program graduates value the learning of knowledge, skills, and practices related to engineering. 3. How robustly program graduates experience feelings of self-efficacy related to their engineering studies. 4. The extent to which program graduates have internalized a strong engineering identity. .

To measure knowledge of design, value, self-efficacy, and identity a survey instrument – Designing Engineers Student Survey – has been developed. This paper reports on the development of this instrument in terms of the social psychological theory that grounds it, the processes for initial instrument domain identification, and preliminary statistical analyses of the first wave of survey results (n = 424) collected from the university’s engineering freshmen class (2017).

The psychometric properties of the scales measuring knowledge of design, value, self-efficacy, and identity will be presented. This will include tests of reliability and internal consistency (i.e., Cronbach’s alpha & item-analysis), dimensionality (i.e., Principal Component Analysis and Exploratory Factor Analysis), and convergent and discriminant validity (i.e., correlation analysis). Additionally, an information-based multi-model comparative approach to Structural Equation Modeling (SEM), will be used to assess portions of the theory of learning and motivation that grounds the overall RED project. It is hoped that the results will render new “components” for understanding value, self-efficacy, and identity among students from under-represented populations striving to become professional engineers.

Walton, T. N., & Liles, R. G. (2018, June), Board 157: Designing Engineers Student Survey: Instrument Development and Preliminary Psychometric Data Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. 10.18260/1-2--29960

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2018 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015