Board 157: Designing Engineers Student Survey: Instrument Development and Preliminary Psychometric Data

Tobin N. Walton; Robin Guill Liles

Download Paper | Permalink

Conference: 2018 ASEE Annual Conference & Exposition
Location: Salt Lake City, Utah
Publication Date: June 23, 2018
Start Date: June 23, 2018
End Date: July 27, 2018
Conference Session: NSF Grantees Poster Session
Tagged Topics: Diversity and NSF Grantees Poster Session
Page Count: 27
DOI: 10.18260/1-2--29960
Permanent URL: https://peer.asee.org/29960
Download Count: 575

Paper Authors

biography

Tobin N. Walton North Carolina A&T State University

visit author page

My research is focused on developing interdisciplinary theoretical frameworks and methodological designs capable of modeling the social and psychological drivers of behavior, decision-making, and information processing across multiple domains (e.g., health, education, the workplace).

visit author page

biography

Robin Guill Liles North Carolina A&T State University

visit author page

Robin Guill Liles is a professor in the Department of Counseling at North Carolina A&T State University. She is also Co-PI for the IUSE/PFE:RED: A Revolution in Engineering Education Motivated by Needs and Design and the Associate Director for Assessment for the ERC for Revolutionizing Metallic Biomaterials.

visit author page

Download Paper | Permalink

Abstract

This paper reports on preliminary results of research activities associated with a recently awarded (July, 2017), RED grant was awarded to one university to revolutionize engineering undergraduate curricula experiences. The purpose of this specific RED project is to infuse design teaching and learning throughout general and specialized engineering curricula and to study the effects of these changes upon enrollment and student success (particularly among women and underrepresented minorities). Pragmatism is at the heart of these curricula changes whereby it is hypothesized that as students are increasingly able to identify “needs” and see design capability as a “solution” to real world problems, they will be more likely to value their engineering education, experience greater self-efficacy as engineers, and ultimately internalize strong engineering identities. In addition to curricula changes, other changes to the teaching and learning experience will include: (a) annual summer design instructor workshops and labs; (b) full-time, on-campus design and testing summer research experience; and (c) off-campus “forward internships” (again, during summer) with industry partners.

Programmatic efficacy will be measured in a manner consistent with accredited engineering programs (i.e., ABET Standards a-k), as well as by the university’s institutional effectiveness protocols guided by the Southern Association of Colleges and Schools (SACS) accreditation standards. Enrollment, retention, graduation, and post-graduation data will also be collected. Beyond these conventional measures, four additional outcome measures have been identified to evaluate programmatic efficacy, specifically:

1. How well program graduates understand the design process and its application to solve needs. 2. How highly program graduates value the learning of knowledge, skills, and practices related to engineering. 3. How robustly program graduates experience feelings of self-efficacy related to their engineering studies. 4. The extent to which program graduates have internalized a strong engineering identity. .

To measure knowledge of design, value, self-efficacy, and identity a survey instrument – Designing Engineers Student Survey – has been developed. This paper reports on the development of this instrument in terms of the social psychological theory that grounds it, the processes for initial instrument domain identification, and preliminary statistical analyses of the first wave of survey results (n = 424) collected from the university’s engineering freshmen class (2017).

The psychometric properties of the scales measuring knowledge of design, value, self-efficacy, and identity will be presented. This will include tests of reliability and internal consistency (i.e., Cronbach’s alpha & item-analysis), dimensionality (i.e., Principal Component Analysis and Exploratory Factor Analysis), and convergent and discriminant validity (i.e., correlation analysis). Additionally, an information-based multi-model comparative approach to Structural Equation Modeling (SEM), will be used to assess portions of the theory of learning and motivation that grounds the overall RED project. It is hoped that the results will render new “components” for understanding value, self-efficacy, and identity among students from under-represented populations striving to become professional engineers.

Citation
Format

Walton, T. N., & Liles, R. G. (2018, June), Board 157: Designing Engineers Student Survey: Instrument Development and Preliminary Psychometric Data Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. 10.18260/1-2--29960

TY  - CPAPER
AB  - This paper reports on preliminary results of research activities associated with a recently awarded (July, 2017), RED grant was awarded to one university to revolutionize engineering undergraduate curricula experiences.  The purpose of this specific RED project is to infuse design teaching and learning throughout general and specialized engineering curricula and to study the effects of these changes upon enrollment and student success (particularly among women and underrepresented minorities).  Pragmatism is at the heart of these curricula changes whereby it is hypothesized that as students are increasingly able to identify “needs” and see design capability as a “solution” to real world problems, they will be more likely to value their engineering education, experience greater self-efficacy as engineers, and ultimately internalize strong engineering identities.  In addition to curricula changes, other changes to the teaching and learning experience will include: (a) annual summer design instructor workshops and labs; (b) full-time, on-campus design and testing summer research experience; and (c) off-campus “forward internships” (again, during summer) with industry partners.

Programmatic efficacy will be measured in a manner consistent with accredited engineering programs (i.e., ABET Standards a-k), as well as by the university’s institutional effectiveness protocols guided by the Southern Association of Colleges and Schools (SACS) accreditation standards. Enrollment, retention, graduation, and post-graduation data will also be collected.  Beyond these conventional measures, four additional outcome measures have been identified to evaluate programmatic efficacy, specifically: 

1.	How well program graduates understand the design process and its application to solve needs.
2.	How highly program graduates value the learning of knowledge, skills, and practices related to engineering.
3.	How robustly program graduates experience feelings of self-efficacy related to their engineering studies.
4.	The extent to which program graduates have internalized a strong engineering identity. .

To measure knowledge of design, value, self-efficacy, and identity a survey instrument – Designing Engineers Student Survey – has been developed.  This paper reports on the development of this instrument in terms of the social psychological theory that grounds it, the processes for initial instrument domain identification, and preliminary statistical analyses of the first wave of survey results  (n =  424) collected from the university’s engineering freshmen class (2017).  

The psychometric properties of the scales measuring knowledge of design, value, self-efficacy, and identity will be presented.  This will include tests of reliability and internal consistency (i.e., Cronbach’s alpha &amp; item-analysis), dimensionality (i.e., Principal Component Analysis and Exploratory Factor Analysis), and convergent and discriminant validity (i.e., correlation analysis).  Additionally, an information-based multi-model comparative approach to Structural Equation Modeling (SEM), will be used to assess portions of the theory of learning and motivation that grounds the overall RED project.  It is hoped that the results will render new “components” for understanding value, self-efficacy, and identity among students from under-represented populations striving to become professional engineers.


AU  - Tobin N. Walton
AU  - Robin Guill Liles
CY  - Salt Lake City, Utah
DA  - 2018/06/23
PB  - ASEE Conferences
TI  - Board 157: Designing Engineers Student Survey: Instrument Development and Preliminary Psychometric Data
UR  - https://peer.asee.org/29960
DO  - 10.18260/1-2--29960
ER  -