A Natural Experiment: NAE's Changing the Conversation Report and Students' Changing Perceptions of Engineering

Elizabeth Litzler; Julie Ann Lorah

Download Paper | Permalink

Conference: 2013 ASEE Annual Conference & Exposition
Location: Atlanta, Georgia
Publication Date: June 23, 2013
Start Date: June 23, 2013
End Date: June 26, 2013
ISSN: 2153-5965
Conference Session: Undergraduate Student Issues: Culture
Tagged Division: Women in Engineering
Page Count: 17
Page Numbers: 23.77.1 - 23.77.17
DOI: 10.18260/1-2--19091
Permanent URL: https://peer.asee.org/19091
Download Count: 390

Request a correction

Paper Authors

biography

Elizabeth Litzler University of Washington

visit author page

Elizabeth Litzler, Ph.D., is the Director for Research at the University of Washington (UW) Center for Workforce Development (CWD) and an Affiliate Assistant Professor in UW Sociology. She directs research projects from conceptualization, methodological design, collection of data and analysis, to dissemination of research findings. Dr. Litzler manages the Sloan-funded Project to Assess Climate in Engineering (PACE), which uses quantitative and qualitative methods to examine the culture for women and underrepresented minorities in 22 engineering colleges nationwide. She also directs the external evaluation for the National Center for Women & Information Technology (NCWIT). In addition to her leadership in the office, Dr. Litzler is a member of the American Society for Engineering Education (ASEE) and a Board Member of Women in Engineering ProActive Network (WEPAN). Her research interests include the educational climate for students in science and engineering and gender and race stratification in education and the workforce.

visit author page

biography

Julie Ann Lorah University of Washington

visit author page

Julie Lorah is a Ph.D. candidate at the University of Washington in the measurement, statistics, and research design program of educational psychology. She is a research assistant at the University of Washington (UW) Center for Workforce Development (CWD) where she works on quantitative data analysis and coordination for the Sloan-funded Project to Assess Climate in Engineering (PACE). Her research interests include applications of statistical methodology and opportunity for under-represented groups.

visit author page

Download Paper | Permalink

Abstract

A Natural Experiment: NAE’s Changing the Conversation Report and Students’ Changing Perceptions of EngineeringThe National Academy of Engineering’s Changing the Conversation report providesrecommendations for how to talk to students about engineering to engage them, interest themand keep them in the field (NAE, 2008). The messages portray engineering as a creativeendeavor that can help others. Specific messages suggested by Changing the Conversationinclude: Engineers are creative problem-solvers, Engineers make a world of difference,Engineering is essential to our health, happiness and safety, and Engineers help shape the future(NAE, 2008).The Project to Assess Climate in Engineering (PACE) administered a climate survey in 21engineering colleges in 2008 which garnered over 10,000 undergraduate student responses. As aresult of the 2008 PACE survey findings, the PACE schools implemented interventions designedto address some of the climate conditions that surfaced in their final reports. PACE was re-funded by the Alfred P. Sloan Foundation to re-administer the survey in 2012. Administrationoccurred at 15 of the 21 original PACE schools in 2012 and over 10,000 students responded.One goal of the PACE study is to help schools understand how their climate has changed overthe past four years, and how those changes might be related to interventions the schoolimplemented.The timing of the PACE surveys and the publication of the Changing the Conversation report in2008 results in an interesting natural experiment (pseudo). This paper will examine changes atthe 15 PACE schools from 2008 to 2012 specifically around survey items about perceptions ofthe engineering field, perceptions of flexibility in engineering, and intended persistence.Independent group analyses were utilized since the data was not truly longitudinal. Initial resultssuggest that schools that have utilized the Changing the Conversation materials to a greaterextent have seen statistically significant gains in student perceptions of the engineering field.The analysis will examine changes over time for women, men, under-represented minorities,non-under-represented minorities, and all students in the aggregate. By disaggregating theresults by demographic group it is easier to tell whether the messaging is disproportionatelyimpacting one group over another.An example of a preliminary finding is that at four of the 15 schools there were statisticallysignificant increases for all students in the aggregate on the means for the survey items: Societyvalues the work engineers do and Engineers help make the world a better place. Seven of theschools had significant increases in the means for all students on the survey item: I expect thatengineering will be a rewarding career. Table 1 is provided to give the reader a sense of theschool level findings across the two of the three areas of interest (perceptions of the engineeringfield, flexibility of engineering, and intent to persist).The paper will include information on the decreases in means that occurred at the 15 schools,descriptions of activities from the schools that saw key improvements on the Changing theConversation related survey items, additional correlational analysis of the relationship betweenperceptions of engineering and responses to the intent to persist, and other related analyses.Results will be reported at both the school level and the individual level.This research paper will provide some evidence of the effectiveness of the Changing theConversation interventions at the PACE schools and will result in useful information that isbroadly generalizable to engineering schools.This could be a research paper, or could possibly become a panel with speakers from PACE andthe schools that were successful in changing their students’ perceptions of engineering. Thiscould also be in a joint WIED/MIND as the groups examined include women and minorities.Table 1. Number of schools (out of 15) that have statistically significant increases in means from2008 to 2012 for each demographic group All Non- Students Females Males URMs URMsPerceptions of Engineering Society values the work engineers do 4 4 3 5 0 Engineering is an occupation that is 2 3 2 2 0 respected by other people Engineers help to make the world a better 4 4 5 6 2 place. I expect that engineering will be a 7 7 7 8 1 rewarding careerFlexibility of Engineering Engineers can leave and come back to their careers more easily than can people 5 2 4 5 0 in other professions Engineering is a field that supports people who want to have children and continue 4 4 3 6 0 working Engineers can design their own work 4 2 0 3 2 schedules Engineering is a field that supports a 5 2 3 4 0 balance between work and family life Engineers are well-paid 5 3 3 3 2

Citation
Format

Litzler, E., & Lorah, J. A. (2013, June), A Natural Experiment: NAE's Changing the Conversation Report and Students' Changing Perceptions of Engineering Paper presented at 2013 ASEE Annual Conference & Exposition, Atlanta, Georgia. 10.18260/1-2--19091

TY  - CPAPER
AB  -         A Natural Experiment: NAE’s Changing the Conversation Report              and Students’ Changing Perceptions of EngineeringThe National Academy of Engineering’s Changing the Conversation report providesrecommendations for how to talk to students about engineering to engage them, interest themand keep them in the field (NAE, 2008). The messages portray engineering as a creativeendeavor that can help others. Specific messages suggested by Changing the Conversationinclude: Engineers are creative problem-solvers, Engineers make a world of difference,Engineering is essential to our health, happiness and safety, and Engineers help shape the future(NAE, 2008).The Project to Assess Climate in Engineering (PACE) administered a climate survey in 21engineering colleges in 2008 which garnered over 10,000 undergraduate student responses. As aresult of the 2008 PACE survey findings, the PACE schools implemented interventions designedto address some of the climate conditions that surfaced in their final reports. PACE was re-funded by the Alfred P. Sloan Foundation to re-administer the survey in 2012. Administrationoccurred at 15 of the 21 original PACE schools in 2012 and over 10,000 students responded.One goal of the PACE study is to help schools understand how their climate has changed overthe past four years, and how those changes might be related to interventions the schoolimplemented.The timing of the PACE surveys and the publication of the Changing the Conversation report in2008 results in an interesting natural experiment (pseudo). This paper will examine changes atthe 15 PACE schools from 2008 to 2012 specifically around survey items about perceptions ofthe engineering field, perceptions of flexibility in engineering, and intended persistence.Independent group analyses were utilized since the data was not truly longitudinal. Initial resultssuggest that schools that have utilized the Changing the Conversation materials to a greaterextent have seen statistically significant gains in student perceptions of the engineering field.The analysis will examine changes over time for women, men, under-represented minorities,non-under-represented minorities, and all students in the aggregate. By disaggregating theresults by demographic group it is easier to tell whether the messaging is disproportionatelyimpacting one group over another.An example of a preliminary finding is that at four of the 15 schools there were statisticallysignificant increases for all students in the aggregate on the means for the survey items: Societyvalues the work engineers do and Engineers help make the world a better place. Seven of theschools had significant increases in the means for all students on the survey item: I expect thatengineering will be a rewarding career. Table 1 is provided to give the reader a sense of theschool level findings across the two of the three areas of interest (perceptions of the engineeringfield, flexibility of engineering, and intent to persist).The paper will include information on the decreases in means that occurred at the 15 schools,descriptions of activities from the schools that saw key improvements on the Changing theConversation related survey items, additional correlational analysis of the relationship betweenperceptions of engineering and responses to the intent to persist, and other related analyses.Results will be reported at both the school level and the individual level.This research paper will provide some evidence of the effectiveness of the Changing theConversation interventions at the PACE schools and will result in useful information that isbroadly generalizable to engineering schools.This could be a research paper, or could possibly become a panel with speakers from PACE andthe schools that were successful in changing their students’ perceptions of engineering. Thiscould also be in a joint WIED/MIND as the groups examined include women and minorities.Table 1. Number of schools (out of 15) that have statistically significant increases in means from2008 to 2012 for each demographic group                                                   All                           Non-                                               Students Females Males URMs URMsPerceptions of Engineering  Society values the work engineers do              4          4        3        5        0  Engineering is an occupation that is                                                    2          3        2        2        0  respected by other people  Engineers help to make the world a better                                                    4          4        5        6        2  place.  I expect that engineering will be a                                                    7          7        7        8        1  rewarding careerFlexibility of Engineering  Engineers can leave and come back to  their careers more easily than can people         5          2        4        5        0  in other professions  Engineering is a field that supports people  who want to have children and continue            4          4        3        6        0  working  Engineers can design their own work                                                    4          2        0        3        2  schedules  Engineering is a field that supports a                                                    5          2        3        4        0  balance between work and family life  Engineers are well-paid                           5          3        3        3        2
AU  - Elizabeth Litzler
AU  - Julie Ann Lorah
CY  - Atlanta, Georgia
DA  - 2013/06/23
PB  - ASEE Conferences
TI  - A Natural Experiment: NAE's Changing the Conversation Report and Students' Changing Perceptions of Engineering
UR  - https://peer.asee.org/19091
DO  - 10.18260/1-2--19091
ER  -