Asee peer logo

Comparing Disparate Outcome Measures for Better Understanding of Engineering Graduates

Download Paper |


2015 ASEE Annual Conference & Exposition


Seattle, Washington

Publication Date

June 14, 2015

Start Date

June 14, 2015

End Date

June 17, 2015





Conference Session

Assessment and Outcomes: ERM Roundtable

Tagged Division

Educational Research and Methods

Tagged Topic


Page Count


Page Numbers

26.371.1 - 26.371.21



Permanent URL

Download Count


Request a correction

Paper Authors


Samantha Ruth Brunhaver Arizona State University

visit author page

Samantha Brunhaver is an Assistant Professor of Engineering in the Fulton Schools of Engineering Polytechnic School. She completed her graduate work in Mechanical Engineering at Stanford University. She also has a B.S. in Mechanical Engineering from Northeastern University. Her research examines the career decision-making and professional identity formation of engineering students, alumni, and practicing engineers. She also conducts studies of new engineering pedagogy that help to improve student engagement and understanding.

visit author page

author page

Shannon Katherine Gilmartin Stanford University Orcid 16x16


Helen L. Chen Stanford University

visit author page

Helen L. Chen is a research scientist in the Designing Education Lab in the Department of Mechanical Engineering and the Director of ePortfolio Initiatives in the Office of the Registrar at Stanford University. She is also a member of the research team in the National Center for Engineering Pathways to Innovation (Epicenter). Chen earned her undergraduate degree from UCLA and her Ph.D. in Communication with a minor in Psychology from Stanford University in 1998. Her current research interests include: 1) engineering and entrepreneurship education; 2) the pedagogy of ePortfolios and reflective practice in higher education; and 3) reimagining the traditional academic transcript.

visit author page

author page

Holly M Matusovich Virginia Tech

author page

Sheri Sheppard Stanford University

Download Paper |


Comparing Disparate Outcome Measures for Better Understanding of Engineering GraduatesInformation about (and from) engineering graduates is gathered for many purposes and in many ways.Federal government agencies survey graduates about their employment and graduate school attainmentsin order to make projections and policy. Engineering programs report graduates’ importance andpreparedness ratings of the ABET a-k outcomes relative to their current job for the purposes ofdemonstrating continuous improvement. In addition, many researchers have asked graduates whether theysee their job or identity as “engineer” in order to classify them as retained or not retained in theengineering workforce. In each scenario, there are strengths and limitations to what the informationcollected can reveal about the current state of engineering education and practice. For example, whilegraduates’ occupations and fields of study connote specific meaning and are easy to understand,measuring engineering retention along these lines excludes those who may use their engineeringbackground in non-traditional contexts. A deeper understanding of engineering graduates’ roles andidentities could result from comparing multiple measures of their post-graduation outcomes, and yet, datawith which to make such comparisons have been scarce.This paper presents findings from an online, NSF-sponsored survey of recent engineering bachelor’sgraduates that was created (in part) to address this issue. In addition to questions about their precollegeand college experiences, survey participants responded to measures about their current level ofengagement with engineering and their undergraduate engineering education. These measures wereborrowed from previous literature on engineering graduate outcomes and asked participants to reflect on(1) their current (or most recent) occupation, (2) their perceptions of their current position as engineeringor non-engineering, (3) their engineering identity, (4) their engineering self-efficacy, (5) the importanceof engineering competencies to their work, and (6) the relatedness between their current position andundergraduate education.The survey was deployed in autumn of 2011 to engineering graduates four years out from earning theirbachelor’s degrees from four U.S. institutions. The survey received 484 responses, representing 26percent of the mail-out sample and 19 percent of all 2007 engineering bachelor’s graduates across the fourschools. For this study, participants’ responses to the engineering-related measures described above wereexamined and compared using non-parametric statistical tests, as appropriate. Prior to analysis, the samplewas weighted for varying institutional sampling rates and individual nonresponse bias to approximateresponses had all invited graduates participated.Preliminary results show that more engineering graduates perceived their current position as engineering(versus non-engineering) than marked that they had an engineering occupation or were enrolled in agraduate engineering degree program. Graduates’ engineering identity, engineering self-efficacy, andratings of the importance of engineering competencies to their work were also more positively andsignificantly correlated with their self-perceptions of doing engineering rather than with their actualemployment or enrollment statuses. Perceived relatedness between graduates’ current position andundergraduate education did not track with their perceived or actual participation in engineeringwork/studies, however, and ratings of perceived relatedness were uniformly low across respondents.These results suggest that engineering programs are successful in instilling students with a strong sense ofengineering self-concept but less successful in showing students how their engineering education appliesto “real-life”. Implications for engineering education, policy, and professional practice will be discussed.

Brunhaver, S. R., & Gilmartin, S. K., & Chen, H. L., & Matusovich, H. M., & Sheppard, S. (2015, June), Comparing Disparate Outcome Measures for Better Understanding of Engineering Graduates Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23710

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015