Designing a Survey Instrument for a National Study of Direct-pathway and Returning Engineering Graduate Students

Erika A. Mosyjowski; Shanna R. Daly; Diane L Peters; Steven J. Skerlos

Download Paper | Permalink

Conference: 2013 ASEE Annual Conference & Exposition
Location: Atlanta, Georgia
Publication Date: June 23, 2013
Start Date: June 23, 2013
End Date: June 26, 2013
ISSN: 2153-5965
Conference Session: Research and Graduate Studies
Tagged Division: Graduate Studies
Page Count: 15
Page Numbers: 23.377.1 - 23.377.15
DOI: 10.18260/1-2--19391
Permanent URL: https://peer.asee.org/19391
Download Count: 385

Request a correction

Paper Authors

biography

Erika A. Mosyjowski University of Michigan

visit author page

Erika Mosyjowski works for the University of Michigan College of Engineering both as a student affairs professional and as a research associate in engineering education. She holds a Master's in Higher Education from the University of Michigan and a Bachelor's degree in Psychology and Sociology from Case Western Reserve University.

visit author page

biography

Shanna R. Daly University of Michigan orcid.org/0000-0002-4698-2973

visit author page

Shanna R. Daly is an Assistant Research Scientist at the University of Michigan in Engineering Education. She is also an Adjunct Assistant Professor, teaching courses in design and creativity, and a Program Manager at the Center for Research in Learning and Teaching in Engineering. Her research focuses on the investigation of design approaches and ideation, ethnography in design, foundations of innovation, creative processes, and engineering practitioners who return to academia for a PhD. Many of her studies involve multiple disciplinary professional and educational contexts and she often collaborates across disciplines, working with scholars in engineering, education, industrial design, and psychology.
She also brings a research lens to understanding how to support successful translation of educational research to practice, and conducts consultations and runs programs to support successful teaching and learning throughout the College of Engineering.

visit author page

author page

Diane L Peters University of Michigan

biography

Steven J. Skerlos University of Michigan

visit author page

Professor Steven J. Skerlos is Arthur F. Thurnau Professor at the University of Michigan. He is a tenured faculty member in Mechanical Engineering with an additional appointment in Civil and Environmental Engineering.

He is Director of the U-M Program in Sustainable Engineering and Co-Director of the Engineering Sustainable Systems Program.

visit author page

Download Paper | Permalink

Abstract

Designing a Survey Instrument for a National Study of Traditional and Returning Engineering Graduate StudentsThough a majority of engineering PhD students begin their doctoral career shortly aftercompleting an undergraduate degree (and perhaps a Master’s), a significant minority of studentsare “returners,” students who pursue a PhD after working outside of academia for five or moreyears. In the first phase of a three year NSF-funded study which aims to characterize thepopulation of returning engineering PhD students, explore the interactions of their previous workexperiences and their academic work, and investigate stakeholder views and institutional policiesrelated to returning PhD students, we developed a nationally-distributed survey to compareexperiences and perspectives of returners and traditional students.The survey development was grounded in Eccles’ Expectancy Value Theory (EVT), a theoreticalframework that explains how and why people make choices, based on the expected results ofthose choices, the costs required to make the choice, and their own interests and values. It wasalso informed by literature on returning students and by a pilot study that involved interviewswith ten returning students about their motivations for returning and their experiences duringgraduate school. The survey included questions about students’ motivation for returning, theirprevious work and school experience, their future career plans, the challenges of graduate school,and their strategies for adapting to these challenges.The development of our questionnaire included numerous levels of review and iteration. Ourresearch team created, discussed, and revised a list of questions based on our framework, pastresearch, and the personal experiences of current and former graduate students. After revisingseveral versions of the instrument, evaluating it for clarity, comprehensiveness, and relevance toour objectives, we asked our advisory board to review the survey. After again revising thesurvey based on our advisors’ feedback, we then piloted the survey with 7 current PhD studentsin STEM fields. We utilized a think-aloud cognitive interviewing technique to ensure thequestionnaire captured these students’ experiences as well as to check for the four components ofmeasurement error identified by Collins (2003), which include: comprehension problems,validity problems, processing difficulties, and pronunciation and communication difficulties.Participant feedback was carefully recorded and compiled in order to identify commondifficulties or suggestions. We then created the final draft of the survey based on this feedback.The focus of this paper and presentation will be on the development of the survey, in which wewill highlight best practices from the literature that informed our creation and refinement process.We will show iterations of the survey and data from the advisory board and our cognitiveinterviews that informed the final version of the instrument. We will also discuss the distributionof the instrument and some preliminary results from our national data collection effort..

Citation
Format

Mosyjowski, E. A., & Daly, S. R., & Peters, D. L., & Skerlos, S. J. (2013, June), Designing a Survey Instrument for a National Study of Direct-pathway and Returning Engineering Graduate Students Paper presented at 2013 ASEE Annual Conference & Exposition, Atlanta, Georgia. 10.18260/1-2--19391

TY  - CPAPER
AB  -     Designing a Survey Instrument for a National Study of Traditional and                  Returning Engineering Graduate StudentsThough a majority of engineering PhD students begin their doctoral career shortly aftercompleting an undergraduate degree (and perhaps a Master’s), a significant minority of studentsare “returners,” students who pursue a PhD after working outside of academia for five or moreyears. In the first phase of a three year NSF-funded study which aims to characterize thepopulation of returning engineering PhD students, explore the interactions of their previous workexperiences and their academic work, and investigate stakeholder views and institutional policiesrelated to returning PhD students, we developed a nationally-distributed survey to compareexperiences and perspectives of returners and traditional students.The survey development was grounded in Eccles’ Expectancy Value Theory (EVT), a theoreticalframework that explains how and why people make choices, based on the expected results ofthose choices, the costs required to make the choice, and their own interests and values. It wasalso informed by literature on returning students and by a pilot study that involved interviewswith ten returning students about their motivations for returning and their experiences duringgraduate school. The survey included questions about students’ motivation for returning, theirprevious work and school experience, their future career plans, the challenges of graduate school,and their strategies for adapting to these challenges.The development of our questionnaire included numerous levels of review and iteration. Ourresearch team created, discussed, and revised a list of questions based on our framework, pastresearch, and the personal experiences of current and former graduate students. After revisingseveral versions of the instrument, evaluating it for clarity, comprehensiveness, and relevance toour objectives, we asked our advisory board to review the survey. After again revising thesurvey based on our advisors’ feedback, we then piloted the survey with 7 current PhD studentsin STEM fields. We utilized a think-aloud cognitive interviewing technique to ensure thequestionnaire captured these students’ experiences as well as to check for the four components ofmeasurement error identified by Collins (2003), which include: comprehension problems,validity problems, processing difficulties, and pronunciation and communication difficulties.Participant feedback was carefully recorded and compiled in order to identify commondifficulties or suggestions. We then created the final draft of the survey based on this feedback.The focus of this paper and presentation will be on the development of the survey, in which wewill highlight best practices from the literature that informed our creation and refinement process.We will show iterations of the survey and data from the advisory board and our cognitiveinterviews that informed the final version of the instrument. We will also discuss the distributionof the instrument and some preliminary results from our national data collection effort..
AU  - Erika A. Mosyjowski
AU  - Shanna R. Daly
AU  - Diane L Peters
AU  - Steven J. Skerlos
CY  - Atlanta, Georgia
DA  - 2013/06/23
PB  - ASEE Conferences
TI  - Designing a Survey Instrument for a National Study of Direct-pathway and Returning Engineering Graduate Students
UR  - https://peer.asee.org/19391
DO  - 10.18260/1-2--19391
ER  -