Asee peer logo

Developing the Postsecondary Student Engagement Survey (PosSES) to Measure Undergraduate Engineering Students’ Out-of-Class Involvement

Download Paper |


2015 ASEE Annual Conference & Exposition


Seattle, Washington

Publication Date

June 14, 2015

Start Date

June 14, 2015

End Date

June 17, 2015





Conference Session

Beyond the Classroom

Tagged Division

Educational Research and Methods

Tagged Topic


Page Count


Page Numbers

26.506.1 - 26.506.17



Permanent URL

Download Count


Request a correction

Paper Authors


Denise Rutledge Simmons PE Virginia Tech Orcid 16x16

visit author page

Dr. Denise R. Simmons, PE, is an assistant professor in the Myers-Lawson School of Construction and in Civil & Environmental Engineering Department, and an affiliate faculty of the Department of Engineering Education at Virginia Polytechnic Institute and State University. She holds a B.S., M.S., and Ph.D. in civil engineering and a graduate certificate in engineering education – all from Clemson University. Until 2012, she was the director of the Savannah River Environmental Sciences Field Station. Dr. Simmons has nearly fourteen years of engineering and project management experience working with public utility companies, a project management consulting company, and a software company. She is a registered professional engineer, project management professional and LEED accredited professional. Her research interests are in investigating students’ development of leadership skills and other professional competencies and in students' involvement in curricular, co-curricular and extra-curricular activities. Dr. Simmons is a NSF CAREER award recipient for her research entitled, “Investigating Co-Curricular Participation of Students Underrepresented in Engineering.”

visit author page


Chosang Tendhar Virginia Tech

visit author page

Chosang Tendhar is a doctoral candidate in the Educational Research and Evaluation (EDRE) program at Virginia Tech. He conducts research in STEM education, specifically students’ engineering major and career intentions. In his research, he uses motivational theories, such as the MUSIC Model of Academic Motivation and Domain Identification model, and advanced methodologies, such as Confirmatory Factor Analysis (CFA), Structural Equation Modeling (SEM), and Longitudinal Data Analysis.

visit author page


Rongrong Yu Virginia Tech

visit author page

Rongrong Yu is a PhD student at the Educational Research and Evaluation Program in School of Education at Virginia Tech. She holds a B.S. degree in psychology and a M.Ed. degree in educational psychology. Her research interests include K-12 student mathematics and science achievement, STEM and gender, and co-curricular involvement.

visit author page


Eric A. Vance Virginia Tech

visit author page

Dr. Eric Vance is an assistant research professor of statistics at Virginia Tech. He is the Director of LISA, Virginia Tech’s Laboratory for Interdisciplinary Statistical Analysis, which met with 1324 clients last year to help them use statistics to solve real-world problems in their research. LISA’s primary mission is to train statisticians to become interdisciplinary collaborators, and since its reformation in 2008, it has trained and mentored 173 statistics students to communicate and collaborate with non-statisticians to solve real-world problems. Dr. Vance is an elected member of the International Statistical Institute, the chair-elect of the ASA Section on Statistical Consulting, the former Vice-chair of Statistics Without Borders. He was a triple major in mathematics, economics, and statistics at UC Berkeley, and then five years later entered graduate school, earning his PhD in 2008 in Statistical Science at Duke University. In the years between his undergraduate and graduate studies, Dr. Vance traveled around the world three times, backpacking through 67 countries in Europe; South and Southeast Asia; Australia and New Zealand; North, Central, and South America; and Africa.

visit author page


Catherine T. Amelink Virginia Tech

visit author page

Dr. Amelink is the Director of Graduate Programs and Assessment in the College of Engineering Virginia Tech and affiliate faculty in the Department of Engineering Education and the Department of Educational Leadership and Policy Studies at Virginia Tech.

visit author page

Download Paper |


TitleDevelopment of the Co-curricular Activities Survey of Engineering Students (CASES) to Measure Involvement, Decisions, Outcomes, Commitment, and Identification of Undergraduate Engineering Students AbstractBackgroundUndergraduate engineering students engage in many curricular-related (associated with a class),co-curricular (associated with school, but not with a class) and extra-curricular (not associatedwith school) activities because students deem them to be educationally purposeful. Participationin these activities is found to have positive outcomes, such as better academic performance andpersistence, and also negative outcomes, such as having less time for schoolwork. In addition,there exist barriers to participating in such activities. Currently, no valid and reliable questionnaireexists to measure these positive and negative outcomes comprehensively.PurposeThis paper describes the development of a questionnaire to accurately measure 1)undergraduate engineering students’ decisions to participate in educationally purposeful activitiesand 2) the students’ outcomes from involvement in these activities. A second goal is to develop atheoretical model to better understand students’ decision making process in committing tomajoring in engineering and pursuing engineering careers. We also measure students’ decisionsnot to participate in such activities and explore the barriers that prevent them from participating inthese activities. Data collected using this questionnaire will help determine outcomes from allactivities students identify as educationally purposeful.MethodsTo develop a robust questionnaire, an expert panel composed of a statistician with seven years ofexperience developing surveys, a higher education scholar with significant research inorganizational, curricular, instructional, and co-curricular practices in engineering schools, and adirector of assessment in a college of engineering participated in the questionnaire developmentprocess. A Domain Identification model, a theoretical framework, informed the constructsmeasured and allowed testing of causality hypothesized among them. Factors for thequestionnaire were identified via a meta-synthesis and meta-analysis of the current literature onundergraduate engineering students’ participation in curricular-related, co-curricular and extra-curricular activities and the outcomes from their involvement in these activities. These factorswere augmented with additional factors identified through focus groups with undergraduateengineering students at three institutions using a Q Sort methodology. The focus groups includedunderrepresented populations in engineering to ensure their input into the factors under study.Data gained from the focus group meetings and examination of existing questionnaires thatmeasure factors related to participation decisions and involvement supported creation of newand/or refined items for the final questionnaire. Finally, a think aloud protocol incorporatedparticipants comments and feedback to refine the items to ensure that their perception,interpretation, and responses to the items matched the developers’ intent.ResultsThe resulting questionnaire collects data on 20 activities and measures involvement (20 items),reasons for participation (12 items), positive and (11 items) negative outcomes accruing fromparticipation (10 items), barriers to participation (13), career commitment (6 items), engineeringidentification (4 items), and proactive personality (10 items).ConclusionBy determining activities and involvement that significantly predict learning outcomes,persistence, and career commitment, the questionnaire can help higher educational institutionadministrators, faculty, and staff gain better insight into students’ decision making process. Thisnew insight can inform strategies that help improve retention of engineering students. This papermakes a methodological contribution by developing a quantitative assessment of outcomes fromactivities students identify as educationally purposeful.

Simmons, D. R., & Tendhar, C., & Yu, R., & Vance, E. A., & Amelink, C. T. (2015, June), Developing the Postsecondary Student Engagement Survey (PosSES) to Measure Undergraduate Engineering Students’ Out-of-Class Involvement Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23845

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015