Can a Five-Minute, Three-Question Survey Foretell First-Year Engineering Student Performance and Retention?

Stephanie M. Gratiano; William John Palm

Download Paper | Permalink

Conference: 2016 ASEE Annual Conference & Exposition
Location: New Orleans, Louisiana
Publication Date: June 26, 2016
Start Date: June 26, 2016
End Date: June 29, 2016
ISBN: 978-0-692-68565-5
ISSN: 2153-5965
Conference Session: First-Year Programs Division Technical Session 3; The Best of All the FPD Papers
Tagged Division: First-Year Programs
Page Count: 25
DOI: 10.18260/p.26427
Permanent URL: https://peer.asee.org/26427
Download Count: 484

Request a correction

Paper Authors

biography

Stephanie M. Gratiano Roger Williams University

visit author page

Stephanie Gratiano is a Junior Undergraduate Mechanical Engineering student at Roger Williams University. She is an officer of both the American Society of Mechanical Engineers (ASME) and Engineers Without Borders (EWB) Chapters at the university. Her future career interests include Product Design or Mechanical Design, and she wishes to further her education in Industrial Design after graduating.

visit author page

biography

William John Palm IV P.E. Roger Williams University

visit author page

William Palm is Assistant Professor of Engineering at Roger Williams University, where he teaches Engineering Graphics and Design, Computer Applications for Engineering, Machine Design, Manufacturing and Assembly, Biomechanics, and Capstone Design. He previously worked as a product design engineer and consultant and taught at the U.S. Coast Guard Academy and Boston University. He holds a PhD in Mechanical Engineering from MIT and is licensed as a Professional Engineer in the Commonwealth of Massachusetts.

visit author page

Download Paper | Permalink

Abstract

This research paper examines first-year student performance and retention within engineering. A considerable body of literature has reported factors influencing performance and retention, including high school GPA and SAT scores,[1,2,3] gender,[4] self-efficacy,[1,5] social status,[2,6,7] hobbies in leisure time,[4] and social integration.[6,7] Although these factors can help explain and even partially predict student outcomes, they can be difficult to measure; typical survey instruments are lengthy and can be invasive of student privacy. To address this limitation, the present paper examines whether a much simpler survey can be used to anticipate student outcomes. The survey is grounded in theories of mindset[7] and cognitive dissonance,[8] and investigates students’ motivations for pursuing an Engineering degree, as well as their initial understanding of engineering prior to the commencement of the program.

The survey was administered to 360 first-semester students in an introductory Engineering Graphics and Design course. At the beginning of the first day of class, students were given a three-question, open-ended questionnaire that asked “In your own words, what do engineers do?”, “Why did you choose engineering?”, and “Was there any particular person or experience that influenced your decision?” Two investigators independently performed open coding on the responses, identifying dozens of codes for both motivations for pursuing engineering and understanding of what it is. The codes were grouped into intrinsic and extrinsic motives (or neither) for choosing engineering. These motives reveal student “growth” and “fixed” mindsets, as suggested by Dweck’s mindset theory[7] and others.[8,9] We hypothesized that students citing motivations reflective of a “growth” mindset would be more likely to perform well and be retained at a higher rate than those citing motives reflective of a “fixed” mindset. The coded data were then analyzed to determine if there were any relationships between particular codes and first-semester GPA or first-year retention in engineering.

Frequently mentioned codes included inspiration from the student’s father (47 students), enjoyment of math (45 students), enjoyment of building things (43 students), passion for problem solving (41 students), and interest in learning how things work (41 students). Infrequently mentioned were inspiration from working with cars (2 students), building robots (4 students), previous occupational experience (9 students), and influence from high school engineering courses (11 students).

Codes that were positively and significantly associated with first-semester GPA included: understanding that engineers study science (p=0.025), enjoyment of math and science (p=0.048), and physics self-efficacy (p=0.059), whereas suggesting that engineers simplify and make life easier (p=0.052) were negatively and significantly related to first-semester GPA.

Codes positively and significantly associated with retention included: long term desire to be an engineer (p=0.034), enjoyment of creating/building things (p=0.046), enjoyment of math and science (p=0.048), physics self-efficacy (p=0.059), and influence from an uncle (p=0.061) or grandfather (p=0.078) who was an engineer. Codes negatively and significantly associated with retention included: influence from a friend who was an engineer (p=0.033), hearing stories about engineering (p=0.033), and parents or family pushing the student to become an engineer (p=0.035).

Although many prior studies have suggested that student self-efficacy is related to retention, [1,5] this study found that student interest was more strongly associated with retention. This finding is supported by Dweck’s mindset theory: students with a “growth” mindset (e.g., “I enjoy math”) would be expected to perform better and thus be retained at a higher rate than those with a “fixed” mindset (e.g., “I am good at math”).[7] We were surprised that few students mentioned activities expressly designed to stimulate interest in engineering, such as robotics competitions and high school engineering classes. Rather, they cited general interests in math, problem solving, and creativity, as well as family influences, all factors that are challenging for the engineering education community to address.

This study examined whether a simple, five-minute, three-question survey can foretell student performance and retention. We conclude that relative to its ease of administration, this survey does provide valuable insight. It confirms some findings from past research, while generating new ones. Its open-ended nature helps in identifying new trends that could be missed by a static closed-form questionnaire. Its minimalism enables easy implementation in an introductory engineering course, where it serves not only as a research tool, but also as a pedagogical aid to help students and teacher discover student misperceptions about engineering, and to customize the curriculum to student interests.

Bibliography:

[1] Honken, N., and Ralston, P.A.S. (2013). Freshman engineering retention: A holistic look. Journal of STEM Education: Innovations and Research, 14(2).

[2] Lotkowski, V.A., Robbins, S.B., and Noeth, R.J. (2004). The role of academic and non-academic factors in improving college retention. American College Testing, Inc. ACT Policy Report.

[3] Augustine, R.D. (1966). Persistence and attrition of engineering students: A study of freshman and sophomore engineering students at three midwestern universities. Michigan State University.

[4] Cech, E., Rubineau, B., Silbey, S., and Seron, C. (2011). Professional role confidence and gendered persistence in engineering. American Sociological Review, 76(5).

[5] Fantz, T.D., Siller, T.J., and Demiranda, M.A.. (2011). Pre-collegiate factors in influencing the self-efficacy of engineering students. Journal of Engineering Education, 100(3).

[6] Gore, J.N. (2010) The importance of freshman experiences in predicting student retention decisions. M.A. thesis, Appalachian State University.

[7] Dweck, C.S. (2008). “Mindsets and math/science achievement,” report prepared for The Carnegie Corporation of New York-Institute for Advanced Study Commission on Mathematics and Science Education, New York, NY. [8] Atkinson, R.L., Atkinson, R.C., Smith, E.E., Bem, D.J., Nolen-Hoeksema, S. (1999). Hilgard’s Introduction to Psychology.

[9] Stump, S., Husman, J., Chung, W.T., and Done, A. (2009). Student beliefs about intelligence: Relationship to learning. Proceedings of the 39th ASEE/IEEE Frontiers in Education Conference, San Antonio, TX.

Citation
Format

Gratiano, S. M., & Palm, W. J. (2016, June), Can a Five-Minute, Three-Question Survey Foretell First-Year Engineering Student Performance and Retention? Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. 10.18260/p.26427

TY  - CPAPER
AB  - This research paper examines first-year student performance and retention within engineering.  A considerable body of literature has reported factors influencing performance and retention, including high school GPA and SAT scores,[1,2,3] gender,[4] self-efficacy,[1,5] social status,[2,6,7] hobbies in leisure time,[4] and social integration.[6,7]  Although these factors can help explain and even partially predict student outcomes, they can be difficult to measure; typical survey instruments are lengthy and can be invasive of student privacy.  To address this limitation, the present paper examines whether a much simpler survey can be used to anticipate student outcomes.  The survey is grounded in theories of mindset[7] and cognitive dissonance,[8] and investigates students’ motivations for pursuing an Engineering degree, as well as their initial understanding of engineering prior to the commencement of the program.

The survey was administered to 360 first-semester students in an introductory Engineering Graphics and Design course.  At the beginning of the first day of class, students were given a three-question, open-ended questionnaire that asked “In your own words, what do engineers do?”, “Why did you choose engineering?”, and “Was there any particular person or experience that influenced your decision?”  Two investigators independently performed open coding on the responses, identifying dozens of codes for both motivations for pursuing engineering and understanding of what it is.  The codes were grouped into intrinsic and extrinsic motives (or neither) for choosing engineering.  These motives reveal student “growth” and “fixed” mindsets, as suggested by Dweck’s mindset theory[7] and others.[8,9]  We hypothesized that students citing motivations reflective of a “growth” mindset would be more likely to perform well and be retained at a higher rate than those citing motives reflective of a “fixed” mindset.  The coded data were then analyzed to determine if there were any relationships between particular codes and first-semester GPA or first-year retention in engineering.

Frequently mentioned codes included inspiration from the student’s father (47 students), enjoyment of math (45 students), enjoyment of building things (43 students), passion for problem solving (41 students), and interest in learning how things work (41 students).  Infrequently mentioned were inspiration from working with cars (2 students), building robots (4 students), previous occupational experience (9 students), and influence from high school engineering courses (11 students). 

Codes that were positively and significantly associated with first-semester GPA included: understanding that engineers study science (p=0.025), enjoyment of math and science (p=0.048), and physics self-efficacy (p=0.059), whereas suggesting that engineers simplify and make life easier (p=0.052) were negatively and significantly related to first-semester GPA.

Codes positively and significantly associated with retention included: long term desire to be an engineer (p=0.034), enjoyment of creating/building things (p=0.046), enjoyment of math and science (p=0.048), physics self-efficacy (p=0.059), and influence from an uncle (p=0.061) or grandfather (p=0.078) who was an engineer.  Codes negatively and significantly associated with retention included: influence from a friend who was an engineer (p=0.033), hearing stories about engineering (p=0.033), and parents or family pushing the student to become an engineer (p=0.035).

Although many prior studies have suggested that student self-efficacy is related to retention, [1,5] this study found that student interest was more strongly associated with retention. This finding is supported by Dweck’s mindset theory: students with a “growth” mindset (e.g., “I enjoy math”) would be expected to perform better and thus be retained at a higher rate than those with a “fixed” mindset (e.g., “I am good at math”).[7]  We were surprised that few students mentioned activities expressly designed to stimulate interest in engineering, such as robotics competitions and high school engineering classes.  Rather, they cited general interests in math, problem solving, and creativity, as well as family influences, all factors that are challenging for the engineering education community to address.

This study examined whether a simple, five-minute, three-question survey can foretell student performance and retention.  We conclude that relative to its ease of administration, this survey does provide valuable insight.  It confirms some findings from past research, while generating new ones.  Its open-ended nature helps in identifying new trends that could be missed by a static closed-form questionnaire.  Its minimalism enables easy implementation in an introductory engineering course, where it serves not only as a research tool, but also as a pedagogical aid to help students and teacher discover student misperceptions about engineering, and to customize the curriculum to student interests.




Bibliography:

[1] 	Honken, N., and Ralston, P.A.S. (2013). Freshman engineering retention: A holistic look. Journal of STEM Education: Innovations and Research, 14(2).

[2]	Lotkowski, V.A., Robbins, S.B., and Noeth, R.J. (2004). The role of academic and non-academic factors in improving college retention. American College Testing, Inc. ACT Policy Report. 

[3]	Augustine, R.D. (1966). Persistence and attrition of engineering students: A study of freshman and sophomore engineering students at three midwestern universities. Michigan State University.

[4]	Cech, E., Rubineau, B., Silbey, S., and Seron, C. (2011). Professional role confidence and gendered persistence in engineering. American Sociological Review, 76(5).

[5]	Fantz, T.D., Siller, T.J., and Demiranda, M.A.. (2011). Pre-collegiate factors in influencing the self-efficacy of engineering students. Journal of Engineering Education, 100(3).

[6]	Gore, J.N. (2010) The importance of freshman experiences in predicting student retention decisions. M.A. thesis, Appalachian State University.

[7]	Dweck, C.S. (2008). “Mindsets and math/science achievement,” report prepared for The Carnegie Corporation of New York-Institute for Advanced Study Commission on Mathematics and Science Education, New York, NY.
	
[8]	Atkinson, R.L., Atkinson, R.C., Smith, E.E., Bem, D.J., Nolen-Hoeksema, S. (1999). Hilgard’s Introduction to Psychology.

[9]	Stump, S., Husman, J., Chung, W.T., and Done, A. (2009). Student beliefs about intelligence: Relationship to learning. Proceedings of the 39th ASEE/IEEE Frontiers in Education Conference, San Antonio, TX.

AU  - Stephanie M. Gratiano
AU  - William John Palm IV P.E.
CY  - New Orleans, Louisiana
DA  - 2016/06/26
PB  - ASEE Conferences
TI  - Can a Five-Minute, Three-Question Survey Foretell First-Year Engineering Student Performance and Retention?
UR  - https://peer.asee.org/26427
DO  - 10.18260/p.26427
ER  -