Developing an Instrument to Measure the Impact of Service on Technical and Professional Learning Outcomes

Adam R. Carberry; Christopher W. Swan

Download Paper | Permalink

Conference: 2011 ASEE Annual Conference & Exposition
Location: Vancouver, BC
Publication Date: June 26, 2011
Start Date: June 26, 2011
End Date: June 29, 2011
ISSN: 2153-5965
Conference Session: Learning Outside the Classroom
Tagged Division: Educational Research and Methods
Page Count: 8
Page Numbers: 22.454.1 - 22.454.8
DOI: 10.18260/1-2--17735
Permanent URL: https://peer.asee.org/17735
Download Count: 361

Request a correction

Paper Authors

biography

Adam R. Carberry Arizona State University orcid.org/0000-0003-0041-7060

visit author page

Adam R. Carberry received his Ph.D. in Engineering Education from the Tufts University Math, Science, Technology, and Engineering Education program in 2010. He holds an M.S. in Chemistry from Tufts University and a B.S. in Material Science Engineering from Alfred University. He is currently working as a Postdoctoral Research Associate in the College of Technology and Innovation, Department of Engineering at Arizona State University investigating engineering student conceptions of modeling.

visit author page

biography

Christopher W. Swan Tufts University orcid.org/0000-0001-5670-8938

visit author page

Chris Swan is an associate professor of Civil and Environmental Engineering with additional affiliations with the Jonathan M. Tisch College of Citizenship and Public Service and the Center for Engineering Education and Outreach at Tufts University. Dr. Swan has also served as chair of Tufts CEE department (2002 - 2007) and as an officer in the Environmental Engineering division of ASEE (2001 - 2005). Dr. Swan’s current interests lie in the areas of waste reuse, and service-based educational efforts in the engineering curriculum. Synergies of these efforts progressed to research on engineering education and training utilizing project-based learning and service-based pedagogies, specifically their potential impacts on student learning and how these impacts may be evaluated and assessed.

visit author page

Download Paper | Permalink

Abstract

Assessing the Impact of Service on Technical and Professional Learning OutcomesAbstractSuccessful educational interventions positively impact learning outcomes. In engineering, asource of content to measure the impact of various educational interventions on learningengineering are the ABET criteria. These criteria list professional and technical learningoutcomes that accredited engineering programs aim to teach their graduates.Pierrakos, Borrego, and Lo (2007, 2008) used the ABET criteria and additional learningoutcomes identified by the National Academy of Engineering’s Center for the Advancement ofScholarship on Engineering Education (NAE CASEE) (Bjorklund & Fortenberry, 2005) toconstruct a 50-item instrument capable of measuring student technical and professional learningoutcomes. Items on the National Engineering Students’ Learning Outcomes Survey (NESLOS)were written to assess knowledge and skills pertaining to, but not limited to: (1) problem-solving,(2) writing and communication skills, (3) understanding and applying knowledge, (4) teamwork,(5) confidence gains, (6) organization and management skills, and (7) interest and engagement ofproject(s).A modified-NESLOS was developed for the purpose of analyzing the impact of engineeringservice opportunities. Modifications were made to reduce the length of the survey, to validate theinstrument, and to provide students with an opportunity to compare the impact of their serviceexperience with the impact of their other engineering coursework.The modified-NESLOS consists of a 15-item instrument developed and used to investigatestudents’ perceptions of the impact of their service experience compared to traditionalcoursework on learning technical (6 items) and professional skills (9 items). Service was chosenas a focus of interest because the field has seen an influx of opportunities over the last decade tomeet student interest and demand. The increase of engineering service-learning courses andextra-curricular service opportunities suggests that faculty and students perceive theseopportunities as valuable learning opportunities.Factor analysis was conducted to ensure validity of the instrument. The results from 261undergraduate engineering students surveyed across the country suggest that a service experiencerelative to traditional coursework has a substantial impact on their learning. Students identifiedthat 34% of what they learned about technical skills and 45% of what they learned aboutprofessional skills was learned through their service activity. While traditional courseworklearning was identified to have a larger impact for both domains, the relative time spent workingin a service environment compared to traditional coursework suggests service to be an importantexperience in this cohort of students’ engineering education. The impact was identified to beeven more important for females who identified significantly higher perceptions of service as asource of learning compared to their male classmates. Scores were consistent across academicyears.ReferencesBjorklund, S., & Fortenberry, N. L. (2005). Final report: Measuring student and faculty engagement in engineering education. Center for the Advancement of Scholarship on Engineering Education (CASEE): National Academy of Engineering.Pierrakos, O., Borrego, M., & Lo, J. (2007). Assessing learning outcomes of senior mechanical engineers in a capstone design experience. Paper presented at the American Society for Engineering Education Annual Conference & Exposition, Honolulu, HI.Pierrakos, O., Borrego, M., & Lo, J. (2008). Preliminary findings from a quantitative study: What are students learning during cooperative education experiences? Paper presented at the American Society for Engineering Education Annual Conference & Exposition.

Citation
Format

Carberry, A. R., & Swan, C. W. (2011, June), Developing an Instrument to Measure the Impact of Service on Technical and Professional Learning Outcomes Paper presented at 2011 ASEE Annual Conference & Exposition, Vancouver, BC. 10.18260/1-2--17735

TY - CPAPER
AB - Assessing the Impact of Service on Technical and Professional Learning OutcomesAbstractSuccessful educational interventions positively impact learning outcomes. In engineering, asource of content to measure the impact of various educational interventions on learningengineering are the ABET criteria. These criteria list professional and technical learningoutcomes that accredited engineering programs aim to teach their graduates.Pierrakos, Borrego, and Lo (2007, 2008) used the ABET criteria and additional learningoutcomes identified by the National Academy of Engineering’s Center for the Advancement ofScholarship on Engineering Education (NAE CASEE) (Bjorklund &amp; Fortenberry, 2005) toconstruct a 50-item instrument capable of measuring student technical and professional learningoutcomes. Items on the National Engineering Students’ Learning Outcomes Survey (NESLOS)were written to assess knowledge and skills pertaining to, but not limited to: (1) problem-solving,(2) writing and communication skills, (3) understanding and applying knowledge, (4) teamwork,(5) confidence gains, (6) organization and management skills, and (7) interest and engagement ofproject(s).A modified-NESLOS was developed for the purpose of analyzing the impact of engineeringservice opportunities. Modifications were made to reduce the length of the survey, to validate theinstrument, and to provide students with an opportunity to compare the impact of their serviceexperience with the impact of their other engineering coursework.The modified-NESLOS consists of a 15-item instrument developed and used to investigatestudents’ perceptions of the impact of their service experience compared to traditionalcoursework on learning technical (6 items) and professional skills (9 items). Service was chosenas a focus of interest because the field has seen an influx of opportunities over the last decade tomeet student interest and demand. The increase of engineering service-learning courses andextra-curricular service opportunities suggests that faculty and students perceive theseopportunities as valuable learning opportunities.Factor analysis was conducted to ensure validity of the instrument. The results from 261undergraduate engineering students surveyed across the country suggest that a service experiencerelative to traditional coursework has a substantial impact on their learning. Students identifiedthat 34% of what they learned about technical skills and 45% of what they learned aboutprofessional skills was learned through their service activity. While traditional courseworklearning was identified to have a larger impact for both domains, the relative time spent workingin a service environment compared to traditional coursework suggests service to be an importantexperience in this cohort of students’ engineering education. The impact was identified to beeven more important for females who identified significantly higher perceptions of service as asource of learning compared to their male classmates. Scores were consistent across academicyears.ReferencesBjorklund, S., &amp; Fortenberry, N. L. (2005). Final report: Measuring student and faculty engagement in engineering education. Center for the Advancement of Scholarship on Engineering Education (CASEE): National Academy of Engineering.Pierrakos, O., Borrego, M., &amp; Lo, J. (2007). Assessing learning outcomes of senior mechanical engineers in a capstone design experience. Paper presented at the American Society for Engineering Education Annual Conference &amp; Exposition, Honolulu, HI.Pierrakos, O., Borrego, M., &amp; Lo, J. (2008). Preliminary findings from a quantitative study: What are students learning during cooperative education experiences? Paper presented at the American Society for Engineering Education Annual Conference &amp; Exposition.
AU - Adam R. Carberry
AU - Christopher W. Swan
CY - Vancouver, BC
DA - 2011/06/26
PB - ASEE Conferences
TI - Developing an Instrument to Measure the Impact of Service on Technical and Professional Learning Outcomes
UR - https://peer.asee.org/17735
DO - 10.18260/1-2--17735
ER -