Asee peer logo

Developing an Instrument to Measure the Impact of Service on Technical and Professional Learning Outcomes

Download Paper |


2011 ASEE Annual Conference & Exposition


Vancouver, BC

Publication Date

June 26, 2011

Start Date

June 26, 2011

End Date

June 29, 2011



Conference Session

Learning Outside the Classroom

Tagged Division

Educational Research and Methods

Page Count


Page Numbers

22.454.1 - 22.454.8



Permanent URL

Download Count


Request a correction

Paper Authors


Adam R. Carberry Arizona State University Orcid 16x16

visit author page

Adam R. Carberry received his Ph.D. in Engineering Education from the Tufts University Math, Science, Technology, and Engineering Education program in 2010. He holds an M.S. in Chemistry from Tufts University and a B.S. in Material Science Engineering from Alfred University. He is currently working as a Postdoctoral Research Associate in the College of Technology and Innovation, Department of Engineering at Arizona State University investigating engineering student conceptions of modeling.

visit author page


Christopher W. Swan Tufts University Orcid 16x16

visit author page

Chris Swan is an associate professor of Civil and Environmental Engineering with additional affiliations with the Jonathan M. Tisch College of Citizenship and Public Service and the Center for Engineering Education and Outreach at Tufts University. Dr. Swan has also served as chair of Tufts CEE department (2002 - 2007) and as an officer in the Environmental Engineering division of ASEE (2001 - 2005). Dr. Swan’s current interests lie in the areas of waste reuse, and service-based educational efforts in the engineering curriculum. Synergies of these efforts progressed to research on engineering education and training utilizing project-based learning and service-based pedagogies, specifically their potential impacts on student learning and how these impacts may be evaluated and assessed.

visit author page

Download Paper |


Assessing the Impact of Service on Technical and Professional Learning OutcomesAbstractSuccessful educational interventions positively impact learning outcomes. In engineering, asource of content to measure the impact of various educational interventions on learningengineering are the ABET criteria. These criteria list professional and technical learningoutcomes that accredited engineering programs aim to teach their graduates.Pierrakos, Borrego, and Lo (2007, 2008) used the ABET criteria and additional learningoutcomes identified by the National Academy of Engineering’s Center for the Advancement ofScholarship on Engineering Education (NAE CASEE) (Bjorklund & Fortenberry, 2005) toconstruct a 50-item instrument capable of measuring student technical and professional learningoutcomes. Items on the National Engineering Students’ Learning Outcomes Survey (NESLOS)were written to assess knowledge and skills pertaining to, but not limited to: (1) problem-solving,(2) writing and communication skills, (3) understanding and applying knowledge, (4) teamwork,(5) confidence gains, (6) organization and management skills, and (7) interest and engagement ofproject(s).A modified-NESLOS was developed for the purpose of analyzing the impact of engineeringservice opportunities. Modifications were made to reduce the length of the survey, to validate theinstrument, and to provide students with an opportunity to compare the impact of their serviceexperience with the impact of their other engineering coursework.The modified-NESLOS consists of a 15-item instrument developed and used to investigatestudents’ perceptions of the impact of their service experience compared to traditionalcoursework on learning technical (6 items) and professional skills (9 items). Service was chosenas a focus of interest because the field has seen an influx of opportunities over the last decade tomeet student interest and demand. The increase of engineering service-learning courses andextra-curricular service opportunities suggests that faculty and students perceive theseopportunities as valuable learning opportunities.Factor analysis was conducted to ensure validity of the instrument. The results from 261undergraduate engineering students surveyed across the country suggest that a service experiencerelative to traditional coursework has a substantial impact on their learning. Students identifiedthat 34% of what they learned about technical skills and 45% of what they learned aboutprofessional skills was learned through their service activity. While traditional courseworklearning was identified to have a larger impact for both domains, the relative time spent workingin a service environment compared to traditional coursework suggests service to be an importantexperience in this cohort of students’ engineering education. The impact was identified to beeven more important for females who identified significantly higher perceptions of service as asource of learning compared to their male classmates. Scores were consistent across academicyears.ReferencesBjorklund, S., & Fortenberry, N. L. (2005). Final report: Measuring student and faculty engagement in engineering education. Center for the Advancement of Scholarship on Engineering Education (CASEE): National Academy of Engineering.Pierrakos, O., Borrego, M., & Lo, J. (2007). Assessing learning outcomes of senior mechanical engineers in a capstone design experience. Paper presented at the American Society for Engineering Education Annual Conference & Exposition, Honolulu, HI.Pierrakos, O., Borrego, M., & Lo, J. (2008). Preliminary findings from a quantitative study: What are students learning during cooperative education experiences? Paper presented at the American Society for Engineering Education Annual Conference & Exposition.

Carberry, A. R., & Swan, C. W. (2011, June), Developing an Instrument to Measure the Impact of Service on Technical and Professional Learning Outcomes Paper presented at 2011 ASEE Annual Conference & Exposition, Vancouver, BC. 10.18260/1-2--17735

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2011 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015