Asee peer logo

Accuracy in Student Placement Data

Download Paper |

Collection

2014 ASEE Annual Conference & Exposition

Location

Indianapolis, Indiana

Publication Date

June 15, 2014

Start Date

June 15, 2014

End Date

June 18, 2014

ISSN

2153-5965

Conference Session

Student Learning, Problem Solving, and Critical Thinking 1

Tagged Division

Educational Research and Methods

Page Count

13

Page Numbers

24.131.1 - 24.131.13

Permanent URL

https://peer.asee.org/20022

Download Count

29

Request a correction

Paper Authors

biography

Cynthia B. Paschal Vanderbilt University

visit author page

Dr. Paschal is associate dean of the Vanderbilt University School of Engineering and is a faculty member in the department of biomedical engineering. She has research experience in medical imaging and engineering education. Paschal earned bachelor's and master's degrees in nuclear engineering from the Massachusetts Institute of Technology and the doctor of philosophy degree in biomedical engineering from Case Western Reserve University.

visit author page

Download Paper |

Abstract

Accuracy in Student Placement DataPlacement of engineering students at the end of their undergraduate studies is one measure of thesuccess of the educational program. For the measure to be helpful in providing feedback abouteducational programs, the reported data must present an accurate picture of placement. Twosignificant issues affect the accuracy these data: bias in the setting of low placement surveyresponse rates and restrictions on eligibility faced by international students.In a limited survey of ten highly ranked U.S. engineering programs where a Ph.D. is the highestdegree offered, undergraduate placement survey response rates ranged from 44 to 97%, with anaverage of 74%. A modest response rate may not be a serious problem if the responses comefrom a truly random sample of the population. However, when non-response bias exists, a lowresponse rate decreases the likely accuracy of the composite data. Even in the absence ofsampling bias, the probability that the data may not reflect an accurate picture of placement goesup with decreasing response rate.Most engineering programs aim to obtain survey data from an entire graduation cohort ratherthan sampling a subset of the cohort. However, as revealed above, response rates varysignificantly and are often much lower than desired. Failure to complete a survey may resultfrom students not receiving the survey at an opportune time or in a convenient medium, such as along survey administered online during final examinations. More critically to the issue of non-response bias, failure to complete a survey may result from students without placement notwanting to report what they may perceive as failure. If this is the case, then the non-responsebias can be severe, causing a reported placement rate to greatly overestimate the true placementrate. To prevent these problems, several techniques have been implemented at this institution toensure very high response rates (>97%). These techniques have been found to be effective: (1)providing faculty in each major with a list of degree candidates from whom surveys arerequested, (2) administering the survey in the month prior to commencement, while the studentsare still on campus and in contact with their faculty members, (3) limiting the survey to onesingle page, (4) administering the survey on paper in a required senior course or at a specialsocial event for seniors, (5) enlisting staff support to obtain surveys from stragglers, (6)acknowledging degree programs with 100% response rates, and (7) publishing anonymizedsummary data each year in a format useful to students and faculty members.Another issue affecting the accuracy placement results is restrictions on eligibility to work facedby international students. As international students on F1 and J1 visas do not have anunrestricted right to work in the U.S.A., many employers do not wish to consider hiring suchcandidates. Thus, it is helpful to conduct separate placement analyses for domestic andinternational students.If accurate data result from a survey with a high response rate and separate analysis for studentswith and without an unrestricted right to work in the U.S., then the results are more useful andlegitimate comparisons between educational programs can be made. In addition, accurate datacan reveal opportunities for increased synergies with key industrial partners andgraduate/professional programs and for improving educational programs.

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2014 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015