Asee peer logo

The Role of Outlier Analysis in Reducing Purposeful Sampling Bias: A Sequential Mixed-Method Approach

Download Paper |

Conference

2015 ASEE Annual Conference & Exposition

Location

Seattle, Washington

Publication Date

June 14, 2015

Start Date

June 14, 2015

End Date

June 17, 2015

ISBN

978-0-692-50180-1

ISSN

2153-5965

Conference Session

Survey and Assessment Development

Tagged Division

Educational Research and Methods

Page Count

14

Page Numbers

26.1570.1 - 26.1570.14

DOI

10.18260/p.24908

Permanent URL

https://peer.asee.org/24908

Download Count

152

Request a correction

Paper Authors

biography

Mariana Tafur-Arciniegas Purdue University, West Lafayette

visit author page

Mariana Tafur is a Ph.D. candidate and a graduate assistant in the School of Engineering Education at Purdue University. She has a M.S., in Education at Los Andes University, Bogota, Colombia; and a B.S., in Electrical Engineering at Los Andes University, Bogota, Colombia. She is a 2010 Fulbright Fellow.
Her research interests include engineering skills development, STEM for non-engineers adults, motivation in STEM to close the technology literacy gap, STEM formative assessment, and Mixed-Methods design.

visit author page

biography

Senay Purzer Purdue University, West Lafayette Orcid 16x16 orcid.org/0000-0003-0784-6079

visit author page

Ṣenay Purzer is an Assistant Professor in the School of Engineering Education. She is the recipient of a 2012 NSF CAREER award, which examines how engineering students approach innovation. She serves on the editorial boards of Science Education and the Journal of Pre-College Engineering Education (JPEER). She received a B.S.E with distinction in Engineering in 2009 and a B.S. degree in Physics Education in 1999. Her M.A. and Ph.D. degrees are in Science Education from Arizona State University earned in 2002 and 2008, respectively.

visit author page

Download Paper |

Abstract

The Role of Outlier Analysis in Reducing Purposeful Sampling Bias: A Sequential Mixed-Method ApproachSampling is an important step in researching. Depending on the research question and qualitativeor quantitative nature of the study the eligibility and size of the population may vary widely. Dueto small size of samples for qualitative analysis, bias may have a larger effect in this type ofresearch where convenient samples are commonly used. The aim of purposefully selectedsamples is to find information-rich cases allowing in-depth analysis instead of generalizablefindings. Using statistical analysis for identifying information-rich cases may reduce bias whileallowing qualitative analysis for in-depth research questions.The purpose of this paper is to describe an outlier analysis followed by a cluster analysis toinform purposeful sampling as part of sequential mixed-methods studies. Three hypotheses aretested: 1) Purposeful sampling can be performed using statistical methods that weight criteriaequally for all prospective participants. 2) Outliers represent critical cases of groups within adesired population for maximum variation or contrast sampling techniques 3) Due to outliernature, sample size affects the quality of critical cases identification.The sample included adults in academia and industry who competed a lifelong learning scale andbackground survey. Using cluster analysis, outliers in four groups were identified based on theinteraction between participants’ lifelong learning and STEM background. Two cloudrepresentations were used for increasing confidence in outlier identification, one using raw scalesfrom surveys and other using ranked data from highest to lowest scores. The first method tookbetween-scales variation into account by calculating linkage to the cluster using distance to anelliptical cloud, while the second took that variation into account by ranking values within eachscale. The purposeful sample comprised all data points identified as outliers using bothstrategies. Central tendencies were analyzed to assure that outliers were representing significantdifferences between groups. This analysis resulted in the identification of outliers withconfidence and show statistically that the outliers were part of a sub-cluster, representing aspecific group in the population.The study provides a valid and rigorous approach to purposeful sampling, enabling to select aconvenient yet unbiased sample. The statistically rigorous selection of participants based incluster analysis led to a wide variety of cases. This range and representation of sub-groups withina larger population may provide a useful selection of participants for qualitative analyses.

Tafur-Arciniegas, M., & Purzer, S. (2015, June), The Role of Outlier Analysis in Reducing Purposeful Sampling Bias: A Sequential Mixed-Method Approach Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24908

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015