Asee peer logo

First Approach to Purposeful Sampling for Determining Key Factors on Outcome Bias

Download Paper |

Conference

2018 ASEE Annual Conference & Exposition

Location

Salt Lake City, Utah

Publication Date

June 23, 2018

Start Date

June 23, 2018

End Date

July 27, 2018

Conference Session

Assessment and Research Tools

Tagged Division

Educational Research and Methods

Page Count

12

Permanent URL

https://peer.asee.org/30518

Download Count

12

Request a correction

Paper Authors

biography

Mariana Tafur-Arciniegas Universidad de los Andes

visit author page

Mariana Tafur-Arciniegas is an assistant professor in the School of Education at University of Los Andes, Bogota-Colombia. She is a Ph.D. in Engineering Education from Purdue University, a M.S. in Education and a B.S. in Electrical Engineering from University of Los Andes. She is a 2010 Fulbright Fellow.
Her research interests include engineering skills development, STEM for non-engineers adults, motivation in STEM to close the technology literacy gap, STEM formative assessment, and Mixed-Methods design.

visit author page

biography

Andres Felipe Lara Contreras Universidad de los Andes

visit author page

Andres Lara is an undergraduate student in Chemical Engineering at Universidad de Los Andes, Bogotá, Colombia, Class of 2017-20. His research interests include engineering learning and approach to technological challenges, nanostructured materials, and self-sustainable energy sources.

visit author page

Download Paper |

Abstract

It is common, in educational research studies, to select purposeful population samples. Independent of the selected method, there is an associated bias to the sampling process because of the subjectivity of the person or group of people who accomplish the task. This paper shows the pilot of a protocol development for observing how people approach purposeful sampling using a real research problem. Engineers and non-engineers were observed completing the task and were monitored for collecting the differences between both points of view, and for identifying possible bias in the purposeful sample. Five stages were conducted for piloting the protocol for gathering information in how purposeful sampling is traditionally performed. Two participants with knowledge in educational research performed the activity for the first iteration of the pilot. In addition, four participants (two engineer students and two non-engineer students) performed a second pilot using an improved protocol. Finally, a panel of experts was asked to review the process for a final protocol. For the activity designed, participants were asked to review 118 profiles of people with diverse academic and social backgrounds. The goal was to choose 3 profiles for each of four categories, aiming to identify those participants who were the best representatives of each of the categories: 1. Engineers with a low level of Lifelong Learning (ELL), 2. Non-Engineer with a low level of Lifelong Learning (nELL); 3. Engineers with a high level of Lifelong Learning (EHL), and 4. Non-Engineer with a high level of Lifelong Learning (nEHL). The time for the assignment was limited to 90 minutes, and a think-aloud protocol was followed for data collection. The iterative process design for the protocol allowed to improve the resources, time management, and activity logistics for the Purposeful Sampling Activity. The time for the whole activity was calibrated and defined as 90 minutes. The time alert was changed from 10 to 15 minutes, and 10 minutes were defined to be given to those individuals who asked for more time. After participants’ feedback, new resources such as pens, sticky notes, eraser, water, and digital database with the information were included. The activity allowed to identify bias in sample selection due to lack of usage of the complete data set. Likewise, each expert defined different criteria for selection, setting diverse start points. This bias was induced by variables such as age, undergraduate background, expertise in a specific field of study and degree of development of specific skills through the professional life. In conclusion, this study showed the design of a protocol for collecting information about how non-intended bias was present in a purposeful sampling. This analysis may guide the process of purposeful selection of samples for qualitative research and provides a tool for measuring the bias reduction between traditional and statistical purposeful selection of information-rich cases.

Tafur-Arciniegas, M., & Lara Contreras, A. F. (2018, June), First Approach to Purposeful Sampling for Determining Key Factors on Outcome Bias Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. https://peer.asee.org/30518

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2018 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015