Salt Lake City, Utah
June 23, 2018
June 23, 2018
July 27, 2018
Educational Research and Methods
It is common, in educational research studies, to select purposeful population samples. Independent of the selected method, there is an associated bias to the sampling process because of the subjectivity of the person or group of people who accomplish the task. This paper shows the pilot of a protocol development for observing how people approach purposeful sampling using a real research problem. Engineers and non-engineers were observed completing the task and were monitored for collecting the differences between both points of view, and for identifying possible bias in the purposeful sample. Five stages were conducted for piloting the protocol for gathering information in how purposeful sampling is traditionally performed. Two participants with knowledge in educational research performed the activity for the first iteration of the pilot. In addition, four participants (two engineer students and two non-engineer students) performed a second pilot using an improved protocol. Finally, a panel of experts was asked to review the process for a final protocol. For the activity designed, participants were asked to review 118 profiles of people with diverse academic and social backgrounds. The goal was to choose 3 profiles for each of four categories, aiming to identify those participants who were the best representatives of each of the categories: 1. Engineers with a low level of Lifelong Learning (ELL), 2. Non-Engineer with a low level of Lifelong Learning (nELL); 3. Engineers with a high level of Lifelong Learning (EHL), and 4. Non-Engineer with a high level of Lifelong Learning (nEHL). The time for the assignment was limited to 90 minutes, and a think-aloud protocol was followed for data collection. The iterative process design for the protocol allowed to improve the resources, time management, and activity logistics for the Purposeful Sampling Activity. The time for the whole activity was calibrated and defined as 90 minutes. The time alert was changed from 10 to 15 minutes, and 10 minutes were defined to be given to those individuals who asked for more time. After participants’ feedback, new resources such as pens, sticky notes, eraser, water, and digital database with the information were included. The activity allowed to identify bias in sample selection due to lack of usage of the complete data set. Likewise, each expert defined different criteria for selection, setting diverse start points. This bias was induced by variables such as age, undergraduate background, expertise in a specific field of study and degree of development of specific skills through the professional life. In conclusion, this study showed the design of a protocol for collecting information about how non-intended bias was present in a purposeful sampling. This analysis may guide the process of purposeful selection of samples for qualitative research and provides a tool for measuring the bias reduction between traditional and statistical purposeful selection of information-rich cases.
Tafur-Arciniegas, M., & Lara Contreras, A. F. (2018, June), First Approach to Purposeful Sampling for Determining Key Factors on Outcome Bias Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. 10.18260/1-2--30518
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2018 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015