June 14, 2015
June 14, 2015
June 17, 2015
Educational Research and Methods
26.476.1 - 26.476.11
Free Listing: Using an Anthropological Structured Technique to Identify Social Capital and Cultural Model Items for Engineering Undergraduate SurveysThis paper describes free listing (FL), a cognitive anthropological structured technique that canbe used to improve the validity of survey instruments and the design of questions on interviewprotocols in engineering education research. Anthropologists use FL to systematically collectdata about specific cultural models. Cultural models are internalized cognitive schemas thatindividuals within a culture share to varying degrees and draw upon to form and organize theirbeliefs, meanings, and practices. In our National Science Foundation funded study, we used FLto understand the cultural model of “success” in undergraduate engineering programs. Our studyasks, “what are the effects of social capital and cultural models of engineering success on theretention and degree attainment of women and minorities in engineering?” In this paper, we willpresent our approach to using FL to design items for our survey instruments that measure socialcapital and cultural models of success among engineering undergraduates as the first step inanswering our research question.In FL, participants are asked a series of questions that represent the major conceptual areas in acultural model as identified by the researcher from previous studies and the literature. For eachquestion, participants are asked to list as many responses as possible. When the participantpauses, they are prompted for additional responses. FL assumes that individuals 1) withextensive knowledge give more responses than those with less knowledge, 2) list most familiarand meaningful responses first, and 3) give responses that reflect their local cultural knowledge.Frequency and rank of each response are reflected in the salience, Smith’s S, a number calculatedusing an anthropological qualitative software program, in our case, ANTHROPAC 4.98. Usingsalience, researchers decide a cutoff line to determine which responses should be examinedfurther and included in the survey or interview protocol.We also discuss the advantages and limitations of the FL technique. For example, the primaryvalue of FL is that participants, especially if they are from an understudied population, oftenidentify beliefs or attitudes about the cultural model that were previously unknown or interpreteddifferently in the literature. By including these responses in the survey or interview protocol, theresearcher can determine if these beliefs or attitudes are shared throughout the study population.One limitation to FL is that there is no definitive method to identify the appropriate sample size.A larger the sample size will likely lead to a greater variety of responses, lessening the likelihoodof a high salience. Hence, is that is important for the researcher to narrowly specify the culturalmodel or cognitive area that they are exploring so that participants can easily mentally unpacktheir knowledge.The rich data obtained from FL can improve the design and validity of survey instruments andinterview protocols. Specific examples of how we used FL in our research study are described indetail, along with implications and recommendations for adoption of the technique to otherengineering education research.
Smith, C. A. S., & Wao, H., & Martin, J. P., & MacDonald, G. T., & Lee, R. S., & Kersaint, G. (2015, June), Designing a Survey for Engineering Undergraduates Using Free Listing - An Anthropological Structured Technique Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23814
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015