June 20, 2010
June 20, 2010
June 23, 2010
Educational Research and Methods
15.22.1 - 15.22.18
A Content Analysis of the Use of Mixed Methods Studies in Engineering Education Abstract
The complex phenomena studied by engineering education researchers frequently require the complementary use of qualitative and quantitative approaches. In light of these needs, many researchers are utilizing mixed methods designs to take advantage of the relative strengths and individual merits of both quantitative and qualitative approaches. This article presents a brief overview of mixed methods research, and provides a content analysis how it is being used and reported in engineering education journals. Using a mixed methods approach, the authors first reviewed the application of mixed methods designs from fifteen articles from 2005 to the present against an existing set of criteria. Each article was then re-examined to explore how the authors address the integration of the quantitative and qualitative data. Finally consideration is given to how the authors of studies in engineering education interpret and report the meaning and use of mixed methods designs. The paper concludes with recommendations for engineering education researchers considering the use of a mixed methods approach.
Despite the publication of several methodological texts and papers written to address the proper use of a mixed methods research design 1-4, there is still a lack of consensus on how to integrate multiple types of data and discuss the results. A content analysis of social science articles published by Bryman 5 found that researchers would state the use of both qualitative and quantitative research methods but only report the results of one data type. Another content analysis of evaluation research articles found that 44% of the articles did not integrate the quantitative and qualitative data 4. Integration of the qualitative and quantitative data is a central tenet of a mixed methods research design. Mixing, defined by Creswell2 involves the “explicit relating of two data sets” and may occur in a number of ways including, merging, connecting or embedding. Instrumental to this definition is the notion of making the integration of two data sets explicit, indicating that where, how and why this happens should be clear to the audience.
Issues of data integration aside, a standard for the presentation of mixed methods research is equally unclear. Sandelowski 6 points out that methods alone do not define the nature of inquiry, but the execution and representation of the methods that signal the key differences. She goes on to discuss the lack of uniformity in presenting qualitative versus quantitative research, highlighting the question of whether data has even been mixed at all, or how the mixing occurred. Bryman 7 suggests that more attention be paid to the writing of mixed methods articles, going a step further to suggest that the “fundamental issue of the degree to which mixed methods researchers genuinely integrate their findings has not been addressed (p.8).” A study by engineering education researchers in 2009 discussed the ways that engineering education researchers could utilize educational research methods 8. Their results indicate that engineering education researchers could address their methods and evaluation criteria for their methods more explicitly. If engineering educations researchers are to maximize the impact of the data they collect, we argue that quantitative and qualitative data must be both integrated and reported in a meaningful way.
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2010 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015