A Mixed Methods Analysis and Evaluation of the Mixed Methods Research Literature in Engineering Education

Rachel Louis Kajfez; Elizabeth G. Creamer

Download Paper | Permalink

Conference: 2014 ASEE Annual Conference & Exposition
Location: Indianapolis, Indiana
Publication Date: June 15, 2014
Start Date: June 15, 2014
End Date: June 18, 2014
ISSN: 2153-5965
Conference Session: Methodological & Theoretical Contributions to Engineering Education 3
Tagged Division: Educational Research and Methods
Page Count: 16
Page Numbers: 24.68.1 - 24.68.16
DOI: 10.18260/1-2--19960
Permanent URL: https://peer.asee.org/19960
Download Count: 3271

Request a correction

Paper Authors

biography

Rachel Louis Kajfez Ohio State University orcid.org/0000-0001-9745-1921

visit author page

Dr. Rachel Louis Kajfez is an Assistant Professor of Practice in the Engineering Education Innovation Center and the Department of Civil, Environmental, and Geodetic Engineering at The Ohio State University. She earned her B.S. and M.S. degrees in Civil Engineering from Ohio State and earned her Ph.D. in Engineering Education from Virginia Tech. Her research interests focus on the intersection between motivation and identity of undergraduate and graduate students, first-year engineering programs, mixed methods research, and innovative approaches to teaching. Currently, she teaches within the first-year engineering program at Ohio State while maintaining an active engineering education research program.

visit author page

biography

Elizabeth G. Creamer Virginia Polytechnic Institute and State University

visit author page

Dr. Elizabeth G. Creamer is professor, Educational Research and Evaluation in the School of Education at Virginia Polytechnic Institute and State University where she teaches graduate level courses in mixed methods research. She is working on a manuscript for a new introductory textbook, Introduction to Fully Integrated Mixed Methods Research.

visit author page

Download Paper | Permalink

Abstract

A Mixed Methods Analysis and Evaluation of the Mixed Methods Research Literature in Engineering EducationMixed methods research is still emerging as an accepted and rigorous method of inquiry. Pastwork on mixed methods shows that there continue to be disagreements among scholars aboutterminology, various ways to classify research designs, and mixed methods researchcharacteristics. Various studies have examined quantitative and qualitative research accepted asrigorous in the field of engineering education, but minimal work has been done on the extent thatmixed methods research has been utilized by researchers in the field and more importantly, thequality of that research. The content analysis presented here builds on previous studies aboutmixed methods in engineering education providing an overview of the mixed methods articlespublished in engineering education journals and a proposed strategy for evaluating the quality ofmixed methods articles. Specifically, our intention is to provide insight into the state of mixedmethods research in the field through a synthesis type paper while discussing ways to improvethis mode of inquiry in future research.To begin to understand mixed methods research in engineering education, a content analysis ofmixed methods research articles in the field was conducted to answer the following researchquestions: (1) What evidence is there that the authors of the sampled publications meetcontemporary definitions of mixed methods research by including procedures to integrate or mixthe qualitative and quantitative data? (2) What articles from the sample best reflectcontemporary assumptions about mixed methods research methodology? To answer thesequestions, articles were gathered from the Journal of Engineering Education, Advances inEngineering Education, and the European Journal of Engineering Education. The articles weregathered from 2005-2011 allowing for trends to be observed in mixed methods research asengineering education has continued to develop over the years. Approximately 30 articles werefound in the journals for analysis that were considered to be mixed methods studies. A study wasconsidered to be mixed methods if a qualitative (inductive) and quantitative (deductive) strand ofdata was included in the study.The content analysis of the articles revealed that there are unique trends related to the quality ofmixed methods research published in the different journals. For example, mixing was mostcommonly observed in articles from the Journal of Engineering Education when compared toAdvances in Engineering Education (8 out of 10 containing mixing opposed to 4 out of 11).Also comparing our findings to past studies that were similar in nature, we observed that manyarticles do not provide a clear rationale for mixing and that mixed methods focused researchquestions are not often provided even in high quality studies. The results of this research helpsinform both engineering education research and mixed methods research by makingrecommendations about how both fields can move forward in continuing to contribute to qualityresearch in education.

Citation
Format

Kajfez, R. L., & Creamer, E. G. (2014, June), A Mixed Methods Analysis and Evaluation of the Mixed Methods Research Literature in Engineering Education Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. 10.18260/1-2--19960

TY  - CPAPER
AB  -                 A Mixed Methods Analysis and Evaluation of the          Mixed Methods Research Literature in Engineering EducationMixed methods research is still emerging as an accepted and rigorous method of inquiry. Pastwork on mixed methods shows that there continue to be disagreements among scholars aboutterminology, various ways to classify research designs, and mixed methods researchcharacteristics. Various studies have examined quantitative and qualitative research accepted asrigorous in the field of engineering education, but minimal work has been done on the extent thatmixed methods research has been utilized by researchers in the field and more importantly, thequality of that research. The content analysis presented here builds on previous studies aboutmixed methods in engineering education providing an overview of the mixed methods articlespublished in engineering education journals and a proposed strategy for evaluating the quality ofmixed methods articles. Specifically, our intention is to provide insight into the state of mixedmethods research in the field through a synthesis type paper while discussing ways to improvethis mode of inquiry in future research.To begin to understand mixed methods research in engineering education, a content analysis ofmixed methods research articles in the field was conducted to answer the following researchquestions: (1) What evidence is there that the authors of the sampled publications meetcontemporary definitions of mixed methods research by including procedures to integrate or mixthe qualitative and quantitative data? (2) What articles from the sample best reflectcontemporary assumptions about mixed methods research methodology? To answer thesequestions, articles were gathered from the Journal of Engineering Education, Advances inEngineering Education, and the European Journal of Engineering Education. The articles weregathered from 2005-2011 allowing for trends to be observed in mixed methods research asengineering education has continued to develop over the years. Approximately 30 articles werefound in the journals for analysis that were considered to be mixed methods studies. A study wasconsidered to be mixed methods if a qualitative (inductive) and quantitative (deductive) strand ofdata was included in the study.The content analysis of the articles revealed that there are unique trends related to the quality ofmixed methods research published in the different journals. For example, mixing was mostcommonly observed in articles from the Journal of Engineering Education when compared toAdvances in Engineering Education (8 out of 10 containing mixing opposed to 4 out of 11).Also comparing our findings to past studies that were similar in nature, we observed that manyarticles do not provide a clear rationale for mixing and that mixed methods focused researchquestions are not often provided even in high quality studies. The results of this research helpsinform both engineering education research and mixed methods research by makingrecommendations about how both fields can move forward in continuing to contribute to qualityresearch in education.
AU  - Rachel Louis Kajfez
AU  - Elizabeth G. Creamer
CY  - Indianapolis, Indiana
DA  - 2014/06/15
PB  - ASEE Conferences
TI  - A Mixed Methods Analysis and Evaluation of the Mixed Methods Research Literature in Engineering Education
UR  - https://peer.asee.org/19960
DO  - 10.18260/1-2--19960
ER  -