Fila, N. D., & Hess, J. L., & Purzer, S. (2015, June), A Dialectic Data Integration Approach for Mixed Methods Survey Validation  Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23376

\bibitem{asee_peer_23376}
  Fila, N. D., \& Hess, J. L., \& Purzer, S. (2015, June), \emph{A Dialectic Data Integration Approach for Mixed Methods Survey Validation}
  Paper presented at 2015 ASEE Annual Conference \& Exposition, Seattle, Washington.
  10.18260/p.23376

Nicholas D. Fila, Justin L Hess, and Senay Purzer.  "A Dialectic Data Integration Approach for Mixed Methods Survey Validation".  2015 ASEE Annual Conference & Exposition, Seattle, Washington, 2015, June.  ASEE Conferences, 2015.  https://peer.asee.org/23376 Internet.  03 May, 2024

\bibitem{asee_peer_23376}
  Nicholas D. Fila, Justin L Hess, and Senay Purzer.
  "A Dialectic Data Integration Approach for Mixed Methods Survey Validation".
  \emph{2015 ASEE Annual Conference \& Exposition, Seattle, Washington, 2015, June}.
  ASEE Conferences, 2015.
  https://peer.asee.org/23376 Internet.  03 May, 2024

@INPROCEEDINGS{asee_peer_23376
author = "Nicholas D. Fila, Justin L Hess, and Senay Purzer"
title = "A Dialectic Data Integration Approach for Mixed Methods Survey Validation"
booktitle = "2015 ASEE Annual Conference \& Exposition"
year = "2015"
month = "June"
address = "Seattle, Washington"
publisher = "ASEE Conferences"
note = {https://peer.asee.org/23376}
number = {10.18260/p.23376}
}

TY  - CPAPER
AB  -                 A Framework for Mixed Methods Survey ValidationQuantitative surveys are often used as tools to gauge student attitudes, knowledge, and behaviorsin engineering education contexts. Results from these surveys can be used to guide curricula, asformative assessment, to compare multiple groups, and for many other purposes. In order toensure that students scores, and any action taken as a result of those scores, are accurate andmeaningful in the implementation context, surveys often go through rigorous validationprocedures. Often these involve procedures such as confirmatory factor analysis to verify thefactor structure or comparison to results on previously validated measures. While these methodshave a longstanding track record, incorporation of qualitative data in a holistic mixed methodsmanner can improve understanding of the survey in the desired educational context. This paperwill present a methodological framework for incorporating both traditional quantitative data andqualitative data into the validation of a survey targeted at engineering students.First, we present the philosophical underpinnings of quantitative and qualitative validation anddiscuss connections that allow both traditions to be incorporated into the same survey validation.Briefly, both qualitative and quantitative validation share two common threads. First, bothemphasize thorough understanding of research participants’ context in order to make meaningfulinterpretation of results. Second, both are built upon utilizing a variety of data forms to develop acomprehensive understanding of phenomena in the survey-takers’ contexts.Stemming from these commonalities, we describe a procedure that incorporates both qualitativeand quantitative data into the evaluation of a Likert-type scale survey. We discuss the types ofdata that might be used in a mixed methods survey validation, with examples of this analysisusing authentic data. We then discuss how quantitative and qualitative data can be mixed to forma deeper understanding of the participants, their educational context, and how survey resultsmight be interpreted in that context among those participants. Analysis, in addition to occurringat quantitative and qualitative levels, also occurs at the item and structure levels. Item-levelanalyses are used to understand how item phrasing may affect student responses. Structure-levelanalyses are used to understand how participants connect to the primary constructs in the survey,and how/whether they are relevant in the participant context. Consideration of all data in unisonprovides a holistic understanding of the survey constructs in the participant contexts andimplications of using the survey as an assessment tool. We discuss tangible ways in whichdifferent types of data may be integrated and lead to more nuanced understanding of the surveyand its implications for the research context. We demonstrate this analysis using a previouslysurvey on a sample of 163 engineering students from a variety of disciplines and academiclevels. We expect that this paper will contribute to understanding of mixed methods research inengineering education as well as introducing a novel method of survey validation.
AU  - Nicholas D. Fila
AU  - Justin L Hess
AU  - Senay Purzer
CY  - Seattle, Washington
DA  - 2015/06/14
PB  - ASEE Conferences
TI  - A Dialectic Data Integration Approach for Mixed Methods Survey Validation
UR  - https://peer.asee.org/23376
DO  - 10.18260/p.23376
ER  -