Asee peer logo

An Anchored Open-Ended Survey Approach in Multiple Case Study Analysis

Download Paper |

Conference

2016 ASEE Annual Conference & Exposition

Location

New Orleans, Louisiana

Publication Date

June 26, 2016

Start Date

June 26, 2016

End Date

June 29, 2016

ISBN

978-0-692-68565-5

ISSN

2153-5965

Conference Session

Research Methods II: Meeting the Challenges of Engineering Education Research

Tagged Division

Educational Research and Methods

Page Count

12

DOI

10.18260/p.26566

Permanent URL

https://peer.asee.org/26566

Download Count

1619

Paper Authors

biography

Walter C. Lee Virginia Tech Orcid 16x16 orcid.org/0000-0001-5082-1411

visit author page

Dr. Walter Lee is an Assistant Professor in the Department of Engineering Education and the Assistant Director for Research in the Center for the Enhancement of Engineering Diversity (CEED), both at Virginia Tech. Lee’s research interests include co-curricular support, student success and retention, and diversity in STEM. He received his Ph.D in Engineering Education from Virginia Tech, M.S. in Industrial & Systems Engineering from Virginia Tech, and B.S. in Industrial Engineering from Clemson University.

visit author page

biography

Benjamin David Lutz Virginia Tech Orcid 16x16 orcid.org/0000-0003-2637-0942

visit author page

Ben Lutz is a PhD student in the Department of Engineering Education at Virginia Tech. His research interests include design teaching and learning, mentoring in design and project work, student experiences in engineering design, the transition from engineering school into the workplace, and also efforts for inclusion and diversity within engineering. His current work is in related understanding how students describe their own learning in engineering, and how that learning supports transfer of learning from school into professional practice as well as exploring students' conceptions of diversity and its importance within engineering fields.

visit author page

Download Paper |

Abstract

Survey questions are typically presented in one of two formats: closed-ended or open-ended. Despite the widespread use of open-ended survey questions in educational research, making sense of the answers to such questions remains a challenge. Whereas closed-ended questions lend themselves to well-known quantitative data-analysis techniques, open-ended questions have a tendency to illicit responses that are more difficult to analyze. Open-ended survey questions are advantageous because they allow researchers to explore responses when 1) answers to a question are not known in advance or 2) narrative responses are more useful. Open-ended survey questions are also used as “catch-all” or venting questions, giving respondents an opportunity to discuss topics that may not have been addressed by closed-ended questions. Given the varied use of the open-ended question format, researchers are often unsure how to use the results, if at all: survey research literature does not provide sufficient examples illustrating suitable data-analysis procedures for this question format. This challenge is further exacerbated when the respondents are reflecting on different contexts, such as different cases or research sites.

The purpose of this paper is to present an anchored open-ended survey approach as a solution for combating the disadvantages of open-ended survey questions while preserving the capacity to explore new or unanticipated responses. The approach uses closed-ended questions to anchor the responses to open-ended questions. An example of an anchored open-ended question is prompting students to indicate whether their interactions with faculty/staff were impacted positively, negatively, or neutrally by a student support center before asking them to provide specific examples of how their experience interacting with faculty/staff was influenced by a student support center.

In this paper, we illustrate the use of anchored open-ended survey questions as a method for multiple case study analysis. First, we present an overview of open-ended survey questions and their usefulness in qualitative research. We then discuss the lead author’s use of anchored open-ended questions in a multi-case study to investigate the impact of co-curricular support on the student experience. Next, we discuss the approach used to analyze the open-ended survey questions and the role of the anchors in completing this process. Lastly, we address the advantages and disadvantages of using open-ended questions in multiple case study analysis. The results of this application demonstrate several ways that engineering education researchers can use anchored open-ended survey questions. Although we applied the anchored open-ended question format to a multiple case study analysis, we believe that additional research approaches could use this data collection and analysis technique as well.

Lee, W. C., & Lutz, B. D. (2016, June), An Anchored Open-Ended Survey Approach in Multiple Case Study Analysis Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. 10.18260/p.26566

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2016 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015