June 14, 2015
June 14, 2015
June 17, 2015
Educational Research and Methods
26.1179.1 - 26.1179.15
Multiple Choice Questions that Test Conceptual Understanding: A Proposal for Qualitative Two-Tier QuestionsMultiple-choice questions have become a common assessment method for a variety ofreasons, including the easy administration and grading of such tests. Although often only usedto test declarative knowledge, they can also be used to test conceptual understanding, acommon learning goal in engineering courses. However, in many cases it is difficult todevelop valid and reliable questions. This may be due to a small number of possibledistractors resulting in a high probability to guess the correct answer or because of similaritiesbetween tasks or phrases resulting in students memorizing patterns.This problem can be overcome by using two-tier multiple choice questions that satisfy severalcriteria:a) The first tier forces students to apply their conceptual understanding to make a predictionabout a specific situation.b) In the second tier students have to choose one of several statements to justify theirprediction.c) Ideally, there is no obvious relation between the items in both tiers.d) Distractors should be based on known student misconceptions.Multiple-choice questions that follow these criteria have, at least implicitly, been used in thepast. As an example, we note the multiple choice version of the "Classroom test of scientificreasoning" developed by Lawson . However, in several cases, these criteria are not appliedsystematically, indicating a non-structured approach and thus a possible lack of literature onthis type of questions.Using examples from electrical engineering, we are going to explain and illustrate the above-mentioned criteria for two-tier multiple choice questions. We are going to point out thebenefits and limitations of this type of questions and compare them to others found inliterature, including assertion-reason questions and those found in the Chemical ConceptsInventory. Additionally, we are going to present the results of a 500-student exam whichincluded four questions meeting the criteria mentioned above. The answers to these closed-ended questions will be compared to the answers to very similar open-ended questions askedin previous years.We encourage the use of this type of question in cases where the aim is not to identify newstudent misconceptions but instead monitor the prevalence of known difficulties. Lawson, Anton E. "The development and validation of a classroom test of formalreasoning." Journal of Research in Science Teaching 15.1 (1978): 11-24.
Timmermann, D., & Kautz, C. H. (2015, June), Multiple Choice Questions that Test Conceptual Understanding: A Proposal for Qualitative Two-Tier Exam Questions Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24516
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015