Asee peer logo

Multiple Choice Questions that Test Conceptual Understanding: A Proposal for Qualitative Two-Tier Exam Questions

Download Paper |


2015 ASEE Annual Conference & Exposition


Seattle, Washington

Publication Date

June 14, 2015

Start Date

June 14, 2015

End Date

June 17, 2015





Conference Session

Concept Inventories and Assessment of Knowledge

Tagged Division

Educational Research and Methods

Page Count


Page Numbers

26.1179.1 - 26.1179.15



Permanent URL

Download Count


Request a correction

Paper Authors


Dion Timmermann Hamburg University of Technology

visit author page

Dion Timmermann studied electrical engineering at Hamburg University of Technology, Hamburg, Germany. In his master thesis he worked on simulation methods for the signal and power analysis of high speed data links. He currently pursues his Ph.D. in the Engineering Education Research Group at Hamburg University of Technology, where he investigates students understanding in introductory electrical engineering.

visit author page


Christian H Kautz Hamburg University of Technology

visit author page

Christian H. Kautz has a Diplom degree in Physics from University of Hamburg and a Ph.D. in Physics (for work in Physics Education Research) from the University of Washington. Currently, he leads the Engineering Education Research Group at Hamburg University of Technology.

visit author page

Download Paper |


Multiple Choice Questions that Test Conceptual Understanding: A Proposal for Qualitative Two-Tier QuestionsMultiple-choice questions have become a common assessment method for a variety ofreasons, including the easy administration and grading of such tests. Although often only usedto test declarative knowledge, they can also be used to test conceptual understanding, acommon learning goal in engineering courses. However, in many cases it is difficult todevelop valid and reliable questions. This may be due to a small number of possibledistractors resulting in a high probability to guess the correct answer or because of similaritiesbetween tasks or phrases resulting in students memorizing patterns.This problem can be overcome by using two-tier multiple choice questions that satisfy severalcriteria:a) The first tier forces students to apply their conceptual understanding to make a predictionabout a specific situation.b) In the second tier students have to choose one of several statements to justify theirprediction.c) Ideally, there is no obvious relation between the items in both tiers.d) Distractors should be based on known student misconceptions.Multiple-choice questions that follow these criteria have, at least implicitly, been used in thepast. As an example, we note the multiple choice version of the "Classroom test of scientificreasoning" developed by Lawson [1]. However, in several cases, these criteria are not appliedsystematically, indicating a non-structured approach and thus a possible lack of literature onthis type of questions.Using examples from electrical engineering, we are going to explain and illustrate the above-mentioned criteria for two-tier multiple choice questions. We are going to point out thebenefits and limitations of this type of questions and compare them to others found inliterature, including assertion-reason questions and those found in the Chemical ConceptsInventory. Additionally, we are going to present the results of a 500-student exam whichincluded four questions meeting the criteria mentioned above. The answers to these closed-ended questions will be compared to the answers to very similar open-ended questions askedin previous years.We encourage the use of this type of question in cases where the aim is not to identify newstudent misconceptions but instead monitor the prevalence of known difficulties.[1] Lawson, Anton E. "The development and validation of a classroom test of formalreasoning." Journal of Research in Science Teaching 15.1 (1978): 11-24.

Timmermann, D., & Kautz, C. H. (2015, June), Multiple Choice Questions that Test Conceptual Understanding: A Proposal for Qualitative Two-Tier Exam Questions Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24516

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015