Are Multiple-Choice Questions Suitable for a Final Examination in a STEM Course?
- Conference
- Location
-
Indianapolis, Indiana
- Publication Date
-
June 15, 2014
- Start Date
-
June 15, 2014
- End Date
-
June 18, 2014
- ISSN
-
2153-5965
- Conference Session
- Tagged Division
-
Division Experimentation & Lab-Oriented Studies
- Tagged Topic
-
NSF Grantees Poster Session
- Page Count
-
13
- Page Numbers
-
24.195.1 - 24.195.13
- DOI
-
10.18260/1-2--20086
- Permanent URL
-
https://peer.asee.org/20086
- Download Count
-
1568
Paper Authors
Garrick A. Aden-Buie University of South Florida
Garrick Aden-Buie is a doctoral student in the Department of Industrial and Management Systems Engineering at the University of South Florida. He received a B.S. in Applied Mathematics and a B.A. in Spanish from Lehigh University. His research interests include predictive modeling for healthcare decision support and sustainable dynamic systems.
Autar Kaw University of South Florida
Dr. Autar Kaw is a professor of mechanical engineering at the University of South Florida. He was named the 2012 U.S. Professor the Year (Doctoral Institutions) by the Carnegie Foundation for the Advancement of Teaching and the Council for Advancement and Support of Education. The U.S. Professor of the Year award is the highest honor in the nation for undergraduate teaching. He received his BE Honors degree in Mechanical Engineering from Birla Institute of Technology and Science (BITS), India in 1981, and his degrees of Ph.D. in 1987 and M.S. in 1984, both in Engineering Mechanics from Clemson University, SC. He joined University of South Florida in 1987.
Professor Kaw's main scholarly interests are in engineering education research, open courseware development, bascule bridge design, fracture mechanics, composite materials, and the state and future of higher education. His research has been funded by National Science Foundation, Air Force Office of Scientific Research, Florida Department of Transportation, and Wright Patterson Air Force Base.
Professor Kaw has written several textbooks on subjects such as composite materials, numerical methods, computer programming and engineering licensure examination.
Since 2002, under Professor Kaw's leadership, he and his colleagues from around the nation have developed, implemented, refined and assessed online resources for an open courseware in Numerical Methods (http://nm.MathForCollege.com). The courseware gets more than a million page views per year. This is in addition to 900,000 views of the YouTube lectures and 150,000 annual visitors to the "numerical methods guy" blog.
Professor Kaw's opinion editorials have appeared in the Tampa Bay Times and Tampa Tribune, and his work has been covered/cited in Chronicle of Higher Education, Inside Higher Education, Congressional Record, ASEE Prism, Campus Technology, Florida Trend Magazine, WUSF, Bay News 9, NSF Discoveries, Voice of America, Times of India, and Indian Express. He has written more than 80 refereed technical papers.
Professor Kaw is a Fellow of the American Society of Mechanical Engineers (ASME) and a member of the American Society of Engineering Education (ASEE). He has also been a Maintenance Engineer (1982) for Ford-Escorts Tractors, India, and a Summer Faculty Fellow (1992) and Visiting Scientist (1991) at Wright Patterson Air Force Base.
Ali Yalcin University of South Florida
Dr. Ali Yalcin received his B.S., M.S., and Ph.D. degrees in Industrial and Systems Engineering from Rutgers University, New Brunswick New Jersey in 1995, 1997 and 2000. He is currently an Associate Professor at the University of South Florida, Industrial and Management Systems Engineering Department, and an Associate Faculty member of the Center for Urban Transportation Research. His research interests include systems modeling, analysis and control, data analysis and decision support in healthcare, information systems and engineering education research. His work has been funded by federal organizations including National Science Foundation and Army Office of Research and medical device manufacturing industry. He has taught courses in the areas of systems modeling and performance analysis, information systems design, production planning, facilities design, and systems simulation. He co-authored the 2006 Joint Publishers Book-of-the-Year textbook, Design of Industrial Information Systems, Elsevier.
Ram Pendyala Arizona State University
Ram M. Pendyala is a Professor of Transportation Systems in the School of Sustainable Engineering and the Built Environment at Arizona State University. His expertise lies in the study of human activity-travel behavior, sustainable mobility strategies, public transportation systems, and the land use, travel, energy, and air quality impacts of a wide range of transportation policies and technologies. Dr. Pendyala has conducted more than $5 million in sponsored research and published nearly 100 peer-reviewed journal articles and book chapters. He serves on the editorial boards of a number of journals including Transportation, Transport Reviews, Journal of Choice Modeling, and Transportation Letters. He is the chair of the Travel Analysis Methods Section of the Transportation Research Board and the immediate past chair of its Committee on Traveler Behavior and Values. He is also the immediate past chair of the International Association for Travel Behaviour Research (IATBR). Dr. Pendyala has his PhD and Masters degrees in Civil Engineering with a specialization in transportation systems from the University of California at Davis. He obtained his undergraduate degree in Civil Engineering from the Indian Institute of Technology - Madras in Chennai, India.
Abstract
Are Multiple-Choice Questions Suitable for a Final Examination in a STEMCourse?With decreasing budgets for teaching assistants, large class sizes, and increasedteaching loads, it is becoming ever more important to effectively utilize resourceswithout sacrificing best practices of assessment. One of the assessment tools that thesecond author uses in a Numerical Methods course at the University of________________ is a multiple-choice final examination. As part of a collaborativegrant study, the fourth author who was giving the same examination to his NumericalMethods course at the __________ University had reservations that a multiple-choicetest would be adequate to assess students’ comprehension of the course material.All authors agree that assessment is more than for assigning a grade – it is for“monitoring their learning, actively evaluate their strategies and their current levels ofunderstanding.” (How People Learn by Bransford, Brown, and Cocking 1999). In theauthors’ collective teaching experience of 59 years, less than 1% of the students returnto see the graded final examination and utilize it as a learning mechanism. Therefore, inmost courses, the final examination only serves the purpose of grading and is not usedas a learning mechanism by an overwhelming majority of students.To evaluate the use of a multiple-choice final examination as a replacement for thesame examination in free-response format, we designed an experimental study that wasconducted over three semesters. The final examination has 24 questions based on 8topics in the course. The questions are based on Bloom’s taxonomy.Semester 1: We gave the final examination questions as free response questions.Each question was graded based on a rubric of a scale of 0-4.Semester 2: We gave the final examination as multiple-choice questions with partialcredit. Each question answered correctly garnered 4 points. If the wrong choice waschosen, it was graded according to the same rubric as Semester 1. Of course, themaximum score would not be more than 3 for an incorrectly answered question.Semester 3: We gave the final examination as multiple-choice questions only. Acorrect choice earned the student 4 points while an incorrect or unanswered choicefetched 0 points. No partial credit was given.A Spearman's rank test was used to compare student performance in the numericalmethods course up to the final exam with student performance on the final exam.Significant correlation was found withIn the semester in which the multiple-choice with partial credit final exam wasadministered, student rankings as determined by performance in the course aresignificantly correlated to rankings of the same students as determined by the finalexam, both with and without the partial credit component. Furthermore, beyond theaddition of extra points for partial correctness, the partial credit component of themultiple-choice exam does not affect student ranking.These results indicate that multiple-choice final exams are effective tools for measuringsuccessful achievement of student learning objectives, and are also efficient withrespect to use of instructor resources and time. The detailed analysis and discussion ofthe results will be available for the draft paper.
Aden-Buie, G. A., & Kaw, A., & Yalcin, A., & Pendyala, R. (2014, June), Are Multiple-Choice Questions Suitable for a Final Examination in a STEM Course? Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. 10.18260/1-2--20086
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2014 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015