June 15, 2014
June 15, 2014
June 18, 2014
Liberal Education/Engineering & Society
24.605.1 - 24.605.18
Final Results of Reliability Testing for the xx Presentation Scoring SystemOur team has completed our final step in the concerted effort to provide an analytical basis forthe reliability of the xx Presentation Scoring System. Previously we have reported on (1)reliability testing results on an earlier version of the presentation scoring system with 19presentation skills and (2) our use of the first stage of a modified Delphi Method to refine thescoring system to a set of 13 skills. Now we can report on the final results of the modifiedDelphi Method, which includes our final reliability testing. Based on our extensive literaturereview, we believe this work is rare for presentation scoring systems. In this final phase, weconducted a study including presentations by students in biomedical engineering and industrialengineering.We report the reliability levels for each of the 13 presentation skills--for example, “linking thematerial presented to the big picture." Approximately 20 presentations were rated by groups ofundergraduate peers, teaching assistants, and faculty members. We calculated the frequency ofexact matches and matches within one point on a 5-point scale. The matching frequency wasdone using both a pairwise approach (all raters are compared to each other equally) and acriterion referenced approach (raters are compared to expert referees). The reliability statisticswere divided into three categories: excellent reliability, good reliability, and reasonable but lowerreliability. The information demonstrates that the scoring system can be used reliably in multipleengineering education settings. We will demonstrate just how to implement the presentationscoring system in different engineering education settings.
Norback, J. S., & Utschig, T. T., & Bonifonte, A. J., & Ross, G. J. (2014, June), Final Results of Reliability Testing for the Norback-Utschig Presentation Scoring System and Implications for Instruction Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. https://peer.asee.org/20496
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2014 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015