New Orleans, Louisiana
June 26, 2016
June 26, 2016
August 28, 2016
Energy Conversion and Conservation
Energy literacy encompasses knowledge of energy principles in technical, social, and economic realms, as well as the ability to critically apply that knowledge to solve problems and form opinions. Collective advancement of energy literacy among the general population is thought to be instrumental in implementing sustainable energy solutions in the near future. As efforts to improve energy literacy have advanced, so has the need to assess the outcomes of those efforts. This paper describes advancements in a recently developed approach of examining energy literacy in student projects through application of a rubric, and the results of a case study using the methodology on the Imagine Tomorrow high school energy competition. Changes made to the approach include a more detailed rater calibration session and a significant increase in the number of raters over a previous cases study which used the same rubric. Similar to the previous study, results show that raters exhibit moderate to substantial agreement when interrater reliability is measured by Kendall’s coefficient of concordance. As a component of this paper, group-wise comparisons of raters (pairs, triplets, and quadruplets) are examined to see if conclusions might have been different with different subsets of raters, both in terms of agreement statistics and in terms of energy literacy characteristics exhibited by various discrete groupings of students. No subset of raters would have resulted in significantly different conclusions in terms of scoring trends, though reliability statistics would be slightly altered. With respect to the competition energy literacy characteristics, it was found that posters created by students participating in more techno-centric challenges, with competition experience, or when mentored by returning advisors scored slightly higher than others. The energy literacy observed was unaffected by gender of the students or the teaching subject of advisors. Continual assessment, and improvement of assessment instruments, is vital as project-based learning continues to be a focal point for teaching about energy, and as organizers plan how to best shape future events to improve energy literacy of our current and future decision-makers.
Langfitt, Q., & Haselbach, L. (2016, June), Rubric-Based Energy Literacy Assessment of Student Posters: Effects of Extended Calibration and Addition of Raters Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. 10.18260/p.26128
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2016 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015