June 14, 2015
June 14, 2015
June 17, 2015
Educational Research and Methods
26.482.1 - 26.482.9
Determining Reliability of Scores from an Energy Literacy RubricThe purpose of this paper is to estimate the reliability of scores from an energy literacy rubricdeveloped on the framework of the U.S. Department of Energy’s “Essential Principles andFundamental Concepts for Energy Education”. Energy literacy holds a place of prominencewithin engineering education, as the planet experiences increased demand for sustainable energygeneration and as the public and lawmakers strive to create energy policy that accounts fortradeoffs between complex options. Given that energy literacy levels in the population have beenfound to be generally low, there is a need to develop educational activities to improve energyliteracy. Example activities include initiatives such as high school energy competitions,development of interdisciplinary curricula, and field experiences and internships.Assessment of student learning is a central function of educational endeavors. Rubrics are avaluable way to assess competencies, such as those associated with energy literacy. Objects ofassessment using a rubric may include a performance, portfolio, poster, or other educationalartifacts. While rubrics provide a means to rate performances and products across multipledimensions and levels of proficiency, a concern in their use is that the rubric ratings aredependent upon the individuals supplying the ratings. Error is introduced into the assessmentprocess through this inherent characteristic of use. A rubric that can demonstrate a high level ofconsistency across raters—suggesting a small amount of measurement error—is highly valuable.Reliability refers to the consistency in observed assessment scores, and represents one piece ofevidence needed to support valid score interpretations and uses. This paper utilizesgeneralizability theory to determine the consistency of scores from the energy literacy rubric.The present study represents a necessary effort to document performance of the rubric forrefinement and to advance the study of energy literacy.
Gotch, C. M., & Langfitt, Q., & French, B. F., & Haselbach, L. (2015, June), Determining Reliability of Scores from an Energy Literacy Rubric Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23820
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015