Seattle, Washington
June 14, 2015
June 14, 2015
June 17, 2015
978-0-692-50180-1
2153-5965
Educational Research and Methods
9
26.482.1 - 26.482.9
10.18260/p.23820
https://peer.asee.org/23820
413
Chad Gotch's interests center on maximizing effective and proper use of educational and psychological measurements. To this end, he studies assessment/measurement literacy among teachers, score reporting, and building validity arguments from both technical and non-technical evidence. These complimentary lines of research inform the life cycle of assessment, from development to use and policy.
Quinn is a PhD student in the School of Civil and Environmental Engineering at Washington State University. His research is mostly focused on sustainability, including work on life cycle assessment and energy literacy assessment.
Brian F. French is a Professor of Educational Psychology with an emphasis in Psychometrics and Research Methods. He is the Director of the Learning and Performance Research Center at Washington State University.
Determining Reliability of Scores from an Energy Literacy RubricThe purpose of this paper is to estimate the reliability of scores from an energy literacy rubricdeveloped on the framework of the U.S. Department of Energy’s “Essential Principles andFundamental Concepts for Energy Education”. Energy literacy holds a place of prominencewithin engineering education, as the planet experiences increased demand for sustainable energygeneration and as the public and lawmakers strive to create energy policy that accounts fortradeoffs between complex options. Given that energy literacy levels in the population have beenfound to be generally low, there is a need to develop educational activities to improve energyliteracy. Example activities include initiatives such as high school energy competitions,development of interdisciplinary curricula, and field experiences and internships.Assessment of student learning is a central function of educational endeavors. Rubrics are avaluable way to assess competencies, such as those associated with energy literacy. Objects ofassessment using a rubric may include a performance, portfolio, poster, or other educationalartifacts. While rubrics provide a means to rate performances and products across multipledimensions and levels of proficiency, a concern in their use is that the rubric ratings aredependent upon the individuals supplying the ratings. Error is introduced into the assessmentprocess through this inherent characteristic of use. A rubric that can demonstrate a high level ofconsistency across raters—suggesting a small amount of measurement error—is highly valuable.Reliability refers to the consistency in observed assessment scores, and represents one piece ofevidence needed to support valid score interpretations and uses. This paper utilizesgeneralizability theory to determine the consistency of scores from the energy literacy rubric.The present study represents a necessary effort to document performance of the rubric forrefinement and to advance the study of energy literacy.
Gotch, C. M., & Langfitt, Q., & French, B. F., & Haselbach, L. (2015, June), Determining Reliability of Scores from an Energy Literacy Rubric Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23820
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015