Refinement of an Energy Literacy Rubric for Artifact Assessment and Application to the Imagine Tomorrow High School Energy Competition

Abstract

National efforts in energy education aim to increase energy literacy and a diverse set of tools is needed to assess the effectiveness of different educational endeavors. As an alternative to the standard testing-based approach for determining individual energy knowledge, this research further advances a rubric-based approach for assessing the energy literacy manifested in artifacts of a competition or project. The goals were to add additional validity, determine if revisions to the rubric and assessment process increased reliability over the initial study on abstracts, and to determine how well suited the approach is for application to posters – the final deliverable of the competition. The rubric was applied to abstracts and posters from Imagine Tomorrow, a high school energy competition, and the resulting scores were analyzed for both standard rater reliability measures and variable-linked trends. Results showed that interrater reliability on the abstract assessment was similar to that of the previous study, the poster assessment showed much higher reliability than the abstract assessment, and many of the same energy literacy trends identified in the previous research were present with additional data allowing for further trend investigation in this study. While this refinement did not appear to contribute to higher reliability, it has created a rubric that is more user friendly and valid. This study has preliminarily demonstrated that the rubric approach may be more appropriate for artifacts with more information, such as posters or reports, than for brief summaries, such as abstracts. In the posters, some trends based on competition variables were able to be identified. This constitutes the next step toward an implementable approach that has the potential to assess certain energy education endeavors.