Salt Lake City, Utah
June 20, 2004
June 20, 2004
June 23, 2004
9.619.1 - 9.619.10
Formative Evaluation of Assessment Instruments for Statics
Sean St.Clair and Nelson Baker Georgia Institute of Technology
This paper describes a formative study that took place within the context of a larger project investigating the effects of technology on knowledge retention. In the larger project, students were evaluated at various points in time to assess their levels of learning and retention. The purpose of the formative study was not to assess students, but to evaluate the pretests, posttests, and examination questions that were later used to assess students. These instruments were tested for usability, reliability, and validity. In the formative study, the instruments were completed by students in two sections of a sophomore level mechanics course. The resulting data were compared via standard statistical techniques and the instruments were found to be reliable. The data were also analyzed for evidence of criterion-related validity and the instruments were found to be highly valid. After some changes were made based on student responses, the instruments were also found to be usable. This paper describes the formative study and the findings on the usability, reliability, and validity of the assessment instruments.
A longitudinal study has been conducted at the School of Civil and Environmental Engineering at Georgia Tech to determine the effects of technology use in the classroom on long-term retention. Specifically, students in three sections of a statics course used two different software titles during the truss analysis portion of the course to reinforce classroom instruction1. The students were assessed prior to using the software and at various points in time after using the software to determine the effects of software use on learning and retention. Three different assessment instruments were used in these evaluations: a pretest, a posttest, and an examination question.
One component of the longitudinal study was to evaluate these assessment instruments to ensure that they were usable, reliable, and valid. This evaluation took place during a formative phase of the study and was conducted prior to gathering the student data on learning and retention. The formative study is the focus of this paper. The summative results on learning and retention will be presented elsewhere.
Walker explained the distinction between formative and summative assessments in his popular Evaluation and Assessment Primer2. Summative assessments are performed at the conclusion of an intervention to determine the ultimate results of that intervention. Formative assessments are conducted during the course of the intervention and are intended to improve some aspect of the
Proceedings of the 2004 American Society for Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education
St.Clair, S., & Baker, N. (2004, June), Formative Evaluation Of Assessment Instruments For Statics Paper presented at 2004 Annual Conference, Salt Lake City, Utah. https://peer.asee.org/13221
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2004 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015