Asee peer logo

Formative Evaluation Of Assessment Instruments For Statics

Download Paper |

Conference

2004 Annual Conference

Location

Salt Lake City, Utah

Publication Date

June 20, 2004

Start Date

June 20, 2004

End Date

June 23, 2004

ISSN

2153-5965

Conference Session

Trends in Mechanics Education

Page Count

10

Page Numbers

9.619.1 - 9.619.10

Permanent URL

https://peer.asee.org/13221

Download Count

68

Request a correction

Paper Authors

author page

Sean St.Clair

author page

Nelson Baker

Download Paper |

Abstract
NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

Session 1168

Formative Evaluation of Assessment Instruments for Statics

Sean St.Clair and Nelson Baker Georgia Institute of Technology

Abstract

This paper describes a formative study that took place within the context of a larger project investigating the effects of technology on knowledge retention. In the larger project, students were evaluated at various points in time to assess their levels of learning and retention. The purpose of the formative study was not to assess students, but to evaluate the pretests, posttests, and examination questions that were later used to assess students. These instruments were tested for usability, reliability, and validity. In the formative study, the instruments were completed by students in two sections of a sophomore level mechanics course. The resulting data were compared via standard statistical techniques and the instruments were found to be reliable. The data were also analyzed for evidence of criterion-related validity and the instruments were found to be highly valid. After some changes were made based on student responses, the instruments were also found to be usable. This paper describes the formative study and the findings on the usability, reliability, and validity of the assessment instruments.

Introduction

A longitudinal study has been conducted at the School of Civil and Environmental Engineering at Georgia Tech to determine the effects of technology use in the classroom on long-term retention. Specifically, students in three sections of a statics course used two different software titles during the truss analysis portion of the course to reinforce classroom instruction1. The students were assessed prior to using the software and at various points in time after using the software to determine the effects of software use on learning and retention. Three different assessment instruments were used in these evaluations: a pretest, a posttest, and an examination question.

One component of the longitudinal study was to evaluate these assessment instruments to ensure that they were usable, reliable, and valid. This evaluation took place during a formative phase of the study and was conducted prior to gathering the student data on learning and retention. The formative study is the focus of this paper. The summative results on learning and retention will be presented elsewhere.

Objectives

Walker explained the distinction between formative and summative assessments in his popular Evaluation and Assessment Primer2. Summative assessments are performed at the conclusion of an intervention to determine the ultimate results of that intervention. Formative assessments are conducted during the course of the intervention and are intended to improve some aspect of the

Proceedings of the 2004 American Society for Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education

St.Clair, S., & Baker, N. (2004, June), Formative Evaluation Of Assessment Instruments For Statics Paper presented at 2004 Annual Conference, Salt Lake City, Utah. https://peer.asee.org/13221

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2004 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015