Asee peer logo

Developing an Observation Protocol to Categorize Formative Assessment in Engineering Courses

Download Paper |

Conference

2018 ASEE Annual Conference & Exposition

Location

Salt Lake City, Utah

Publication Date

June 23, 2018

Start Date

June 23, 2018

End Date

July 27, 2018

Conference Session

Research Methods

Tagged Division

Educational Research and Methods

Page Count

6

Permanent URL

https://peer.asee.org/30300

Download Count

65

Request a correction

Paper Authors

biography

Max William Blackburn University of Michigan

visit author page

Max Blackburn is a fourth year undergraduate Electrical Engineering student at the University of Michigan, focusing in Power systems and Energy. He is currently assisting Dr. Cynthia Finelli with research concerning the effects of flexible learning spaces and formative assessment techniques.

visit author page

biography

Aaron W. Johnson University of Michigan

visit author page

Aaron W. Johnson is a postdoctoral research fellow at the University of Michigan. He received his Ph.D. in Aeronautics and Astronautics from the Massachusetts Institute of Technology in 2014, after which he served as a postdoctoral research fellow at the Tufts University Center for Engineering Education and Outreach. Aaron also obtained a master's degree from MIT in 2010 and a bachelor's degree from the University of Michigan in 2008, both in aerospace engineering.

visit author page

biography

Cynthia J. Finelli University of Michigan Orcid 16x16 orcid.org/0000-0001-9148-1492

visit author page

Dr. Cynthia Finelli is Associate Professor of Electrical Engineering and Computer Science, Associate Professor of Education, and Director and Graduate Chair for Engineering Education Research Programs at University of Michigan (U-M). Dr. Finelli is a fellow in the American Society of Engineering Education, a Deputy Editor of the Journal for Engineering Education, an Associate Editor of the IEEE Transactions on Education, and past chair of the Educational Research and Methods Division of ASEE. She founded the Center for Research on Learning and Teaching in Engineering at U-M in 2003 and served as its Director for 12 years. Prior to joining U-M, Dr. Finelli was the Richard L. Terrell Professor of Excellence in Teaching, founding director of the Center for Excellence in Teaching and Learning, and associate professor of electrical engineering at Kettering University.

Dr. Finelli's current research interests include student resistance to active learning, faculty adoption of evidence-based teaching practices, the use of technology and innovative pedagogies on student learning and success, and the impact of a flexible classroom space on faculty teaching and student learning. She also led a project to develop a taxonomy for the field of engineering education research, and she was part of a team that studied ethical decision-making in engineering students.

visit author page

Download Paper |

Abstract

Developing An Observation Protocol to Categorize Formative Assessment in Engineering Courses - Blackburn, Johnson, & Finelli

This work-in-progress paper details the development of an observation protocol to categorize student and instructor behavior during different types of formative assessment in the engineering classroom. Formative assessment is any formal or informal procedure that instructors use to elicit information about student comprehension in order to modify their teaching practices in real-time. It is in contrast to summative assessment, in which quizzes, exams, or project reports are used to gauge student understanding at the end of an instructional unit or semester. Different types of formative assessment give instructors a range of information about student understanding. For example, some types—such as “clicker” questions—provide a quantitative measure of how many students can answer a question correctly, while other types—such as impromptu “in the moment” responses to nonverbal cues, questions, or statements from students—can provide a deeper understanding of how students are conceptualizing different class topics or approaching problems. Our goal for this work-in-progress paper is to present the development of a classroom observation protocol to allow researchers to categorize the type of formative assessment used in undergraduate engineering courses. The protocol will be framed around the following research questions: 1) How do instructors implement formative assessment in large undergraduate engineering courses?, 2) How do students respond to different instances of formative assessment?, and 3) How do instructors change their teaching based on the positive or negative feedback they receive from students through formative assessment? Answering these research questions will allow us to assess which formative assessment strategies give instructors “deeper” understanding of student thinking. As the first step in developing our classroom observation protocol, we are currently reviewing 50 hours of video recordings from 7 undergraduate engineering courses at a large Midwestern university. We are identifying instances of formative assessment in these recordings and then coding the instances according to three categories: 1) the type of formative assessment, 2) students’ response to this formative assessment, and 3) changes made by the instructor based on students’ response. After developing our initial set of codes, we will combine similar codes in each category to develop a final list of codes for our classroom observation protocol. In this work-in-progress paper we will present our methodology and initial list of codes in each category, and we will seek feedback from the community in refining our list.

This research is part of a larger project to ascertain how different classroom settings can affect student-instructor interaction in large undergraduate engineering courses. The classroom observation protocol developed within this project will be used to study flexible classroom spaces that have movable tables and chairs and that, as a result, may allow the instructor to interact better with students and may ultimately lead to “deeper” types of formative assessment.

Blackburn, M. W., & Johnson, A. W., & Finelli, C. J. (2018, June), Developing an Observation Protocol to Categorize Formative Assessment in Engineering Courses Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. https://peer.asee.org/30300

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2018 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015