Instructional Feedback Practices in First-Year Engineering Technical Writing Assignments: Qualitative Coding Synthesis, Analysis and Comparison

Connor Jenkins; Krista Kecskemety

Download Paper | Permalink

Conference: 2022 ASEE Annual Conference & Exposition
Location: Minneapolis, MN
Publication Date: August 23, 2022
Start Date: June 26, 2022
End Date: June 29, 2022
Conference Session: First-Year Programs Division Technical Session 10: Best of First-Year Programs Division
Page Count: 18
DOI: 10.18260/1-2--41538
Permanent URL: https://peer.asee.org/41538
Download Count: 328

Paper Authors

biography

Connor Jenkins The Ohio State University

visit author page

Connor is an Electrical Engineering PhD Student from Ohio State who graduated from the Ohio State University with a B.S. in Electrical and Computer Engineering in 2021. He currently works as a graduate research associate in the Wearable and Implantable Technology group at the ElectroScience Laboratory. His electrical engineering research interests include bioelectromagnetics, and electromagnetic device design, while his engineering education research focuses on feedback and first-year experience.

visit author page

biography

Krista Kecskemety The Ohio State University

visit author page

Krista Kecskemety is an Associate Professor of Practice in the Department of Engineering Education at The Ohio State University and the Director of the Fundamentals of Engineering for Honors Program. Krista received her B.S. in Aerospace Engineering at The Ohio State University in 2006 and received her M.S. from Ohio State in 2007. In 2012, Krista completed her Ph.D. in Aerospace Engineering at Ohio State. Her engineering education research interests include investigating first-year engineering student experiences, faculty experiences, and the research to practice cycle within first-year engineering.

visit author page

Download Paper | Permalink

Abstract

This complete research paper will describe both the synthesis and application of two coding schemes for written feedback for technical communication. The application of both coding schemes on written feedback left on a sample student lab report will also be reported. Technical communication is any form of communication that includes specialized technical information. This can include documents such as scientific reports, presentations, manuals, or even communications with colleagues. The ability to effectively communicate technical information is often seen as an asset in industry and is a required student outcome according to ABET. Technical communication skills are often taught to engineering students through assignments such as lab reports, presentations, or other technical documentation. Students can then gauge their mastery of technical communication through scores on such assignments, or through other feedback left by reviewers. Feedback can be particularly useful for students because it can effectively enhance learning. Important factors to consider when maximizing the effectiveness of feedback include timing, quantity, and content. Timing is often controlled by the structure of a course and the availability of the reviewers, while quantity and content are controlled by the individual reviewers training and previous experiences.

At a large midwestern university, technical communication skills are taught to first year engineering students through one of several required courses. Each of these courses are taught by a teaching team consisting of one instructor, one graduate teaching assistant (GTA) and three or more undergraduate teaching assistants (UTAs). UTAs in these courses are responsible for scoring most course assignments and providing feedback on student work. Despite the proven importance of high-quality feedback to student learning, most of the focus for UTA training is on scoring. The written feedback portion of their role is left mainly to examples shown during the original training session. Because written feedback is mostly self-monitored by the UTAs, the quality of and experiences that inform UTA written feedback are unknown.

A group of first-year engineering UTAs, one instructor, one GTA, and one engineering technical communication (ETC) faculty member graded and left feedback on a large technical writing sample. Their comments were then broken down into individual ideas. After a review of several coding schemes, no single method was sufficient to differentiate between dissimilar ideas seen in the sample. The review also led to the categorization of coding schemes into two categories: content and functional classification methods. Content classification methods focus on the explicit content or information in a piece of feedback, while functional classification methods focus on the purpose for the feedback. Two new coding schemes were synthesized, one for each classification method. Each idea was coded using both the content and the functional classification method. Coding was completed by two researchers and discussed until a consensus was reached for all coded ideas.

Results are examined by taking the number of ideas a code received per participant and dividing by the total number of ideas the same participant left. The participants were then grouped by role and compared. The small sample size and qualitative nature of the data made statistical analysis impractical, so observations were primarily used.

In general, the distributions in both coding schemes for the UTAs and the Experts (instructor, GTA and ETC faculty) were not similar. This indicates that the UTA and Expert groups do not have a shared idea of what written feedback they should give to students on technical communication assignments. Within the UTA group, code distributions also had a large spread, indicating that the UTAs may not share common experiences to draw upon while leaving written feedback. Further studies are needed to confirm this using insight from the participants.

Citation
Format

Jenkins, C., & Kecskemety, K. (2022, August), Instructional Feedback Practices in First-Year Engineering Technical Writing Assignments: Qualitative Coding Synthesis, Analysis and Comparison Paper presented at 2022 ASEE Annual Conference & Exposition, Minneapolis, MN. 10.18260/1-2--41538

TY - CPAPER
AB - This complete research paper will describe both the synthesis and application of two coding schemes for written feedback for technical communication. The application of both coding schemes on written feedback left on a sample student lab report will also be reported. Technical communication is any form of communication that includes specialized technical information. This can include documents such as scientific reports, presentations, manuals, or even communications with colleagues. The ability to effectively communicate technical information is often seen as an asset in industry and is a required student outcome according to ABET. Technical communication skills are often taught to engineering students through assignments such as lab reports, presentations, or other technical documentation. Students can then gauge their mastery of technical communication through scores on such assignments, or through other feedback left by reviewers. Feedback can be particularly useful for students because it can effectively enhance learning. Important factors to consider when maximizing the effectiveness of feedback include timing, quantity, and content. Timing is often controlled by the structure of a course and the availability of the reviewers, while quantity and content are controlled by the individual reviewers training and previous experiences.

At a large midwestern university, technical communication skills are taught to first year engineering students through one of several required courses. Each of these courses are taught by a teaching team consisting of one instructor, one graduate teaching assistant (GTA) and three or more undergraduate teaching assistants (UTAs). UTAs in these courses are responsible for scoring most course assignments and providing feedback on student work. Despite the proven importance of high-quality feedback to student learning, most of the focus for UTA training is on scoring. The written feedback portion of their role is left mainly to examples shown during the original training session. Because written feedback is mostly self-monitored by the UTAs, the quality of and experiences that inform UTA written feedback are unknown.

A group of first-year engineering UTAs, one instructor, one GTA, and one engineering technical communication (ETC) faculty member graded and left feedback on a large technical writing sample. Their comments were then broken down into individual ideas. After a review of several coding schemes, no single method was sufficient to differentiate between dissimilar ideas seen in the sample. The review also led to the categorization of coding schemes into two categories: content and functional classification methods. Content classification methods focus on the explicit content or information in a piece of feedback, while functional classification methods focus on the purpose for the feedback. Two new coding schemes were synthesized, one for each classification method. Each idea was coded using both the content and the functional classification method. Coding was completed by two researchers and discussed until a consensus was reached for all coded ideas.

Results are examined by taking the number of ideas a code received per participant and dividing by the total number of ideas the same participant left. The participants were then grouped by role and compared. The small sample size and qualitative nature of the data made statistical analysis impractical, so observations were primarily used.

In general, the distributions in both coding schemes for the UTAs and the Experts (instructor, GTA and ETC faculty) were not similar. This indicates that the UTA and Expert groups do not have a shared idea of what written feedback they should give to students on technical communication assignments. Within the UTA group, code distributions also had a large spread, indicating that the UTAs may not share common experiences to draw upon while leaving written feedback. Further studies are needed to confirm this using insight from the participants.
AU - Connor Jenkins
AU - Krista Kecskemety
CY - Minneapolis, MN
DA - 2022/08/23
PB - ASEE Conferences
TI - Instructional Feedback Practices in First-Year Engineering Technical Writing Assignments: Qualitative Coding Synthesis, Analysis and Comparison
UR - https://peer.asee.org/41538
DO - 10.18260/1-2--41538
ER -