A Systematic Review of Argument-assessment Frameworks in Engineering Education

Madison E. Andrews; Priyadarshan N. Patil

Download Paper | Permalink

Conference: 2021 ASEE Virtual Annual Conference Content Access
Location: Virtual Conference
Publication Date: July 26, 2021
Start Date: July 26, 2021
End Date: July 19, 2022
Conference Session: Assessing Hard-to-Measure Constructs in Engineering Education: Assessment Design and Validation Studies
Tagged Division: Educational Research and Methods
Tagged Topic: Diversity
Page Count: 19
DOI: 10.18260/1-2--36620
Permanent URL: https://peer.asee.org/36620
Download Count: 497

Paper Authors

biography

Madison E. Andrews University of Texas at Austin orcid.org/0000-0002-9653-9785

visit author page

Madison Andrews is a STEM Education doctoral student and graduate research assistant for the Center for Engineering Education at the University of Texas at Austin. She received her B.S. in Mechanical Engineering from Clemson University in 2017 and her M.S. in Mechanical Engineering from the University of Texas at Austin in 2020.

visit author page

biography

Priyadarshan N. Patil University of Texas at Austin orcid.org/0000-0001-8747-4679

visit author page

Priyadarshan Patil is an Operations Research doctoral candidate and graduate research assistant for the Center for Transportation Research at the University of Texas at Austin. Priyadarshan received his B.Tech. in Civil Engineering from Indian Institute of Technology Madras in 2015.

visit author page

Download Paper | Permalink

Abstract

Argumentation, the process in which students construct spoken or written arguments to articulate and justify claims or explanations, has been well-studied in the context of mathematics and science education. Engineering has not received the same treatment, as very few studies assess the quality and nature of arguments in engineering education. While it was non-existent a decade ago, there has been a shift towards understanding and usage of argumentation frameworks in engineering. The development of frameworks that can be used to assess the quality of student generated arguments is a foundational step in the adoption of argumentation in the field and researchers need access to and awareness about framework to gauge engineering arguments systematically.

In order to better understand the adoption of argumentation in engineering education, our research team conducted a systematic literature review. Systematic reviews can provide comprehensive summaries of previously conducted research, assessing both the general understanding of and the gaps within the literature of focus. In this review, a comprehensive collection of relevant publications was compiled by identifying appropriate search terms, databases and inclusion criteria. An initial search identified 478 results. Once 223 duplicates were removed, the titles and abstracts of the remaining 255 publications were screened and 201 records were removed because of their irrelevance to the topic of interest. Finally, the full-texts of 54 articles were assessed for eligibility and articles were excluded based on (1) lacked a framework (n = 25); (2) irrelevance to engineering in higher education (n = 15); and (3) examined the process of argumentation, rather than a produced argument (n = 2).

The full texts of the 11 qualifying studies were then examined and coded to reveal trends within the existing body of knowledge. Analysis revealed that only two types of analytic frameworks were used to examine the quality of student arguments in engineering education; both rely heavily on structural elements of arguments. These frameworks, which can clearly demarcate the structure of an argument, provide replicable templates for instruction and analysis that can be applied in a variety of contexts and compared between disciplines. However, literature in science and math education point to more nuanced approaches to assessing the nature and/or quality of arguments than by simply identifying structural components and assigning an arbitrary value assessment to them. Implications and future research are discussed.

Citation
Format

Andrews, M. E., & Patil, P. N. (2021, July), A Systematic Review of Argument-assessment Frameworks in Engineering Education Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36620

TY - CPAPER
AB - Argumentation, the process in which students construct spoken or written arguments to articulate and justify claims or explanations, has been well-studied in the context of mathematics and science education. Engineering has not received the same treatment, as very few studies assess the quality and nature of arguments in engineering education. While it was non-existent a decade ago, there has been a shift towards understanding and usage of argumentation frameworks in engineering. The development of frameworks that can be used to assess the quality of student generated arguments is a foundational step in the adoption of argumentation in the field and researchers need access to and awareness about framework to gauge engineering arguments systematically.

In order to better understand the adoption of argumentation in engineering education, our research team conducted a systematic literature review. Systematic reviews can provide comprehensive summaries of previously conducted research, assessing both the general understanding of and the gaps within the literature of focus. In this review, a comprehensive collection of relevant publications was compiled by identifying appropriate search terms, databases and inclusion criteria. An initial search identified 478 results. Once 223 duplicates were removed, the titles and abstracts of the remaining 255 publications were screened and 201 records were removed because of their irrelevance to the topic of interest. Finally, the full-texts of 54 articles were assessed for eligibility and articles were excluded based on (1) lacked a framework (n = 25); (2) irrelevance to engineering in higher education (n = 15); and (3) examined the process of argumentation, rather than a produced argument (n = 2).

The full texts of the 11 qualifying studies were then examined and coded to reveal trends within the existing body of knowledge. Analysis revealed that only two types of analytic frameworks were used to examine the quality of student arguments in engineering education; both rely heavily on structural elements of arguments. These frameworks, which can clearly demarcate the structure of an argument, provide replicable templates for instruction and analysis that can be applied in a variety of contexts and compared between disciplines. However, literature in science and math education point to more nuanced approaches to assessing the nature and/or quality of arguments than by simply identifying structural components and assigning an arbitrary value assessment to them. Implications and future research are discussed.

AU - Madison E. Andrews
AU - Priyadarshan N. Patil
CY - Virtual Conference
DA - 2021/07/26
PB - ASEE Conferences
TI - A Systematic Review of Argument-assessment Frameworks in Engineering Education
UR - https://peer.asee.org/36620
DO - 10.18260/1-2--36620
ER -