Asee peer logo

Automating Detection of Framing Agency in Design Team Talk

Download Paper |


2020 ASEE Virtual Annual Conference Content Access


Virtual On line

Publication Date

June 22, 2020

Start Date

June 22, 2020

End Date

June 26, 2021

Conference Session

Design Teams 1

Tagged Division

Design in Engineering Education

Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Ardeshir Raihanian Mashhadi University at Buffalo, SUNY

visit author page

Dr. Ardeshir Raihanian is an assistant professor of teaching in the Department of Mechanical and Aerospace Engineering at University at Buffalo. His research interests include user-centric design, sustainable design, user behavior simulation and agent based modeling. He also researches and publishes in areas surrounding engineering education. He has won multiple awards, including Design for Manufacture and the Life Cycle Technical Committee Best Paper(2017) and the International Life Cycle Academy Award for the best paper on Sustainable Consumption (2017). He is also responsible for teaching introductory, intermediate and advanced design related courses in the Department of Mechanical and Aerospace Engineering at University at Buffalo.

visit author page


Vanessa Svihla University of New Mexico Orcid 16x16

visit author page

Dr. Vanessa Svihla is a learning scientist and associate professor at the University of New Mexico in the Organization, Information and Learning Sciences program and in the Chemical and Biological Engineering Department. She served as Co-PI on an NSF RET Grant and a USDA NIFA grant, and is currently co-PI on three NSF-funded projects in engineering and computer science education, including a Revolutionizing Engineering Departments project. She was selected as a National Academy of Education / Spencer Postdoctoral Fellow and a 2018 NSF CAREER awardee in engineering education research. Dr. Svihla studies learning in authentic, real world conditions; this includes a two-strand research program focused on (1) authentic assessment, often aided by interactive technology, and (2) design learning, in which she studies engineers designing devices, scientists designing investigations, teachers designing learning experiences and students designing to learn.

visit author page

Download Paper |


Those who teach design contend with issues of authenticity and engagement. A problem that is too narrow or open can be challenging for students, yet finding that Goldilocks middle ground is complicated by many factors. Framing agency—making decisions that are consequential to framing and solving design problems—appears to provide clarity about student engagement with different types of design problems. However, detecting framing agency in design team talk is a labor intensive process. The purpose of this study was to evaluate a variety of text mining approaches for suitability in detecting framing agency in transcribed talk. We found no correlation between human-coding and sentiment analysis. However, interestingly, polarity derived from sentiment analysis did differentiate between a team that displayed almost no framing agency and those that did, with the former showing a high level of positivity. This reflects the lack of struggle, high certainty and agreeability the team displayed as they quickly agreed they all had similar and therefore correct answers. We also trained a Regularized Support Vector Machine Classifier to predict levels of framing agency, with the human-coded data as training data. The model showed 89% accuracy in detecting high framing agency. Given the recent increase in quality of auto-transcription tools, such approaches may lead to in-situ detectors in future.

Raihanian Mashhadi, A., & Svihla, V. (2020, June), Automating Detection of Framing Agency in Design Team Talk Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--34199

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2020 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015