Asee peer logo

Computerized Algorithmic Approaches for Evaluating Systems Thinking of Both Engineers and Non-Engineers

Download Paper |


2021 ASEE Virtual Annual Conference Content Access


Virtual Conference

Publication Date

July 26, 2021

Start Date

July 26, 2021

End Date

July 19, 2022

Conference Session

TELPhE Division Technical Session 1: Expanding Technological and Engineering Literacies

Tagged Division

Technological and Engineering Literacy/Philosophy of Engineering

Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


John Krupczak Jr Hope College

visit author page

Professor of Engineering, Hope College, Holland, Michigan. Former Chair of the ASEE Technological Literacy Division; Former Chair of the ASEE Liberal Education Division; Senior Fellow CASEE, National Academy of Engineering, 2008-2010; Program Officer, National Science Foundation, Division of Undergraduate Education 2013-2016.

visit author page


A. Mehran Shahhosseini Indiana State University

visit author page

A. Mehran Shahhosseini is a Professor in the Department of Applied Engineering and Technology
Management and director of the PhD Program in Technology Management at Indiana State University. He has published over 50 articles in different journals and conference proceedings. He has served as an investigator for research projects sponsored by National Science Foundation, Ford Motor Company, and the US Army. Before working at Indiana State University, he was a faculty in the University of Louisville for 10 years. He also has over four years of industrial experience.
He received his D.Eng. degree in Mechanical Engineering from Lamar University (USA) in 1999, M.Sc. in Materials Engineering from Isfahan University of Technology in 1991, and B.Sc. in Metallurgical Engineering from Tehran University in 1988. He is a member of ASEE, ASME, and ATMAE.

visit author page


Darin R. Stephenson Hope College

visit author page

Professor of Mathematics at Hope College. Former Chair of the Executive Committee, Mathematical Association of America, Michigan Section. PhD and MS in mathematics from the University of Michigan. BS in Mathematics from the University of Kentucky.

visit author page

Download Paper |


This paper reports on an automated assessment to improve the systems thinking abilities of both engineering and non-engineering students. The program compares any node-and-link diagram of a system created by a student to an expert-generated diagram and produce a similarity rating. The ability to describe a technological system in the form of a diagram is an important element of engineering literacy. Creating a diagram that shows how system inputs are transformed into outputs by a network of interconnected components is also one characteristic of systems-level thinking. Systems-level thinking is critical for engineers as most modern engineered products are technological systems. Systems thinking includes the ability to adopt a holistic, integrative, synthesis perspective, identify system elements and their interactions, and recognize dynamic characteristics. Currently systems thinking is time-consuming for instructors to assess manually. This work reports on the use of automated assessment. The automated assessment compares favorably with by-hand evaluation of student diagrams by the instructor. The study was conducted with Introduction to Engineering students. Students we asked to create system diagrams of common household appliances. On average, the instructor rated each question slightly lower than the algorithm by about 5-9 percent. These are very encouraging results given that only the current basic algorithm was employed with no additional training of the algorithm for this specific problem. The time required to conduct a computer evaluation was about one minute per diagram. Automated instructional aids will become increasingly important in higher education. This work furthers the National Academy of Engineering Grand Challenge of advancing personalized learning by investigating the extent to which the critical engineering skill of systems thinking can be algorithmically assessed and improved. Systems thinking is a vital capability that industry finds lacking in entry-level engineers. The project will improve system analysis ability of engineers. Systems thinking is a portion of engineering available to anyone without the mathematics prerequisites. The non-engineering students participating in the study achieve results comparable to engineering students.

Krupczak, J., & Shahhosseini, A. M., & Stephenson, D. R. (2021, July), Computerized Algorithmic Approaches for Evaluating Systems Thinking of Both Engineers and Non-Engineers Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36831

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015