Introducing systems thinking as a framework for teaching and assessing threat modeling competency

Siddhant Sanjay Joshi; Preeti Mukherjee; Kirsten A. Davis; James C Davis

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Software Engineering Division (SWED) Technical Session #1
Tagged Division: Software Engineering Division (SWED)
Tagged Topic: Diversity
Permanent URL: https://peer.asee.org/47684

Request a correction

Paper Authors

biography

Siddhant Sanjay Joshi School of Engineering Education, Purdue University, West Lafayette orcid.org/0000-0003-2208-9002

visit author page

Siddhant is a Ph.D. candidate in the School of Engineering Education at Purdue University, West Lafayette. His research interests are in developing sociotechnical systems thinking skills in engineering graduates, understanding the role of emotions during complex problem-solving, and understanding the readiness of engineers for workforce

visit author page

author page

Preeti Mukherjee Purdue University

biography

Kirsten A. Davis Purdue University orcid.org/0000-0002-9929-5587

visit author page

Kirsten Davis is an assistant professor in the School of Engineering Education at Purdue University. Her research explores the intentional design and assessment of global engineering programs, student development through experiential learning, and approaches for teaching and assessing systems thinking skills. Kirsten holds a B.S. in Engineering & Management from Clarkson University and an M.A.Ed. in Higher Education, M.S. in Systems Engineering, and Ph.D. in Engineering Education, all from Virginia Tech.

visit author page

author page

James C Davis Purdue University orcid.org/0000-0003-2495-686X

Download Paper | Permalink

Abstract

Computing systems face diverse and substantial cybersecurity threats. Software engineers can mitigate some of these threats through appropriate software design and analysis, provided they are trained in appropriate competencies. One fundamental cybersecurity competency is threat modeling (Xiong & Lagerström, 2019), which is a systematic approach to identifying, mapping, and mitigating design-level security problems (Soares Cruzes et al., 2018). There are many frameworks for teaching threat modeling, but our analysis of these frameworks and existing coursework suggests that (1) these approaches tend to be focused on component-level analysis rather than educating students to reason holistically about a system’s cybersecurity, and (2) there is no rubric for assessing a student’s threat modeling competency.

To address these concerns, we propose systems thinking as a framework for teaching and assessing threat modeling competency. Prior studies by Young & Leveson (2013) and Yan (2020) suggest systems thinking can be a suitable approach for understanding and mitigating cybersecurity threats. Further, Tisdale (2015) synthesizes literature to argue that a holistic approach like systems thinking is needed to address cybersecurity risks. The purpose of this work-in-progress study is therefore to develop and pilot a rubric that uses systems thinking as a way to assess the threat modeling approach of computer engineering students. Based on our findings, we also discuss how systems thinking could be integrated into the teaching of threat modeling.

To conduct this study, we are developing a novel rubric for assessing threat modeling competency based on systems thinking (e.g., System Engineering approach (Ross et al., 2018)). We will use this rubric to assess threat models created during upper-level software design projects at a large midwestern university in the USA (24 student teams in Fall 2021 and 37 student teams in Spring 2023). We will compare these scores to the baseline rubric used in the course, which was derived directly from the industry standard STRIDE threat modeling framework. Our work will contribute by helping educators understand: (1) trends in threat modeling approaches undertaken by students; (2) identifying blindspots in their threat modeling approach; (3) describing a new rubric for assessing threat modeling based on systems thinking; and (4) envisioning in detail the opportunity for using systems thinking in threat modeling teaching and assessment.

References Ross, R., McEvilley, M., & Oren, J. C. (2018). Systems security engineering: Considerations for a multidisciplinary approach in the engineering of trustworthy secure systems, volume 1 (NIST SP 800-160v1; p. NIST SP 800-160v1). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-160v1 Soares Cruzes, D., Gilje Jaatun, M., Bernsmed, K., & Tøndel, I. A. (2018). Challenges and Experiences with Applying Microsoft Threat Modeling in Agile Development Projects. 2018 25th Australasian Software Engineering Conference (ASWEC), 111–120. https://doi.org/10.1109/ASWEC.2018.00023 Tisdale, S. M. (2015). Cybersecurity: Challenges from a systems, complexity, knowledge management and business intelligence perspective. Issues In Information Systems, 16(III), 191–198. https://doi.org/10.48009/3_iis_2015_191-198 Xiong, W., & Lagerström, R. (2019). Threat modeling – A systematic literature review. Computers & Security, 84, 53–69. https://doi.org/10.1016/j.cose.2019.03.010 Yan, D. (2020). A systems thinking for cybersecurity modeling. arXiv Preprint arXiv:2001.05734. Young, W., & Leveson, N. (2013). Systems thinking for safety and security. Proceedings of the 29th Annual Computer Security Applications Conference, 1–8. https://doi.org/10.1145/2523649.2530277

Citation
Format

Joshi, S. S., & Mukherjee, P., & Davis, K. A., & Davis, J. C. (2024, June), Introducing systems thinking as a framework for teaching and assessing threat modeling competency Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/47684

TY  - CPAPER
AB  - Computing systems face diverse and substantial cybersecurity threats. Software engineers can mitigate some of these threats through appropriate software design and analysis, provided they are trained in appropriate competencies. One fundamental cybersecurity competency is threat modeling (Xiong &amp;amp; Lagerström, 2019), which is a systematic approach to identifying, mapping, and mitigating design-level security problems (Soares Cruzes et al., 2018). There are many frameworks for teaching threat modeling, but our analysis of these frameworks and existing coursework suggests that (1) these approaches tend to be focused on component-level analysis rather than educating students to reason holistically about a system’s cybersecurity, and (2) there is no rubric for assessing a student’s threat modeling competency.

To address these concerns, we propose systems thinking as a framework for teaching and assessing threat modeling competency. Prior studies by Young &amp;amp; Leveson (2013) and Yan (2020) suggest systems thinking can be a suitable approach for understanding and mitigating cybersecurity threats. Further, Tisdale (2015) synthesizes literature to argue that a holistic approach like systems thinking is needed to address cybersecurity risks. The purpose of this work-in-progress study is therefore to develop and pilot a rubric that uses systems thinking as a way to assess the threat modeling approach of computer engineering students. Based on our findings, we also discuss how systems thinking could be integrated into the teaching of threat modeling.

To conduct this study, we are developing a novel rubric for assessing threat modeling competency based on systems thinking (e.g., System Engineering approach (Ross et al., 2018)). We will use this rubric to assess threat models created during upper-level software design projects at a large midwestern university in the USA (24 student teams in Fall 2021 and 37 student teams in Spring 2023). We will compare these scores to the baseline rubric used in the course, which was derived directly from the industry standard STRIDE threat modeling framework. Our work will contribute by helping educators understand: (1) trends in threat modeling approaches undertaken by students; (2) identifying blindspots in their threat modeling approach; (3) describing a new rubric for assessing threat modeling based on systems thinking; and (4) envisioning in detail the opportunity for using systems thinking in threat modeling teaching and assessment.

References
Ross, R., McEvilley, M., &amp;amp; Oren, J. C. (2018). Systems security engineering: Considerations for a multidisciplinary approach in the engineering of trustworthy secure systems, volume 1 (NIST SP 800-160v1; p. NIST SP 800-160v1). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-160v1
Soares Cruzes, D., Gilje Jaatun, M., Bernsmed, K., &amp;amp; Tøndel, I. A. (2018). Challenges and Experiences with Applying Microsoft Threat Modeling in Agile Development Projects. 2018 25th Australasian Software Engineering Conference (ASWEC), 111–120. https://doi.org/10.1109/ASWEC.2018.00023
Tisdale, S. M. (2015). Cybersecurity: Challenges from a systems, complexity, knowledge management and business intelligence perspective. Issues In Information Systems, 16(III), 191–198. https://doi.org/10.48009/3_iis_2015_191-198
Xiong, W., &amp;amp; Lagerström, R. (2019). Threat modeling – A systematic literature review. Computers &amp;amp; Security, 84, 53–69. https://doi.org/10.1016/j.cose.2019.03.010
Yan, D. (2020). A systems thinking for cybersecurity modeling. arXiv Preprint arXiv:2001.05734.
Young, W., &amp;amp; Leveson, N. (2013). Systems thinking for safety and security. Proceedings of the 29th Annual Computer Security Applications Conference, 1–8. https://doi.org/10.1145/2523649.2530277


AU  - Siddhant Sanjay Joshi
AU  - Preeti Mukherjee
AU  - Kirsten A. Davis
AU  - James C Davis
CY  - Portland, Oregon
DA  - 2024/06/23
PB  - ASEE Conferences
TI  - Introducing systems thinking as a framework for teaching and assessing threat modeling competency
UR  - https://peer.asee.org/47684
ER  -