Asee peer logo

Make Your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation

Download Paper |

Conference

2020 ASEE Virtual Annual Conference Content Access

Location

Virtual On line

Publication Date

June 22, 2020

Start Date

June 22, 2020

End Date

June 26, 2021

Conference Session

Computing and Information Technology Division Technical Session 5

Tagged Division

Computing and Information Technology

Tagged Topic

Diversity

Page Count

20

DOI

10.18260/1-2--34937

Permanent URL

https://peer.asee.org/34937

Download Count

384

Request a correction

Paper Authors

biography

Bin Cong California State University at Fullerton

visit author page

Dr. Bin Cong received his PhD degree in Computer Science from the University of Texas, Dallas. He joined California State University Fullerton in 1998 where he is currently a Professor of Computer Science. He is one of the founding members of the Master of Software Engineering program at CSUF and served as the Coordinator during its first cohort. Currently, he serves as the Chair of Assessment and Improvement Committee in the Computer Science Department. He is a Certified CMMI High Maturity Lead and Instructor. His current research interests include software process, quality control, Agile and Lean development and CMMI based process assessment and improvement.

In 2011 Dr. Cong was named as Distinguished Faculty Member from College of ECS at CSUF.

visit author page

biography

Christopher Ryu California State University at Fullerton

visit author page

Dr. Christopher Ryu is Professor and Chair in the Computer Science Department at California State University, Fullerton (CSUF). He received his B.S degree from Inha University in Korea and PhD degree from the University of Houston, all in Computer Science. His research and teaching interests involve Machine learning, Artificial intelligence, Data science, Computational finance, Software design and architecture. Prior to join CSUF, he worked as a Software Engineer with EDS and Volts Group, Houston, TX for the oil and gas utility management system, during 1997-1999. Dr. Ryu is a 2007 recipient of the Outstanding Teacher and Scholar Award from California State University, Fullerton.

visit author page

biography

Raman Menon Unnikrishnan California State University at Fullerton

visit author page

Dr. Raman Menon ("Unni") Unnikrishnan is a Professor of Electrical Engineering and Computer Engineering at California State University Fullerton. He is also a Distinguished Visiting Professor of Engineering at Mukesh Patel School of Technology Management and Engineering of NMIMS in Mumbai, India. He received his BS degree from the University of Kerala, India, MS degree from South Dakota State University and PhD from the University of Missouri-Columbia, all in electrical engineering.
During 2001-2016 he was the Dean of the College of Engineering and Computer Science at California State University, Fullerton. Prior to that he was the Head of the Electrical Engineering Department at RIT in Rochester, NY. Fullerton Chamber of Commerce recognized him in 2015 as the “Educator of the Year.” In 2016 he received ASEE’s “Distinguished Educator Award” from the ECE Division. Dr. Unnikrishnan was a member of the Accreditation Committee for American Society of Engineering Education (ASEE). He was a Commissioner of the Engineering Accreditation Commission of ABET during 2008-13 and chaired the Committee for Accreditation Activities (CEAA) for IEEE during 2016-18. He continues to be a program evaluator for ASEE and IEEE.
Dr. Unnikrishnan is a Fellow of IEEE and a Life Member of ASEE.

visit author page

Download Paper |

Abstract

Make your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation

Abstract Designing a proper metric framework to support assessment of student outcomes is always a challenge. The challenge is even more pronounced in large computer science programs where many required courses have multiple sections and many of these sections are staffed by adjunct faculty. Furthermore, the culture of attainment of student outcomes and its connection to continuous improvement is still evolving in computer science. The issues faced include the following items.  The Data collected cannot be readily traced or connected to Student Outcomes (SOs).  Consistency of data cannot be maintained when multiple sections are offered and taught by different instructors.  The time consuming data collection, analysis, and reporting could evolve into an accreditation game and not a process focused on continuous improvement.  Lack of institutionalization results in many ad hoc attempts with little sustainability over long periods of time.  Changes in accreditation criteria, however small, require reformatting the entire process.  Disconnect between the vocabulary used in industry (affecting part-time faculty) and that used by accreditation professionals. In this paper, we will show how basic principles of quality assurance that are integral parts of software development and software engineering are adopted to the assessment and continuous improvement system in ABET accreditation. Using such an approach, the Computer Science program at our University has been able to establish a routine and value-outcome driven process to support continuous improvement in tracking Student Outcomes. We hope that our results and experience will be of interest to similar computer science or engineering programs nationwide. The paper will include four parts: 1. Establishment and maintenance of a traceable (Student Outcomes – Course Outcomes – Performance Indicators) data framework. 2. Building a one-stop for all platform that incorporates automated data collection and presentation. 3. Institutionalization of a data-driven PDCA (Plan – Do – Check – Act) improvement cycle. 4. Illustrative examples to demonstrate how PI data is used to identify, improve and validate a curriculum improvement initiative. GQIMP (Goal–Question–Indicator–Measure–Process) is a common practice used to establish a metric system for software quality and process improvement. It provides a methodology to build data support system for process driven improvement. Compliance with ABET guidelines is an outcomes driven process where the curriculum improvement is integral to the attainment of SOs. We deployed a similar concept and built a SOOP (Student Outcome–course Outcome–Performance Indicator) framework where carefully chosen and well-understood Performance Indicators are the basis for data collection and analysis. As is the case with all software systems, the actual data collection and validation is tedious and time consuming. It could place a big burden on instructors and it could generate negative feeling toward the accreditation process in general. The burden was ameliorated by deploying the SOOP framework since it provides a good foundation to automate the process. We have developed a system that automates data submission, data summarization, and data presentation. Our system significantly reduces the effort required. It also provides support to let data submitters to see the results. The hardest part of deploying data driven improvement is institutionalization. The challenges were many. How do we make it routine part of the academic culture? How to make it sustainable and robust enough to withstand personnel changes at the department or college levels? How to make faculty the owners of the courses, their PIs, and their improvement even though the individual courses must contribute in very specific ways towards the attainment of SOs. We have found that CMMI (Capability Maturity Model Integration) provides a great set of practices to help institutionalize a process. It has the following 12 practices: 1. GP 2.1 Establish an Organizational Policy 2. GP 2.2 Plan the Process 3. GP 2.3 Provide Resources 4. GP 2.4 Assign Responsibility 5. GP 2.5 Train People 6. GP 2.6 Control Work Products 7. GP 2.7 Identify and Involve Relevant Stakeholders 8. GP 2.8 Monitor and Control the Process 9. GP 2.9 Objectively Evaluate Adherence 10. GP 2.10 Review Status with Higher Level Management 11. GP 3.1 Establish a Defined Process 12. GP 3.2 Collect Process Related Experiences

The paper will describe how we implemented the 12 practices to nurture an improvement culture and make ABET work routine and value driven. Finally, we will demonstrate specific examples such as how the PDCA worked effectively under SOOP in standardizing a three-course sequence: CPSC 120 Introduction to Programming, CPSC121 Object-Oriented Programming and CPSC131 Data Structures.

Cong, B., & Ryu, C., & Unnikrishnan, R. M. (2020, June), Make Your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--34937

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2020 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015