Cong, B., & Ryu, C., & Unnikrishnan, R. M. (2020, June), Make Your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation  Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--34937

\bibitem{asee_peer_34937}
  Cong, B., \& Ryu, C., \& Unnikrishnan, R. M. (2020, June), \emph{Make Your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation}
  Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line .
  10.18260/1-2--34937

Bin Cong, Christopher Ryu, and Raman Menon Unnikrishnan.  "Make Your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation".  2020 ASEE Virtual Annual Conference Content Access, Virtual On line , 2020, June.  ASEE Conferences, 2020.  https://peer.asee.org/34937 Internet.  13 Dec, 2024

\bibitem{asee_peer_34937}
  Bin Cong, Christopher Ryu, and Raman Menon Unnikrishnan.
  "Make Your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation".
  \emph{2020 ASEE Virtual Annual Conference Content Access, Virtual On line , 2020, June}.
  ASEE Conferences, 2020.
  https://peer.asee.org/34937 Internet.  13 Dec, 2024

@INPROCEEDINGS{asee_peer_34937
author = "Bin Cong, Christopher Ryu, and Raman Menon Unnikrishnan"
title = "Make Your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation"
booktitle = "2020 ASEE Virtual Annual Conference Content Access"
year = "2020"
month = "June"
address = "Virtual On line "
publisher = "ASEE Conferences"
note = {https://peer.asee.org/34937}
number = {10.18260/1-2--34937}
}

TY  - CPAPER
AB  - Make your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation

Abstract
Designing a proper metric framework to support assessment of student outcomes is always a challenge. The challenge is even more pronounced in large computer science programs where many required courses have multiple sections and many of these sections are staffed by adjunct faculty. Furthermore, the culture of attainment of student outcomes and its connection to continuous improvement is still evolving in computer science. The issues faced include the following items.
	The Data collected cannot be readily traced or connected to Student Outcomes (SOs).
	Consistency of data cannot be maintained when multiple sections are offered and taught by different instructors.
	The time consuming data collection, analysis, and reporting could evolve into an accreditation game and not a process focused on continuous improvement. 
	Lack of institutionalization results in many ad hoc attempts with little sustainability over long periods of time.
	Changes in accreditation criteria, however small, require reformatting the entire process.
	Disconnect between the vocabulary used in industry (affecting part-time faculty) and that used by accreditation professionals.
In this paper, we will show how basic principles of quality assurance that are integral parts of software development and software engineering are adopted to the assessment and continuous improvement system in ABET accreditation. Using such an approach, the Computer Science program at our University has been able to establish a routine and value-outcome driven process to support continuous improvement in tracking Student Outcomes. We hope that our results and experience will be of interest to similar computer science or engineering programs nationwide.
The paper will include four parts:
1.	Establishment and maintenance of a traceable (Student Outcomes – Course Outcomes – Performance Indicators) data framework.
2.	Building a one-stop for all platform that incorporates automated data collection and presentation.
3.	Institutionalization of a data-driven PDCA (Plan – Do – Check – Act) improvement cycle.
4.	Illustrative examples to demonstrate how PI data is used to identify, improve and validate a curriculum improvement initiative.
GQIMP (Goal–Question–Indicator–Measure–Process) is a common practice used to establish a metric system for software quality and process improvement. It provides a methodology to build data support system for process driven improvement. Compliance with ABET guidelines is an outcomes driven process where the curriculum improvement is integral to the attainment of SOs. We deployed a similar concept and built a SOOP (Student Outcome–course Outcome–Performance Indicator) framework where carefully chosen and well-understood Performance Indicators are the basis for data collection and analysis.
As is the case with all software systems, the actual data collection and validation is tedious and time consuming. It could place a big burden on instructors and it could generate negative feeling toward the accreditation process in general. The burden was ameliorated by deploying the SOOP framework since it provides a good foundation to automate the process. We have developed a system that automates data submission, data summarization, and data presentation. Our system significantly reduces the effort required. It also provides support to let data submitters to see the results.
The hardest part of deploying data driven improvement is institutionalization. The challenges were many. How do we make it routine part of the academic culture? How to make it sustainable and robust enough to withstand personnel changes at the department or college levels? How to make faculty the owners of the courses, their PIs, and their improvement even though the individual courses must contribute in very specific ways towards the attainment of SOs. 
We have found that CMMI (Capability Maturity Model Integration) provides a great set of practices to help institutionalize a process. It has the following 12 practices:
1.	GP 2.1 Establish an Organizational Policy
2.	GP 2.2 Plan the Process
3.	GP 2.3 Provide Resources
4.	GP 2.4 Assign Responsibility
5.	GP 2.5 Train People
6.	GP 2.6 Control Work Products
7.	GP 2.7 Identify and Involve Relevant Stakeholders
8.	GP 2.8 Monitor and Control the Process
9.	GP 2.9 Objectively Evaluate Adherence
10.	GP 2.10 Review Status with Higher Level Management
11.	GP 3.1 Establish a Defined Process
12.	GP 3.2 Collect Process Related Experiences

The paper will describe how we implemented the 12 practices to nurture an improvement culture and make ABET work routine and value driven. 
Finally, we will demonstrate specific examples such as how the PDCA worked effectively under SOOP in standardizing a three-course sequence: CPSC 120 Introduction to Programming, CPSC121 Object-Oriented Programming and CPSC131 Data Structures. 

AU  - Bin Cong
AU  - Christopher Ryu
AU  - Raman Menon Unnikrishnan
CY  - Virtual On line 
DA  - 2020/06/22
PB  - ASEE Conferences
TI  - Make Your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation
UR  - https://peer.asee.org/34937
DO  - 10.18260/1-2--34937
ER  -