June 28, 1998
June 28, 1998
July 1, 1998
3.324.1 - 3.324.12
Implications of Statistical Process Monitoring for ABET 2000 Program Evaluation: An Example Using Freshman Engineering Attitudes Mary Besterfield-Sacre, Nancy Y. Amaya, Larry J. Shuman, and Cynthia J. Atman University of Texas - El Paso /University of Pittsburghi
ABET’s new criteria, “EC 2000,” has brought assessment and evaluation to the forefront of engineering education. Concomitantly, the focus of ABET’s program evaluation has shifted from “what are you [the program] doing?” to “how is what you’re doing achieving the desired outcomes [what are your students doing]? 1.” In short, accreditation will be concerned as much or more with outcomes rather than inputs or processes. More important, accreditors will be looking at how problems are identified and improvements are made in order to affect the program’s outcomes. Therefore, a comprehensive evaluation system should not only accurately measure student outcomes, through proper instrument development and administration, but also possess a well-designed feedback mechanism that allows for the tracking of these outcomes over time. By appropriately tracking outcomes, engineering educators should be able to identify areas for improvement, as well as monitor the effectiveness of programmatic interventions.
One basis for such a feedback mechanism is provided by process control charts 2. However, when using data that are either categorical or ordinal in nature, such as to closed-form questionnaires or test scores, naively applying traditional X and R charts can yield erroneous conclusions. This paper will discuss a potential role for process control monitoring of student outcomes for program evaluation and improvement. Using example data about freshman engineering student attitudes, we will present two non-parametric control-charting methods that are appropriate for either survey data or data that are known to be non-normal. We will then illustrate how the charts may be used to identify improvement opportunities and track interventions.
Introduction – Use of Quality Control Charts in Engineering Education
EC 2000 3 directs engineering faculty to not only demonstrate that their students have achieved eleven specific outcomes upon graduation, but it also encourages them to continuously improve, in innovative ways, the learning experience. To accomplish this, engineering educators will need evaluation protocols and measurement instruments that will facilitate feedback and the resultant improvements.
Statistical process control (SPC) and control charting, in particular, can be used as a feedback mechanism. Industry has commonly used SPC techniques to assure that production remains “in control” according to pre-determined specifications and process capability. Recently, several authors have proposed applying these concepts to engineering education; i.e., to assure that educational processes and outcomes are in control. See for example 4,5,6. Why would an institution be interested in control charting? As engineering educators become more
i Dr. Besterfield-Sacre and Ms. Amaya are at the University of Texas - El Paso, Drs. Shuman and Atman are at the University of Pittsburgh.
Amaya, N. Y., & Besterfield-Sacre, M. E., & Shuman, L. J., & Atman, C. (1998, June), Implications Of Statistical Process Monitoring For Abet 2000 Program Evaluation: An Example Using Freshman Engineering Attitudes Paper presented at 1998 Annual Conference, Seattle, Washington. https://peer.asee.org/7176
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 1998 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015