Asee peer logo

Software For Measuring The Intellectual Development Of Students : Advantages And Limitations

Download Paper |


2002 Annual Conference


Montreal, Canada

Publication Date

June 16, 2002

Start Date

June 16, 2002

End Date

June 19, 2002



Conference Session

Learning Styles

Page Count


Page Numbers

7.1007.1 - 7.1007.18



Permanent URL

Download Count


Request a correction

Paper Authors

author page

Michael Pavelich

author page

Barbara Olds

author page

Ronald Miller

Download Paper |

NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

Main Menu


Michael J. Pavelich, Ronald L. Miller and Barbara M. Olds

Colorado School of Mines


Most methods currently available to measure intellectual development in college students are either marginally reliable or are expensive and time consuming. In an attempt to circumvent these difficulties, we have developed Cogito ã, a software package which uses a neural network to find patterns in "noisy" paper-and-pencil data and relate them to the Perry or Reflective Judgment models of intellectual development. The project was supported by a grant from FIPSE. We will report the results of testing this software on 88 students and faculty from two colleges and students from a high school. Data from standard Reflective Judgment interviews and from Cogitoã have been analyzed in a variety of ways using neural-net software. The better fits show correlation coefficients between Cogito ã and interview ratings of 0.5-0.8. Most other fits show correlations below 0.4. These results are slightly to significantly better than previous paper-and- pencil instruments for measuring intellectual development. We will discuss what our results mean for effective assessment. Are R values in the 0.5-0.6 range good enough? Why is there an apparent ceiling on R values for paper-and-pencil instruments?

Keywords ¾ intellectual development, assessment, neural network, Cogito ©

Introduction and Objective

Most engineering programs expect that their students will develop intellectually in addition to acquiring knowledge and skills in a specific engineering discipline. However, nearly all measures of student achievement are focused on content knowledge, process ability (e.g. design), or communication skills; students are assumed to be developing intellectually, especially in their ability to think critically, but rarely are meaningful data collected and reported which support such an assumption. However, the recent movement towards outcomes assessm ent now requires reliable measures of students’ abilities to make reasoned decisions as they solve complex problems. For example, in the U. S. the Accreditation Board for Engineering and Technology (ABET) requires institutions to develop assessment processes which can demonstrate “that the outcomes important to the mission of the institution and the objectives of the program are being measured”. 1

Perhaps the most recognized and valid method to quantify maturation of college students’ intellectual abilities relies on developmental process models such as Perry’s Model of

Proceedings of the 2002 American Society for Engineering Education Annual Conference & Exposition Copyright Ó 2002, American Society for Engineering Education

Main Menu

Pavelich, M., & Olds, B., & Miller, R. (2002, June), Software For Measuring The Intellectual Development Of Students : Advantages And Limitations Paper presented at 2002 Annual Conference, Montreal, Canada. 10.18260/1-2--11146

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2002 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015