Asee peer logo

Drawing Valid Inferences From The Nested Structure Of Engineering Education Data: Application Of A Hierarchical Linear Model To The Succeed Longitudinal Database

Download Paper |

Conference

2005 Annual Conference

Location

Portland, Oregon

Publication Date

June 12, 2005

Start Date

June 12, 2005

End Date

June 15, 2005

ISSN

2153-5965

Conference Session

Engineering Education Research and Assessment III

Page Count

9

Page Numbers

10.492.1 - 10.492.9

Permanent URL

https://peer.asee.org/14944

Download Count

15

Request a correction

Paper Authors

author page

Miguel A. Padilla

author page

Timothy J. Anderson

author page

Guili Zhang

Download Paper |

Abstract
NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

Drawing Valid Inferences from the Nested Structure of Engineering Education Data: Application of a Hierarchical Linear Model to the SUCCEED Longitudinal Database

Miguel A. Padilla, Guili Zhang, and Timothy J. Anderson Educational Psychology and Chemical Engineering, University of Florida

Matthew W. Ohland, General Engineering, Clemson University

Abstract

Although hierarchical linear models are seldom used in engineering educational research, the nested structure of students in various colleges of engineering and the longitudinal nature of student records supports the use of such models. Hierarchical linear models account for the nested structure and can test hypotheses on both the schools and the students within the schools simultaneously, thereby eliminating aggregation bias and misestimated standard errors that result when the nested structure is ignored. In the present study, a hierarchical linear model is fitted to the SUCCEED longitudinal database using only students that graduated. As an example, cumulative GPA is regressed on Carnegie school classification, school setting, degree received, gender gap, and citizenship gap with SAT total score and number of terms attended as covariates. The results indicate that there is significant cumulative GPA variance between schools, accounting for 19% of the variance. Additionally, the gender gap and citizenship gap accounted for 6% of the within school cumulative GPA variance, but school setting accounted for 61% of the between school citizenship gap variance. In particular, students that receive their degree in engineering had the highest cumulative GPA. Non-citizens tended to have higher cumulative GPAs than citizens. Another finding is total SAT score is more predictive of cumulative GPA in urban schools than suburban schools. Finally, urban and/or research schools had the strongest relationship between number of terms until graduation with cumulative GPA in that longer times to graduation are associated with lower cumulative GPA.

Introduction

The Southeastern University and College Coalition for Engineering EDucation (SUCCEED) compiled a student database to help evaluate the impact of its various experiments in undergraduate engineering education. This comprehensive longitudinal database contains the academic records of all students enrolled in the nine SUCCEED universities during the period 1987 to 2002. The extent of the database in terms of the number of students, length of time, and number of universities enables the exploration of a variety of educational questions with statistical significance. Perhaps the most important use of such an extensive database is to understand the relationship between a specific outcome (e.g. cumulative GPA) on various factors (e.g., preparation – SAT scores, gender, discipline).

For purposes of quantitative analysis and generalizability, it is common to represent this relationship with a mathematical model, with linear models being most common. It is important to realize, however, that the data in the SUCCEED database does not result from an experimental design. That is to say, students were not randomly selected from the population and then

Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright © 2005, American Society for Engineering Education

Padilla, M. A., & Anderson, T. J., & Ohland, M., & Zhang, G. (2005, June), Drawing Valid Inferences From The Nested Structure Of Engineering Education Data: Application Of A Hierarchical Linear Model To The Succeed Longitudinal Database Paper presented at 2005 Annual Conference, Portland, Oregon. https://peer.asee.org/14944

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2005 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015