Evaluation of DEEP POOL on Student Learning Outcomes Attainment

Matthew J. Traum; Emre Selvi; Adele Hanlon

Download Paper | Permalink

Conference: 2019 ASEE Annual Conference & Exposition
Location: Tampa, Florida
Publication Date: June 15, 2019
Start Date: June 15, 2019
End Date: October 19, 2019
Conference Session: Entrepreneurship & Engineering Innovation Division Technical Session 8
Tagged Division: Entrepreneurship & Engineering Innovation
Page Count: 16
DOI: 10.18260/1-2--32763
Permanent URL: https://peer.asee.org/32763
Download Count: 394

Request a correction

Paper Authors

biography

Matthew J. Traum Engineer Inc. orcid.org/0000-0002-1105-0439

visit author page

Dr. Matthew J. Traum is founding CEO at Engineer Inc., a Florida-based STEM education social enterprise start-up. Traum invented @HOLM™ lab kits to enable students in on-line courses to build and run engineering experiments remotely at home.

Before founding Engineer Inc., Dr. Traum was a well-known higher education administrator, fund raiser, educator, and researcher with co-authorship of 12 peer-reviewed research journal articles, 18 refereed research conference articles, and 20 refereed pedagogical conference articles. As a PI or Co-PI, Traum has attracted over $841 K in funding for research, education, and entrepreneurial ventures from multiple sources including NSF, NASA, ASHRAE, AIAA, Sigma Xi, the Texas State Energy Conservation Office, and several industry sponsors including Toshiba and Oshkosh.

Most recently as Associate Professor and Director of Engineering Programs at Philadelphia University, Dr. Traum led the Mechanical Engineering Program through a successful ABET interim visit resulting in no deficiencies, weaknesses, or concerns.

Previously, Dr. Traum was an assistant professor at the Milwaukee School of Engineering (MSOE), one of the top-ten undergraduate-serving engineering universities in the U.S. Dr. Traum coordinated MSOE's first crowd-funded senior design project. He also co-founded with students EASENET, a start-up renewable energy company to commercialize waste-to-energy biomass processors.

Dr. Traum began his academic career as a founding faculty member in the Mechanical & Energy Engineering Department at the University of North Texas - Denton where he established a successful, externally-funded researcher incubator that trained undergraduates to perform experimental research and encouraged matriculation to graduate school.

Traum received a Ph.D. in mechanical engineering from the Massachusetts Institute of Technology where he held a research assistantship at MIT’s Institute for Soldier Nanotechnologies. At MIT he invented a new nano-enabled garment to provide simultaneous ballistic and thermal protection to infantry soldiers. Dr. Traum also holds a master’s degree in mechanical engineering from MIT with a focus on cryogenics and two bachelor’s degrees from the University of California, Irvine: one in mechanical engineering and the second in aerospace engineering. In addition, he attended the University of Bristol, UK as a non-matriculating visiting scholar where he completed an M.Eng thesis in the Department of Aerospace Engineering on low-speed rotorcraft control.

visit author page

biography

Emre Selvi Jacksonville University

visit author page

Emre Selvi is an Assistant Professor of Engineering at Jacksonville University. He received his academic degrees in Mechanical Engineering; B.S. and M.S. from Middle East Technical University and PhD from Texas Tech University. Prior to starting his Ph.D. in 2004, he worked as a Design and Production Engineer for Aselsan Inc. over four years. His research interests are high pressure material science and engineering design, especially as it relate to educational environments.

visit author page

biography

Adele Hanlon Jacksonville University

visit author page

Dr. Adele Hanlon
Associate Professor of Mathematics Education
Jacksonville University
Dr. Hanlon has been in the field of education for 36 years. She earned a Bachelor of Science degree in Chemical Engineering from the University of Tennessee, a Master’s in Education with an emphasis in mathematics from the University of Central Oklahoma and a PhD. in Education with an emphasis in mathematics education from Oklahoma State University. She has taught in K-12, although she has spent the last 30 years in higher education.

visit author page

Download Paper | Permalink

Abstract

This paper evaluates a new pedagogical approach: “Developing Engineering Education Products via Project Ownership Oriented Learning” (DEEP POOL). We hypothesize that student engagement, enthusiasm, and interest in laboratory work increases when labs are structured so student activities support the entrepreneurial development, construction, testing, and commercialization of real products for an engineering company. Increased student excitement and participation should produce student achievement of Learning Outcomes on par with or better than conventional lab courses.

Jacksonville University’s 2017 Fall Mechanics of Materials course was DEEP POOL structured to focus student laboratory time toward creating, fabricating, testing, and analyzing new and novel test sample coupons for the PASCO EX-5515A Materials Stress-Strain Experiment in collaboration with Engineer Inc., an engineering education technology social enterprise state-up. The lab exposed students to topics typically taught in conventional Mechanics of Materials laboratories. However, weekly exercises were also intentionally structured to map students’ efforts directly to development of new commercial products for Engineer Inc.

Student attainment of Learning Outcomes was evaluated to measure DEEP POOL effectiveness. Enrolled students who agreed to participate after informed consent notification (n = 7) were the study population. No additional participant recruiting was done. Participants self-assessed their own Learning Outcome achievement via pre/post surveys. Complementary direct assessment occurred using different exam problems covering similar concepts embedded within assessments given at the beginning and end of class. Course impact was interpreted based on pre/post differences between assessments both indirect and direct. To mitigate small sample size, one-tailed Wilcoxon Signed Rank Testing was applied to both data sets. To provide data visualization, descriptive statistics were also evaluated by comparing pre/post averages of students’ self-reported results.

Indirect assessment showed that exposure to DEEP POOL moved students’ averaged self-reported Learning Outcome achievement up across all outcomes assessed. The largest improvements (with averages increasing 1.71 points on a 4-point scale) were in skills related to ABET (k) and ABET (h). The next largest gain (the average increased 1.29 points on a 4-point scale) was in ABET (b). One-tailed Wilcoxon Signed Rank Test of indirect assessment data corroborate these results. Questions related to ABET (k) (Z = −2.366, p < 0.05), (h) (Z = −2.023, p < 0.05), and (b) (Z = −2.366, p < 0.05) revealed statistical improvement (≥ 95% confidence) in the median values. Questions related to ABET (c) (Z = −2.155, p < 0.05) and (g) (Z = −1.955, p < 0.05) showed no statistical improvement (≥ 95% confidence) in the median values; and questions related to ABET (a), (d), and (e) did not meet the hypothesis testing threshold because the sample size, n, was too small.

One-Tailed Wilcoxon Signed Rank Tests of direct assessment data revealed that one question related to ABET (e) (Z = −1.787, p < 0.05) showed no statistical improvement (≥ 95% confidence) in the medians. Other questions linked to ABET (a), (b), (c), (e), (h), and (k) did not meet the hypothesis testing threshold because n was too small. Critically however, direct assessment of ABET (c) (Z = −2.023, p < 0.05) “an ability to design a system, component, or process to meet desired needs within realistic constraints” did reveal statistical improvement (≥ 95% confidence) in the medians for student Learning Outcome achievement. This skill set is strongly associated with entrepreneurial new product development central to DEEP POOL.

In summary, this study revealed indirect/direct assessment evidence of improvement in three/one ABET-linked Learning Outcomes respectively. This result, along with the preponderance of metrics that did not meet the threshold for evaluation, motivate need for a follow-on study comparing the learning effectiveness of DEEP POOL versus a control, a conventionally-taught laboratory course. Should DEEP POOL’s effectiveness prove equal to or better than its conventional analog, this novel laboratory pedagogy can emerge as a powerful way to conduct entrepreneurial new product development activities in engineering laboratories in partnership with industry.

Citation
Format

Traum, M. J., & Selvi, E., & Hanlon, A. (2019, June), Evaluation of DEEP POOL on Student Learning Outcomes Attainment Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--32763

TY  - CPAPER
AB  - This paper evaluates a new pedagogical approach: “Developing Engineering Education Products via Project Ownership Oriented Learning” (DEEP POOL). We hypothesize that student engagement, enthusiasm, and interest in laboratory work increases when labs are structured so student activities support the entrepreneurial development, construction, testing, and commercialization of real products for an engineering company. Increased student excitement and participation should produce student achievement of Learning Outcomes on par with or better than conventional lab courses.

Jacksonville University’s 2017 Fall Mechanics of Materials course was DEEP POOL structured to focus student laboratory time toward creating, fabricating, testing, and analyzing new and novel test sample coupons for the PASCO EX-5515A Materials Stress-Strain Experiment in collaboration with Engineer Inc., an engineering education technology social enterprise state-up. The lab exposed students to topics typically taught in conventional Mechanics of Materials laboratories. However, weekly exercises were also intentionally structured to map students’ efforts directly to development of new commercial products for Engineer Inc.

Student attainment of Learning Outcomes was evaluated to measure DEEP POOL effectiveness. Enrolled students who agreed to participate after informed consent notification (n = 7) were the study population. No additional participant recruiting was done. Participants self-assessed their own Learning Outcome achievement via pre/post surveys. Complementary direct assessment occurred using different exam problems covering similar concepts embedded within assessments given at the beginning and end of class. Course impact was interpreted based on pre/post differences between assessments both indirect and direct. To mitigate small sample size, one-tailed Wilcoxon Signed Rank Testing was applied to both data sets. To provide data visualization, descriptive statistics were also evaluated by comparing pre/post averages of students’ self-reported results.

Indirect assessment showed that exposure to DEEP POOL moved students’ averaged self-reported Learning Outcome achievement up across all outcomes assessed. The largest improvements (with averages increasing 1.71 points on a 4-point scale) were in skills related to ABET (k) and ABET (h). The next largest gain (the average increased 1.29 points on a 4-point scale) was in ABET (b). One-tailed Wilcoxon Signed Rank Test of indirect assessment data corroborate these results. Questions related to ABET (k) (Z = −2.366, p &lt; 0.05), (h) (Z = −2.023, p &lt; 0.05), and (b) (Z = −2.366, p &lt; 0.05) revealed statistical improvement (≥ 95% confidence) in the median values. Questions related to ABET (c) (Z = −2.155, p &lt; 0.05) and (g) (Z = −1.955, p &lt; 0.05) showed no statistical improvement (≥ 95% confidence) in the median values; and questions related to ABET (a), (d), and (e) did not meet the hypothesis testing threshold because the sample size, n, was too small.

One-Tailed Wilcoxon Signed Rank Tests of direct assessment data revealed that one question related to ABET (e) (Z = −1.787, p &lt; 0.05) showed no statistical improvement (≥ 95% confidence) in the medians. Other questions linked to ABET (a), (b), (c), (e), (h), and (k) did not meet the hypothesis testing threshold because n was too small. Critically however, direct assessment of ABET (c) (Z = −2.023, p &lt; 0.05) “an ability to design a system, component, or process to meet desired needs within realistic constraints” did reveal statistical improvement (≥ 95% confidence) in the medians for student Learning Outcome achievement. This skill set is strongly associated with entrepreneurial new product development central to DEEP POOL.

In summary, this study revealed indirect/direct assessment evidence of improvement in three/one ABET-linked Learning Outcomes respectively. This result, along with the preponderance of metrics that did not meet the threshold for evaluation, motivate need for a follow-on study comparing the learning effectiveness of DEEP POOL versus a control, a conventionally-taught laboratory course. Should DEEP POOL’s effectiveness prove equal to or better than its conventional analog, this novel laboratory pedagogy can emerge as a powerful way to conduct entrepreneurial new product development activities in engineering laboratories in partnership with industry.
AU  - Matthew J. Traum
AU  - Emre Selvi
AU  - Adele Hanlon
CY  - Tampa, Florida
DA  - 2019/06/15
PB  - ASEE Conferences
TI  - Evaluation of DEEP POOL on Student Learning Outcomes Attainment
UR  - https://peer.asee.org/32763
DO  - 10.18260/1-2--32763
ER  -