Fundamental: Optimizing a Teacher Professional Development Program for Teaching STEM with Robotics Through Design-based Research

S.M. Mizanoor Rahman; Veena Jayasree Krishnan; Vikram Kapila

Download Paper | Permalink

Conference: 2018 ASEE Annual Conference & Exposition
Location: Salt Lake City, Utah
Publication Date: June 23, 2018
Start Date: June 23, 2018
End Date: July 27, 2018
Conference Session: Robotics
Tagged Division: Pre-College Engineering Education
Tagged Topic: Diversity
Page Count: 20
DOI: 10.18260/1-2--30551
Permanent URL: https://sftp.asee.org/30551
Download Count: 618

Request a correction

Paper Authors

biography

S.M. Mizanoor Rahman New York University

visit author page

Mizanoor Rahman received Ph.D. degree in Mechanical Engineering from Mie University at Tsu, Japan. He then worked as a research fellow at the National University of Singapore (NUS), a researcher at Vrije University of Brussels (Belgium) and a postdoctoral associate at Clemson University, USA. He is currently working as a postdoctoral associate at the Mechanical and Aerospace Engineering Department, Tandon School of Engineering, New York University (NYU), NY, USA. His research and teaching interests include robotics, mechatronics, control systems, electro-mechanical design, human factors/ergonomics, engineering psychology, virtual reality, artificial intelligence, computer vision, biomimetics and biomechanics with applications to industrial manipulation and manufacturing, healthcare and rehabilitation, social services, unmanned autonomous vehicle (aerial and ground, indoor and outdoor) systems and STEM education.

visit author page

biography

Veena Jayasree Krishnan New York University

visit author page

Veena Jayasree Krishnan received a Master of Technology (M. Tech.) degree in Mechatronics from Vellore Institute of Technology, Vellore, India in 2012. She has two years of research experience at the Indian Institute of Science, Bangalore, India. She is currently pursuing Ph.D. in Mechanical Engineering at NYU Tandon School of Engineering. She is serving as a research assistant under an NSF-funded DR K-12 research project to promote integration of robotics in middle school science and math education. For her doctoral research, she conducts mechatronics and robotics research in the Mechatronics, Controls, and Robotics Laboratory at NYU.

visit author page

biography

Vikram Kapila New York University orcid.org/0000-0001-5994-256X

visit author page

Vikram Kapila is a Professor of Mechanical Engineering at NYU Tandon School of Engineering (NYU Tandon), where he directs a Mechatronics, Controls, and Robotics Laboratory, a Research Experience for Teachers Site in Mechatronics and Entrepreneurship, a DR K-12 research project, and an ITEST research project, all funded by NSF. He has held visiting positions with the Air Force Research Laboratories in Dayton, OH. His research interests include K-12 STEM education, mechatronics, robotics, and control system technology. Under a Research Experience for Teachers Site, a DR K-12 project, and GK-12 Fellows programs, funded by NSF, and the Central Brooklyn STEM Initiative (CBSI), funded by six philanthropic foundations, he has conducted significant K-12 education, training, mentoring, and outreach activities to integrate engineering concepts in science classrooms and labs of dozens of New York City public schools. He received NYU Tandon’s 2002, 2008, 2011, and 2014 Jacobs Excellence in Education Award, 2002 Jacobs Innovation Grant, 2003 Distinguished Teacher Award, and 2012 Inaugural Distinguished Award for Excellence in the category Inspiration through Leadership. Moreover, he is a recipient of 2014-2015 University Distinguished Teaching Award at NYU. His scholarly activities have included 3 edited books, 9 chapters in edited books, 1 book review, 61 journal articles, and 140 conference papers. He has mentored 1 B.S., 26 M.S., and 5 Ph.D. thesis students; 47 undergraduate research students and 11 undergraduate senior design project teams; over 480 K-12 teachers and 115 high school student researchers; and 18 undergraduate GK-12 Fellows and 59 graduate GK-12 Fellows. Moreover, he directs K-12 education, training, mentoring, and outreach programs that enrich the STEM education of over 1,000 students annually.

visit author page

Download Paper | Permalink

Abstract

This paper illustrates the use of periodic program evaluation and feedback to measure and optimize the design and implementation performance of a professional development (PD) program for in-service teachers on teaching STEM with robotics through design-based research (DBR). We recruited 23 in-service middle school science and math teachers and designed a three-week, eight-hrs. per day, PD program for participants to learn how to develop robotics-aided lessons and how to implement the developed lessons in a classroom environment. Under the iterative refinement framework of DBR, we divided the three-week PD program into three successive iterations, i.e., each week as an iteration.

We selected to express the quality (performance level) of the design and implementation of the program using several key evaluation criteria (key performance indicators or KPIs), e.g., quality of instruction by the instructors, quality of instruction materials, selection of appropriate mode/method of instructions, scheduling and time management, addressing equity, diversity and individual needs, balance in delivery of contents (appropriate combination of math, science, education, and robotics topics), maintaining pre-requisites, quality of reflections and addressing feedbacks from participants, quality of surveys and assessments, team building and teaming, proper guideline toward innovation in new lesson planning, and level of engagement of the teachers. We developed an objective function based on the above criteria and assigned weight to each criterion. We used two separate Likert scales with scores between 1 and 10 to determine the weights and assess the performance level of each criterion through teacher-surveys. We conducted the surveys at the end of each week (iteration), determined the objective function value (OFV), analyzed the outcomes, and took necessary actions to enhance the OFV in the next iteration(s). The maximum possible OFV (OFVmax) for the selected KPIs, weights, and scales is 1,200. We assume that the OFVmax is the optimal or targeted performance level, and out of the 3 weeks, the week with the highest OFV shows the achieved local optimum performance level. We assume that the constraints are the same/similar in each week. We adopt the following two hypotheses.

H1: The DBR process causes the OFVs to increase (i.e., with successive refinements, the teachers perceive the program to improve) toward the optimal value as the iteration propagates.

H2: The weights of the performance criteria as perceived and evaluated by the teachers vary between iterations.

The results show that, guided by the average scores of weights and scores assigned by teachers to KPIs, the DBR refinements yield the OFVs for the first, second, and third iterations as 803, 859, and 897, respectively, thus validating the first hypothesis. To examine the second hypothesis, variations in weights perceived by the teachers between the iterations are analyzed using paired t-tests. Variations in perceived weights in each iteration are also analyzed.

The proposed approach is novel in that it acts as an assessment tool that may bring objectivity in the design and implementation of PD programs. The DBR can act as a continuous improvement tool to enhance the quality of the PD program iteratively. Both approaches can jointly enhance the effectiveness and performance of the PD program. The proposed integrated (quantitative and qualitative evaluations and DBR) approach can be used to enhance the effectiveness of other PD and educational programs and courses.

Citation
Format

Rahman, S. M., & Jayasree Krishnan, V., & Kapila, V. (2018, June), Fundamental: Optimizing a Teacher Professional Development Program for Teaching STEM with Robotics Through Design-based Research Paper presented at 2018 ASEE Annual Conference & Exposition , Salt Lake City, Utah. 10.18260/1-2--30551

TY - CPAPER
AB - This paper illustrates the use of periodic program evaluation and feedback to measure and optimize the design and implementation performance of a professional development (PD) program for in-service teachers on teaching STEM with robotics through design-based research (DBR). We recruited 23 in-service middle school science and math teachers and designed a three-week, eight-hrs. per day, PD program for participants to learn how to develop robotics-aided lessons and how to implement the developed lessons in a classroom environment. Under the iterative refinement framework of DBR, we divided the three-week PD program into three successive iterations, i.e., each week as an iteration.

We selected to express the quality (performance level) of the design and implementation of the program using several key evaluation criteria (key performance indicators or KPIs), e.g., quality of instruction by the instructors, quality of instruction materials, selection of appropriate mode/method of instructions, scheduling and time management, addressing equity, diversity and individual needs, balance in delivery of contents (appropriate combination of math, science, education, and robotics topics), maintaining pre-requisites, quality of reflections and addressing feedbacks from participants, quality of surveys and assessments, team building and teaming, proper guideline toward innovation in new lesson planning, and level of engagement of the teachers. We developed an objective function based on the above criteria and assigned weight to each criterion. We used two separate Likert scales with scores between 1 and 10 to determine the weights and assess the performance level of each criterion through teacher-surveys. We conducted the surveys at the end of each week (iteration), determined the objective function value (OFV), analyzed the outcomes, and took necessary actions to enhance the OFV in the next iteration(s). The maximum possible OFV (OFVmax) for the selected KPIs, weights, and scales is 1,200. We assume that the OFVmax is the optimal or targeted performance level, and out of the 3 weeks, the week with the highest OFV shows the achieved local optimum performance level. We assume that the constraints are the same/similar in each week. We adopt the following two hypotheses.

H1: The DBR process causes the OFVs to increase (i.e., with successive refinements, the teachers perceive the program to improve) toward the optimal value as the iteration propagates.

H2: The weights of the performance criteria as perceived and evaluated by the teachers vary between iterations.

The results show that, guided by the average scores of weights and scores assigned by teachers to KPIs, the DBR refinements yield the OFVs for the first, second, and third iterations as 803, 859, and 897, respectively, thus validating the first hypothesis. To examine the second hypothesis, variations in weights perceived by the teachers between the iterations are analyzed using paired t-tests. Variations in perceived weights in each iteration are also analyzed.

The proposed approach is novel in that it acts as an assessment tool that may bring objectivity in the design and implementation of PD programs. The DBR can act as a continuous improvement tool to enhance the quality of the PD program iteratively. Both approaches can jointly enhance the effectiveness and performance of the PD program. The proposed integrated (quantitative and qualitative evaluations and DBR) approach can be used to enhance the effectiveness of other PD and educational programs and courses.
AU - S.M. Mizanoor Rahman
AU - Veena Jayasree Krishnan
AU - Vikram Kapila
CY - Salt Lake City, Utah
DA - 2018/06/23
PB - ASEE Conferences
TI - Fundamental: Optimizing a Teacher Professional Development Program for Teaching STEM with Robotics Through Design-based Research
UR - https://sftp.asee.org/30551
DO - 10.18260/1-2--30551
ER -