Asee peer logo

Did It Work? Analysis Of Ways To Measure The Impact Of An After School Robotics Outreach Program.

Download Paper |


2009 Annual Conference & Exposition


Austin, Texas

Publication Date

June 14, 2009

Start Date

June 14, 2009

End Date

June 17, 2009



Conference Session

Assessment of K-12 Engineering Programs and Issues

Tagged Division

K-12 & Pre-College Engineering

Page Count


Page Numbers

14.488.1 - 14.488.9



Permanent URL

Download Count


Request a correction

Paper Authors


Austin Talley University of Texas, Austin

visit author page

AUSTIN TALLEY is a graduate student in the Mechanical Engineering Department at The University of Texas at Austin. His research focus is in design methodology and engineering education. He received his B.S. from Texas A&M University and M.S.E. from The University of Texas at Austin.

visit author page


Marilyn Fowler Austin Children's Museum

visit author page

MARILYN L. FOWLER is a science educator with over 35 years experience with schoolchildren. Her years in education include classroom teaching at the elementary and college levels, science curriculum specialist, consultant, science coach, and currently, as a teacher of robotics and science. In 1991 she completed her Ph.D. research in problem solving strategies that led to adolescent girls' success on a mechanical design problem, and that interest led her to collaborate with engineers at UT Austin to create the award-winning DTEACh teacher education program (1995 Fred Merryfield Award, ASEE), which continues today. She leads afterschool classes in LEGO Mindstorms Robotics for the Austin Children's Museum in Austin-area schools.

visit author page


Christina Soontornvat Austin Children's Museum

visit author page

CHRISTINA SOONTORNVAT obtained her BS in Mechanical Engineering from Trinity University in 2002 and her MS in Science Education from the University of Texas at Austin in 2007. She has worked at the Austin Children’s Museum since 2006 and now serves as the Science Content Developer. She has taught ACM’s robotics after-school programs and currently coordinates the Museum’s outreach classes in local elementary schools. Christina’s other duties at the Museum include developing the science content for exhibits and programs, and assisting in the planning for the new Museum. Contact:

visit author page


Kathy Schmidt University of Texas, Austin

visit author page

KATHY J. SCHMIDT is the Director of the Faculty Innovation Center for the College of Engineering at the University of Texas at Austin. In this position, she promotes the College of
Engineering's commitment to finding ways to enrich teaching and learning. She works in all aspects of education including design and development, faculty training, learner support, and evaluation. Contact

visit author page

Download Paper |

NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

Did it Work? - Analysis of Ways to Measure the Impact of an Afterschool Robotics Outreach Program

1. Abstract

All over the nation afterschool programs are implemented to assist children. This paper focuses on the challenge of evaluating the outcomes of an afterschool robotics program that is an outreach program of the Austin Children’s Museum. Students use LEGO Mindstorms to explore robotics. The program works with third through fifth grade students at inner-city elementary schools in weekly afterschool sessions for eight weeks at each school. The goals of the program include: enhanced academic skills such as graphing and visual discrimination; increased interest and motivation for future experiences and positive attitudes towards mathematics, science and technology; and greater awareness of technology/teamwork processes such as brainstorming, planning teamwork and troubleshooting. This paper presents the effort to assess the impact the program has on the students in the area of interest. The goals of the program are outside the scope of traditional classroom goals and therefore require assessment different from standardized tests. The assessment is brief due to the short program (eight sessions), the age of the students (third through fifth grade), and the voluntary nature of the program. It was essential to minimize time assessment to maximize the children’s experience. Due to these young students’ written comprehension level, traditional assessment techniques were not appropriate. These challenges led to an assessment that includes a short pre and post written quiz of the students’ comprehension of content areas, pre and post verbal interview of the students, and in-session records of students’ abilities to demonstrate understanding of the content discussed. This paper discusses the success and shortcoming of the different ways of implementing the assessment. The analysis focuses on the ability of the assessment to measure a change in students’ skills or attitudes. The analysis also discusses how each of the assessment techniques impacted the program and provides insight in an area that many programs find challenging.

2. Robotic Background

The materials used for the after-school robotics program were the LEGO® MINDSTORMS® NXT Robotics system. The LEGO® MINDSTORMS® Robotics allows an individual to create autonomous electromechanical inventions that are based on the LEGO building systems. The system features an automatic programmable control unit (NXT brick) that has four inputs and three outputs. Outputs for the NXT brick are motors and lamps (lights). Inputs for the NXT brick are light, sound, rotation, distance, touch and other custom sensors. The robots are built from LEGO Technic components and other craft materials. Programs to control the robots are written on computers with the NXT software and then transferred to the robots. An example of an NXT robotics project is shown in Figure 1.

Talley, A., & Fowler, M., & Soontornvat, C., & Schmidt, K. (2009, June), Did It Work? Analysis Of Ways To Measure The Impact Of An After School Robotics Outreach Program. Paper presented at 2009 Annual Conference & Exposition, Austin, Texas. 10.18260/1-2--5238

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2009 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015