Using Online Endless Quizzes as Graded Homework

Gwen Lee-Thomas, Ph.D.; Autar Kaw; Ali Yalcin

Download Paper | Permalink

Conference: 2011 ASEE Annual Conference & Exposition
Location: Vancouver, BC
Publication Date: June 26, 2011
Start Date: June 26, 2011
End Date: June 29, 2011
ISSN: 2153-5965
Conference Session: NSF Grantees Poster Session
Tagged Topic: NSF Grantees
Page Count: 16
Page Numbers: 22.1626.1 - 22.1626.16
DOI: 10.18260/1-2--18967
Permanent URL: https://peer.asee.org/18967
Download Count: 535

Paper Authors

biography

Gwen Lee-Thomas, Ph.D. Old Dominion University orcid.org/0000-0001-9063-2770

visit author page

Dr. Gwen Lee-Thomas is as assistant professor of graduate education at Old Dominion University. She is actively engaged in a wide variety of federal and locally funded evaluation and research activities on STEM related works and has over 12 years of experience. Specifically, her experiences include director of assessment at Rose-Hulman Institute of Technology where she has served as the assessment liaison for a 10-member consortium of the NSF funded Foundation Coalition on the Integrated First Year Experience in Engineering. Dr. Lee-Thomas has been serving as the external evaluator for numerous organizations and universities; panel reviewer for U.S. DOE GAANN Fellowships (2009, 2010), NSF EEP (2005-08), and S-STEM (2008). Her assessment findings and evaluative works are reported in IEEE, presented in ASEE and FIE conference proceedings, and acknowledged in Mixed-Nuts on several different projects. Dr. Lee-Thomas also presented her evaluative work as a key component in an award-winning NPR radio broadcast “Sounds of Progress” on The Women In Science, Technology, Engineering and Mathematics ON THE AIR! as part of a NSF funded project with Norfolk State University's College of Science, Engineering and Technology.

visit author page

biography

Autar Kaw University of South Florida

visit author page

Autar K Kaw is a Professor of Mechanical Engineering and Jerome Krivanek Distinguished Teacher at the University of South Florida. With major funding from National Science Foundation, he is developing award winning online resources for an undergraduate course in Numerical Methods. He is the recipient of the 2004 Council for Advancement and Support of Education (CASE) & the Carnegie Foundation for the Advancement of Teaching (CFAT) Florida Professor of the Year Award.

visit author page

biography

Ali Yalcin University of South Florida

visit author page

Ali Yalcin is an Associate Professor of Industrial and Management Systems Engineering Department at the University of South Florida, and an Associate Faculty member of the Center for Urban Transportation Research. His research interests include modeling, analysis and control of discrete event systems, production planning and control, industrial information systems, data analysis and knowledge discovery, and engineering education research. He has taught courses in the areas of systems modeling and analysis, information systems design, production planning, facilities design, and systems simulation. He also co-authored the 2006 Joint Publishers Book-of-the-Year textbook, Design of Industrial Information Systems, Elsevier.
Address: Department of Industrial and Management Systems Engineering, University of South Florida, 4202 E Fowler Ave ENB118, Tampa FL 33620-5350; telephone: (+1) 813-974-5590; e-mail: ayalcin@usf.edu

visit author page

Download Paper | Permalink

Abstract

Using Online Endless Quizzes as Graded HomeworkIn an effort to find the best use of limited teaching assistant’s time in today’s economy , astudy reported at last year’s ASEE 2010 conference by two of the authors of this paperindicated that there is no statistically significant difference in student performance whenhomework is collected and graded versus assigned and not graded. Student performancewas measured via a multiple-choice final examination. The study was conducted in aNumerical Methods course at a large university in the southeast of USA over a period ofthree years encompassing data from over 300 hundred Mechanical Engineering students.Because of the use of course management systems such as Blackboard(http://blackboard.com) in most universities and the availability of quiz makers such asRespondus (http://respondus.com), we wanted to determine if using graded onlinequizzes would improve the student performance, while minimizing grading time.Although other online homework and grading systems such as WebAssign(http://www.webassign.net/ are available, there are substantial recurring costs associatedwith the use of such systems both for students and faculty, and at this time availability ofengineering topics in these systems is very limited.As a pilot study, using Respondus, we developed endless online quizzes (EOQ) for thethree subtopics of Simultaneous Linear Equations topic taught in the class – Backgroundof Matrix Algebra, Gaussian Elimination and LU Decomposition. All quizzes had 6-7questions, and were of algorithmic form. This form allows the instructor to choose someor all variables to take values within a range, and develop a formula for the correctanswer. These formulas were developed by writing scripts with symbolic computation inMATLAB and Maple.When a student takes the quiz, the system randomly chooses the values of the selectedvariables, and he/she answers the question by filling in the answer field. The student’sanswer is checked against the correct value. Feedback, including the answer and thecorrectness of the answer, is given immediately.The students were given a limited amount of time (10 days from the start of the first subtopic) but unlimited attempts to complete all the three quizzes. The number of attempts,and the time taken and score for each attempt was automatically recorded by Blackboard.Student performance was evaluated by the number of correct answers in theSimultaneous Linear Equations portion of the multiple-choice final examination (total of4 questions out of a 32-question test). One tailed t-test comparing the performance ofstudents in the 2009 spring/summer terms (µ=2.72, σ2=0.87) with those in the 2010spring/summer terms (µ=2.89, σ2=0.88) showed a significant improvement; t(232)=1.428and p=0.08. In 2009, homework was assigned from the book but not picked for grade,while in 2010, we assigned and graded (3% of overall grade) the online quizzes for thetopic of Simultaneous Linear Equations.Focus groups were also conducted about the experience with the EOQ. Overall, thestudents indicated that they really enjoyed working with EOQ for several reasons. First,there was no time limit and hence there was no stress or pressure to complete theproblems. This allowed the students to stop and take a break or think through what theyneeded to do to complete the problem. Second, the students indicated that EOQ werehelpful in preparing them for their exams, allowed them to study more than they wouldhave without EOQ, and enjoyed getting the immediate feedback to help with “the littlethings.” Students also responded to a set of questions that asked how (or if) EOQ hadhelped them with understanding their coursework. The students indicated that thequestions in EOQ helped them identify what they knew and what they did not know.The effect of time spent, number of attempts and quiz scores on student performance willbe reported in the draft of the conference paper.

Citation
Format

Ph.D., G. L., & Kaw, A., & Yalcin, A. (2011, June), Using Online Endless Quizzes as Graded Homework Paper presented at 2011 ASEE Annual Conference & Exposition, Vancouver, BC. 10.18260/1-2--18967

TY  - CPAPER
AB  -                  Using Online Endless Quizzes as Graded HomeworkIn an effort to find the best use of limited teaching assistant’s time in today’s economy , astudy reported at last year’s ASEE 2010 conference by two of the authors of this paperindicated that there is no statistically significant difference in student performance whenhomework is collected and graded versus assigned and not graded. Student performancewas measured via a multiple-choice final examination. The study was conducted in aNumerical Methods course at a large university in the southeast of USA over a period ofthree years encompassing data from over 300 hundred Mechanical Engineering students.Because of the use of course management systems such as Blackboard(http://blackboard.com) in most universities and the availability of quiz makers such asRespondus (http://respondus.com), we wanted to determine if using graded onlinequizzes would improve the student performance, while minimizing grading time.Although other online homework and grading systems such as WebAssign(http://www.webassign.net/ are available, there are substantial recurring costs associatedwith the use of such systems both for students and faculty, and at this time availability ofengineering topics in these systems is very limited.As a pilot study, using Respondus, we developed endless online quizzes (EOQ) for thethree subtopics of Simultaneous Linear Equations topic taught in the class – Backgroundof Matrix Algebra, Gaussian Elimination and LU Decomposition. All quizzes had 6-7questions, and were of algorithmic form. This form allows the instructor to choose someor all variables to take values within a range, and develop a formula for the correctanswer. These formulas were developed by writing scripts with symbolic computation inMATLAB and Maple.When a student takes the quiz, the system randomly chooses the values of the selectedvariables, and he/she answers the question by filling in the answer field. The student’sanswer is checked against the correct value. Feedback, including the answer and thecorrectness of the answer, is given immediately.The students were given a limited amount of time (10 days from the start of the first subtopic) but unlimited attempts to complete all the three quizzes. The number of attempts,and the time taken and score for each attempt was automatically recorded by Blackboard.Student performance was evaluated by the number of correct answers in theSimultaneous Linear Equations portion of the multiple-choice final examination (total of4 questions out of a 32-question test). One tailed t-test comparing the performance ofstudents in the 2009 spring/summer terms (µ=2.72, σ2=0.87) with those in the 2010spring/summer terms (µ=2.89, σ2=0.88) showed a significant improvement; t(232)=1.428and p=0.08. In 2009, homework was assigned from the book but not picked for grade,while in 2010, we assigned and graded (3% of overall grade) the online quizzes for thetopic of Simultaneous Linear Equations.Focus groups were also conducted about the experience with the EOQ. Overall, thestudents indicated that they really enjoyed working with EOQ for several reasons. First,there was no time limit and hence there was no stress or pressure to complete theproblems. This allowed the students to stop and take a break or think through what theyneeded to do to complete the problem. Second, the students indicated that EOQ werehelpful in preparing them for their exams, allowed them to study more than they wouldhave without EOQ, and enjoyed getting the immediate feedback to help with “the littlethings.” Students also responded to a set of questions that asked how (or if) EOQ hadhelped them with understanding their coursework. The students indicated that thequestions in EOQ helped them identify what they knew and what they did not know.The effect of time spent, number of attempts and quiz scores on student performance willbe reported in the draft of the conference paper.
AU  - Gwen Lee-Thomas, Ph.D.
AU  - Autar Kaw
AU  - Ali Yalcin
CY  - Vancouver, BC
DA  - 2011/06/26
PB  - ASEE Conferences
TI  - Using Online Endless Quizzes as Graded Homework 
UR  - https://peer.asee.org/18967
DO  - 10.18260/1-2--18967
ER  -