An empirical study of the effect of granting multiple tries for online homework

When deploying online homework in physics courses, an important consideration is how many tries learners should be allowed to solve numerical free-response problems. While on the one hand, this number should be large enough to allow learners mastery of concepts and avoid copying, on the other hand, granting too many allowed tries encourages counter-productive behavior. We investigate data from an introductory calculus-based physics course that allowed different numbers of tries in different semesters. It turns out that the probabilities for successfully completing or abandoning problems during a particular try are independent of the number of tries already made, which indicates that students do not learn from their earlier tries. We also find that the probability for successfully completing a problem during a particular try decreases with the number of allowed tries, likely due to increased carelessness or guessing, while the probability to give up on a problem after a particular try is largely independent of the number of allowed tries. These findings lead to a mathematical model for learner usage of multiple tries, which predicts an optimum number of five allowed tries.