Problem roulette: Studying introductory physics in the cloud

We introduce Problem Roulette (PR), a web-based study service at the University of Michigan that offers random-within-topic access to a large library of past exam problems in introductory physics courses. Built on public-private cloud infrastructure, PR served nearly 1000 students during Fall 2012 term, delivering more than 60,000 problem pages. The service complements that of commercial publishing houses by offering problems authored by local professors and by explicitly aligning topics with exam content. We describe the service architecture, including reporting and analytical capabilities, and present an initial evaluation of the impact of its use. Among roughly 500 students studying electromagnetism, we find that the 229 students who worked fifty or more problems over the term outperformed their complement by 0.40 grade points (on a 4.0 scale). This improvement partly reflects a selection bias that academically stronger students used the service more frequently. Adjusting for this selection bias, we find a grade point improvement of 0.22, significantly above the random noise level of 0.04. The simple message to students is that working five or more additional problems per week can lead to a quarter-letter grade improvement in introductory physics. Student comments emphasize the importance of randomness in helping them to synthesize concepts. The PR source code is publicly available.