Taking on astronomy misconceptions isn’t easy

Is the order in which topics are presented crucial in helping students correct their misconceptions? Our intuition was that within a suitably identified subtopic, the optimal approach was first to address misconceptions that were least strongly held and only then move on to more persistent ones. To help us construct what we hoped would be optimal pedagogy, we applied item response theory (IRT) to quantify the inherent persistence with which the misconceptions are held. But we found that our intuitively optimal ordering pedagogy was not particularly effective.

If IRT analysis is to be valid, the data must be unidimensional. That is, a single trait, such as the tendency to retain a belief that may be a misconception, should account for the variance in the responses to the ABI. To generate subtopics that in our estimation would yield unidimensional data, we first grouped misconceptions topically; for example, one topic included statements about galaxies. We then used a statistical software package called SPSS to determine subtopics—the basis for our IRT analysis—that consisted of tightly correlated misconceptions. The box to the left displays the three subtopics related to galaxy misconceptions. It shows, for instance, that students who believe that new planets and stars don’t form today are also likely to believe that the solar system is not in the Milky Way or any other galaxy.

Based on the responses to a statement in our beliefs inventory, IRT calculates how likely a student is to have a particular misconception given the student’s overall tendency to retain misconceptions. The plot in the figure shows the results for the misconception that galaxies are randomly distributed. The variable along the horizontal axis represents the student’s overall proclivity to retain misconceptions and is defined so that its mean is 0 and its standard deviation is 1.

The horizontal coordinate where the curves labeled 2 and 3 in the figure intersect corresponds to a transition from“believed the misconception but learned otherwise from taking AST 109” to “still believe the misconception even after instruction.” A characteristic 2–3 transition coordinate exists for each statement in the ABI. A low transition coordinate means that the misconception is retained even by students who have a weak tendency to hold fast to wrong ideas; low transition coordinates thus correspond to misconceptions that are hard to shake if a student has believed them for a long time.

That galaxies are randomly distributed is one of a dozen galaxy-related misconceptions in the ABI. The boxshows all 12, along with their 2–3 transition coordinates. As mentioned earlier,they have been subdivided into three groups as determined by an SPSS analysis. In each galaxy subtopic, we determined an optimal order in which to teach individual galaxy-related statements by ranking their corresponding transition coordinates. We ordered the subtopics as a whole similarly; for example, the box indicates that misconceptions associated with the visual properties of galaxies are, collectively, the least strongly held. Thus, reading from top to bottom, the box shows our proposed order for addressing the 12 galaxy misconceptions in the ABI. The analogous 1–2 transitions, which may produce a different ranking than the 2–3 transitions, give an optimal order that may be better tailored to children and adolescents.

To test whether our proposed ordering would do a better job than conventional ordering in helping students to abandon their misconceptions, we designed single-lecture videos on black holes (to serve as a consistency check) and galaxies,to be presented to classes as a replacement for live lectures. The videos shown in the spring 2012 semester presented the concepts for both black holes and galaxies in the optimal order as suggested by our analysis. In the spring 2013 semester, we retained the proposed optimal ordering for the black hole videos but presented the galaxies lectures with the concepts in the order normally taught at the University of Maine.

Immediately before and after each video lecture, we assessed the students’preconceived beliefs using a multiple-choice test whose questions were tailored to the video of the day. As a consistency check, we confirmed that test gains on the black hole topic were nearly identical between the two years. We expected, though,that our optimal ordering of galaxy topics would give overall higher test gains than the conventional ordering. In fact, gains on the galaxy test scores between the two arrangements of topical material were also identical. Perhaps misconceptions about galaxies are more persistent than we originally thought, and we suspect that the same may be true in other astronomical topics as well. Fortunately, even though we didn’t find an optimal order to help students abandon their own mistaken beliefs, we still learned a lot about the persistence of individual misconceptions. And that information, we think, will enable instructors to improve their pedagogy by spending more time addressing student misconceptions that most strongly persist.

Andrej Favia just completed a temporary appointment as assistant professor of physics, and Neil Comins and David Batuskiare professors of physics, all at the University of Maine in Orono.