To characterize the evolution of student understanding better than what is possible by pre-and post-testing, we posed simple conceptual questions several times per week to separate, randomly selected groups of introductory physics students. This design avoids issues of retesting and allows for tracking of student understanding of a given topic during the course with a resolution on the order of days. Based on the data from over 1600 students over five quarters, we found significant and interesting changes in performance on time scales of days and weeks. We found that the response curves of performance versus time can be divided into four categories: flat (no change), step-up, step-up and decay, and step-down. We examined changes on a 1 day time scale and found that changes in performance did not coincide with relevant traditional lectures or laboratories, but coincide with due dates of relevant on-line homework that provides immediate feedback. This method is well suited to measuring the effect of particular instructional events as well as providing insight into the nature of student understanding and learning.
REFERENCES
1.
E. C.
Sayre
and A. F.
Heckler
, “Peaks and decays of student knowledge in and introductory E & M course
,” Phys. Rev. ST Phys. Educ. Res.
5
, 013101
(2009
).2.
R. S.
Siegler
and K.
Crowley
, “The microgenetic method: A direct means for studying cognitive development
,” Am. Psychol.
46
, 606
–620
(1991
).3.
D.
Kuhn
, “Microgenetic study of change: What has it told us?
,” Psychol. Sci.
6
, 133
–139
(1995
).4.
D. P.
Maloney
, T. L.
O’Kuma
, C. J.
Hieggelke
, and A.
Van Heuvelen
, “Surveying students’ conceptual knowledge of electricity and magnetism
,” Am. J. Phys.
69
, S12
–S23
(2001
).5.
P. V.
Engelhardt
and R. J.
Beichner
, “Students’ understanding of direct current resistive electrical circuits
,” Am. J. Phys.
72
, 98
–115
(2004
).6.
G. B.
Semb
and J. A.
Ellis
, “Knowledge taught in school: What is remembered?
,” Rev. Educ. Res.
64
, 253
–286
(1994
).7.
D. C.
Rubin
and A. E.
Wenzel
, “One hundred years of forgetting: A quantitative description of retention
,” Psychol. Rev.
103
, 734
–760
(1996
).8.
S. M.
Austin
and K. E.
Gilbert
, “Student performance in a Keller-Plan course in introductory electricity and magnetism
,” Am. J. Phys.
41
, 12
–18
(1973
).9.
G. E.
Francis
, J. P.
Adams
, and E. J.
Noonan
, “Do they stay fixed?
,” Phys. Teach.
36
, 488
–490
(1998
).10.
A.
Savinainen
, P.
Scott
, and J.
Viiri
, “Using a bridging representation and social interactions to foster conceptual change: Designing and evaluating and instructional sequence for Newton’s third law
,” Sci. Educ.
89
, 175
–195
(2005
).11.
R. K.
Thornton
, “Conceptual dynamics: Following changing student views of force and motion
,” AIP Conf. Proc.
399
, 241
–266
(1997
).12.
J.
Petri
and H.
Niedderer
, “A learning pathway in high school level quantum atomic physics
,” Int. J. Sci. Educ.
20
(9
), 1075
–1088
(1998
).13.
D. E.
Meltzer
, “How do you hit a moving target? Addressing the dynamics of students’ thinking
,” in Proceedings of 2007 Physics Education Research Conference
, edited by J.
Marx
, P.
Heron
, and S.
Franklin
(AIP
, Melville, NY
, 2004
), pp. 7
–10
.14.
L.
Bao
and E. F.
Redish
, “Model analysis: Representing and assessing the dynamics of student learning
,” Phys. Rev. ST Phys. Educ. Res.
2
, 010103
(2006
).15.
R. R.
Hake
, “Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses
,” Am. J. Phys.
66
, 64
–74
(1998
).16.
L.
Ding
, N. W.
Reay
, A.
Lee
, and L.
Bao
, “Effects of testing conditions on conceptual survey results
,” Phys. Rev. ST Phys. Educ. Res.
4
, 010112
(2008
).17.
H. L.
Roediger
III, “Relativity of remembering: Why the laws of memory vanished
,” Annu. Rev. Psychol.
59
, 225
–254
(2008
).18.
J. A.
McGeoch
, “Forgetting and the law of disuse
,” Psychol. Rev.
39
, 352
–370
(1932
).19.
M. C.
Anderson
and J. H.
Neely
, “Interference and inhibition in memory retrieval
,” in Memory. Handbook of Perception and Cognition
, 2nd ed., edited by E. L.
Bjork
and R. A.
Bjork
(Academic
, San Diego, CA
, 1996
), pp. 237
–313
.20.
J. D.
Karpicke
and H. L.
Roediger
III, “The critical importance of retrieval for learning
,” Science
319
, 966
–968
(2008
).21.
A. C.
Butler
and H. L.
Roediger
III, “Testing improves long-term retention in a simulated classroom setting
,” Eur. J. Cogn. Psychol.
19
, 514
–527
(2007
).22.
C.
Henderson
, “Common concerns about the force concept inventory
,” Phys. Teach.
40
, 542
–547
(2002
).23.
J. C.
Dutton
, “WebAssign: A better homework delivery tool
,” Technology Source, January/February 2001
.24.
D.
Halliday
, R.
Resnick
, and J.
Walker
, Fundamentals of Physics
, 6th ed. (Wiley
, New York
, 2008
).25.
I. J.
Myung
, “Tutorial on maximum likelihood estimation
,” J. Math. Psychol.
47
, 90
–100
(2003
).26.
J. B.
Willett
, “Questions and answers in the measurement of change
,” in Review of Research in Education
, edited by E. Z.
Riothkopf
(American Educational Research Association
, Washington, D.C.
, 1988
), Vol. 15
.27.
D.
Rogosa
, D.
Brandt
, and M.
Zimowski
, “A growth curve approach to the measurement of change
,” Psychol. Bull.
92
, 726
–748
(1982
).28.
As an analogy, we might consider that the theory of stellar evolution is empirically based on the cross sectional data because longitudinal data collection is not feasible, except for perhaps computer simulation data, the models of which must ultimately be based on the cross sectional observations.
29.
I. A.
Halloun
and D.
Hestenes
, “The initial knowledge state of college physics students
,” Am. J. Phys.
53
, 1043
–1055
(1985
).30.
R.
Rosenblatt
, E. C.
Sayre
, and A. F.
Heckler
, “Toward a comprehensive picture of student understanding of force, velocity, and acceleration
,” in Proceedings of 2008 Physics Education Research Conference
, edited by C.
Henderson
, M.
Sabella
, and L.
Hsu
(AIP
, Melville, NY
, 2008
).31.
Cohen’s effect size is a standard measure that quantifies the size of the difference between the means of two samples. It is defined as the number of standard deviations one mean is from the other: , where is the pooled standard deviation of the two samples.
32.
L. C.
McDermott
and P. S.
Shaffer
, “Research as a guide for curriculum development: An example from introductory electricity. Part 1: Design of instructional strategies
,” Am. J. Phys.
60
, 994
–1003
(1992
).33.
P. S.
Shaffer
and L. C.
McDermott
, “Research as a guide for curriculum development: And example from introductory electricity. Part 2: Investigation of student understanding
,” Am. J. Phys.
60
, 1003
–1013
(1992
).34.
We are not claiming that the students are explicitly (and incorrectly) thinking “ is a scalar, so I have to answer in this way.” Rather, it is more likely that they are not thinking much at all about the need to distinguish the difference between vector and scalar quantities when determining and .
35.
There appears to be a dip in the direction-of-field response in weeks 3 and 4, and answers seem to be random during these weeks. It is not clear why this response occurred. For example, there is no decrease in average student grade during this time. It might be due to some kind of interference as well.
36.
The interference of learning magnetic force on answering electric force questions was observed by T. Scaife and A. Heckler in earlier work. A manuscript describing this observation is in preparation.
37.
F. N.
Dempster
, “Spacing effects and their implications for theory and practice
,” Educ. Psychol. Rev.
1
, 309
–330
(1989
).38.
R.
Seabrook
, G. D. A.
Brown
, and J. E.
Solity
, “Distributed and massed practice: From laboratory to classroom
,” Appl. Cognit. Psychol.
19
, 107
–122
(2005
).39.
P. I.
Pavlik
and J. R.
Anderson
, “Using a model to compute optimal schedule of practice
,” J. Exp. Psychol., Appl.
14
, 101
–117
(2008
).© 2010 American Association of Physics Teachers.
2010
American Association of Physics Teachers
AAPT members receive access to the American Journal of Physics and The Physics Teacher as a member benefit. To learn more about this member benefit and becoming an AAPT member, visit the Joining AAPT page.
