The Force Concept Inventory (FCI) has influenced the development of many research-based pedagogies. However, no data exists on the FCI’s internal consistency or test-retest reliability. The FCI was administered twice to one hundred students during the first week of classes in an electricity and magnetism course with no review of mechanics between test administrations. High Kuder–Richardson reliability coefficient values, which estimate the average correlation of scores obtained on all possible halves of the test, suggest strong internal consistency. However, 31% of the responses changed from test to retest, suggesting weak reliability for individual questions. A chi-square analysis shows that change in responses was neither consistent nor completely random. The puzzling conclusion is that although individual FCI responses are not reliable, the FCI total score is highly reliable.

1.
D.
Hestenes
,
M.
Wells
, and
G.
Swackhamer
, “
Force concept inventory
,”
Phys. Teach.
30
,
141
158
(
1992
).
2.
I. A.
Halloun
and
D.
Hestenes
, “
The initial knowledge state of college physics students
,”
Am. J. Phys.
53
,
1043
1055
(
1985
).
3.
I. A.
Halloun
and
D.
Hestenes
, “
Common-sense concepts about motion
,”
Am. J. Phys.
53
,
1056
1065
(
1985
).
4.
E
Mazur
,
Peer Instruction: A User’s Manual
(
Prentice Hall
,
Upper Saddle River, NJ
,
1997
).
5.
L. C.
McDermott
and
E. F.
Redish
, “
Resource letter: PER-1: Physics education research
,”
Am. J. Phys.
67
,
755
767
(
1999
).
6.
R. R.
Hake
, “
Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses
,”
Am. J. Phys.
66
,
64
74
(
1998
).
7.
The order of magnitude of the FCI data at Arizona State University is at least 50000 cases. Through the Interactive Learning Toolkit test module, the Mazur group at Harvard University has close to 10000 cases. There are several hundreds of undocumented FCI administrations each semester worldwide. We would not be surprised if the order of magnitude of FCI administrations was close to one million since its initial publication in 1992.
8.
R. P.
McDonald
,
Test Theory: A Unified Treatment
(
Lawrence Erlbaum Associates
,
Mahwah, NJ
,
1999
).
9.
R. K.
Thornton
and
D. R.
Sokoloff
, “
Assessing student learning of Newton’s laws: The force and motion conceptual evaluation and the evaluation of active learning laboratory and lecture curricula
,”
Am. J. Phys.
66
,
338
351
(
1998
).
10.
R. K.
Thornton
,
D.
Kuhl
,
K.
Cummings
, and
J.
Marx
, “
Comparing the force and motion conceptual evaluation and the force concept inventory
,”
Phys. Rev. STPhys. Educ. Res.
5
,
010105
(
2009
).
11.
J. M.
Bland
and
D. G.
Altman
, “
Cronbach’s alpha
,”
Br. Med. J.
314
,
572
(
1997
).
12.
R. C.
Sprinthall
,
Basic Statistical Analysis
7th ed. (
Allyn and Bacon
,
Boston
,
2003
). Also see <en.wikipedia.org/wiki/Z-test>.
13.
D.
Hammer
,
A.
Elby
,
R. E.
Scherr
, and
E. F.
Redish
, “
Resources, framing, and transfer
,” in
Transfer of Learning from a Modern Multidisciplinary Perspective
, edited by
J.
Mestre
(
Information Age Publishing
,
Greenwich, CT
,
2005
)
14.
E. F.
Redish
, “
A theoretical framework for physics education research: Modeling student thinking
,” in
Proceedings of the 2004 Enrico Fermi Summer School
,
Course CLVI, Italian Physical Society
, edited by
E.
Redish
,
C.
Tarsitani
, and
M.
Vicentini
, pp.
1
63
.
15.
M. S.
Sabella
and
E. F.
Redish
, “
Knowledge organization and activation in physics problem solving
,”
Am. J. Phys.
75
,
1017
1029
(
2007
).
16.
D.
Huffman
and
P.
Heller
, “
What does the force concept inventory actually measure?
,”
Phys. Teach.
33
,
138
143
(
1995
).
17.
D.
Hestenes
and
I.
Halloun
, “
Interpreting the Force Concept Inventory: A response to March 1995 Critique by Huffman and Heller
,”
Phys. Teach.
33
,
502
504
(
1995
).
18.
H.
Dedic
,
S.
Rosenfield
, and
N.
Lasry
, “
Are all wrong FCI answers equivalent?
,”
AIP Conf. Proc.
1289
,
125
128
(
2010
).
19.
S.
Vosniadou
, “
Capturing and modeling the process of conceptual change
,”
Learn. Instr.
4
,
45
69
(
1994
).
AAPT members receive access to the American Journal of Physics and The Physics Teacher as a member benefit. To learn more about this member benefit and becoming an AAPT member, visit the Joining AAPT page.