The study described here extends the applicability of the Newtonian Gravity Concept Inventory (NGCI) to college algebra-based physics classes, beyond the general education astronomy courses for which it was originally developed. The four conceptual domains probed by the NGCI (Directionality, Force Law, Independence of Other Forces, and Threshold) are well suited for investigating students' reasoning about gravity in both populations, making the NGCI a highly versatile instrument. Classical test theory statistical analysis with physics student responses pre-instruction (N = 1,392) and post-instruction (N = 929) from eight colleges and universities across the United States indicate that the NGCI is composed of items with appropriate difficulty and discrimination and is reliable for this population. Also, expert review and student interviews support the NGCI's validity for the physics population. Emergent similarities and differences in how physics students reason about gravity compared to astronomy students are discussed, as well as future directions for analyzing the instrument's item parameters across both populations.

1.
Constructivism: Theory, Perspectives, and Practice
, edited by
C. T.
Fosnot
(
Teachers College Press
,
New York
,
1996
).
2.
How People Learn: Brain, Mind, Experience, and School
, edited by
J. D.
Bransford
,
A. L.
Brown
, and
R. R.
Cocking
(
National Academy of Sciences
,
Washington, DC
,
1995
).
3.
A.
diSessa
, “
Toward an epistemology of physics
,”
Cognit. Instr.
10
,
105
225
(
1993
).
4.
S.
Vosniadou
, “
Capturing and modeling the process of conceptual change
,”
Learn. Instr.
4
,
45
69
(
1994
).
5.
L.
Bao
and
E. F.
Redish
, “
Model analysis: Representing and assessing the dynamics of student learning
,”
Phys. Rev. ST–Phys. Educ. Res.
2
,
010103
(
2006
).
6.
R. R.
Hake
, “
Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses
,”
Am. J. Phys.
66
(
1
),
64
74
(
1998
).
7.
R.
Thornton
and
D.
Sokoloff
, “
Assessing student learning of Newton's laws: The force and motion conceptual evaluation and the evaluation of active learning laboratory and lecture curricula
,”
Am. J. Phys.
66
(
4
),
338
352
(
1998
).
8.
J. M.
Bailey
, “
Development of a concept inventory to assess students' understanding and reasoning difficulties about the properties and formation of stars
,” Doctoral thesis in Teaching and Education, University of Arizona,
2006
.
9.
L.
Ding
,
R.
Chabay
,
B.
Sherwood
, and
R.
Beichner
, “
Evaluating an electricity and magnetism assessment tool: Brief electricity and magnetism assessment
,”
Phys. Rev. ST–Phys. Educ. Res.
2
,
010105-1
7
(
2006
).
10.
E. M.
Bardar
,
E. E.
Prather
,
K.
Brecher
, and
T. F.
Slater
, “
Development and validation of the light and spectroscopy concept inventory
,”
Astron. Educ. Rev.
2
(
5
),
103
113
(
2007
).
11.
E. E.
Prather
,
A. L.
Rudolph
,
G.
Brissenden
, and
W. M.
Schlingman
, “
A national study assessing the teaching and learning of introductory astronomy. Part I. The effect of interactive instruction
,”
Am. J. Phys.
77
,
320
330
(
2009
).
12.
L.
Ding
and
R.
Beichner
, “
Approaches to data analysis of multiple-choice questions
,”
Phys. Rev. ST–Phys. Educ. Res.
5
,
020103-1
17
(
2009
).
13.
W. M.
Schlingman
,
E. E.
Prather
,
C. S.
Wallace
,
A. L.
Rudolph
, and
G.
Brissenden
, “
A classical test theory analysis of the light and spectroscopy concept inventory national data set
,”
Astron. Educ. Rev.
11
,
010107
(
2012
).
14.
D.
Hestenes
,
M.
Wells
, and
G.
Swackhamer
, “
Force concept inventory
,”
Phys. Teach.
30
,
141
158
(
1992
).
15.
K.
Williamson
and
S.
Willoughby
, “
Student understanding of gravity in introductory college astronomy
,”
Astron. Educ. Rev.
11
,
010105
(
2012
).
16.
A.
Fraknoi
, “
Enrollments in astronomy 101 courses
,”
Astron. Educ. Rev.
1
(
1
),
121
123
(
2002
).
17.
I.
Halloun
and
D.
Hestenes
, “
Common sense concepts about motion
,”
Am. J. Phys.
53
(
11
),
1056
1065
(
1985
).
18.
I.
Galili
, “
Interpretation of students' understanding of the concept of weightlessness
,”
Res. Sci. Educ.
25
(
1
),
51
74
(
1995
).
19.
J.
Dostal
, “
Student concepts of gravity
,” Masters thesis in Physics, Iowa State University,
2005
.
20.
K.
Williamson
,
S.
Willoughby
, and
E. E.
Prather
, “
Development of the Newtonian gravity concept inventory
,”
Astron. Educ. Rev.
12
(
1
),
010107
(
2013
).
21.
J.
Benson
and
F.
Clark
, “
A guide for instrument development and validation
,”
Am. J. Occup. Ther.
36
(
12
),
789
800
(
1982
).
22.
J.
Creswell
,
Qualitative Inquiry and Research Design
, 2nd ed. (
Sage Publications
,
Thousand Oaks, CA
,
2007
).
23.
A.
Strauss
and
C.
Juliet
, “
Grounded theory methodology: An overview
,” edited by
N.
Denzin
and
Y.
Lincoln
, in
Handbook of Qualitative Research
, 1st ed. (
Sage Publications
,
Thousand Oaks, CA
,
1994
), pp.
273
284
.
24.
T. M.
Haladyna
,
S. M.
Downing
, and
M. C.
Rodriguez
, “
A review of multiple-choice item-writing guidelines for classroom assessment
,”
Appl. Meas. Educ.
15
(
3
),
309
334
(
2002
).
25.
C.
Wallace
and
J. M.
Bailey
, “
Do concept inventories actually measure anything?
,”
Astron. Educ. Rev.
9
,
010116
(
2010
).
26.
M. T.
Kane
, “
An argument-based approach to validity
,”
Psychol. Bull.
112
,
527
535
(
1992
).
27.
R. F.
Gunstone
and
R. T.
White
, “
Understanding of gravity
,”
Sci. Educ.
65
,
291
299
(
1981
).
28.
M.
Piburn
, “
Misconceptions about gravity held by college students
,”
Conference Proceedings, NARST Annual Meeting, Lake of the Ozarks, Missouri
, April 10–13. (ERIC Document Reproduction Service No. 292 616,
1998
).
29.
R. E.
Feeley
, “
Identifying student concepts of gravity
,” Masters thesis in Science and Teaching, The University of Maine,
2007
.
30.
A.
Asghar
and
J. C.
Libarkin
, “
Gravity, magnetism and ‘down’: Non-physics college students' conceptions of gravity
,”
Sci. Educ.
19
(
1
),
42
55
(
2010
).
31.
P. J.
Mulvey
and
S.
Nicholson
, “
Enrollments and degrees report
,”
AIP Report, Number R-151-45
(
2011
).
32.

However, we do not claim that the classroom environments in this sample are representative of algebra-based college physics courses nationwide. Those instructors who voluntarily participated in this study are likely more involved in education reform efforts than average.

33.

Majors are self-reported, so any distinction between “Science, Engineering, or Architecture” and “Health or Medical Sciences” would be at the discretion of the students answering.

34.
F. M.
Lord
and
M. R.
Novick
,
Statistical Theories of Mental Test Scores
(
Addison-Wesley
,
Reading, MA
,
1968
).
35.
R. K.
Hambleton
and
R. J.
Jones
, “
Comparison of classical test theory and item response theory and their application to test development
,”
Educ. Meas.: Issues Pract.
12
,
38
47
(
1993
).
36.
D.
George
and
P.
Mallery
,
SPSS for Windows Step by Step: A Simple Guide and Reference
(
Pearson Education
,
Boston, MA
,
2009
).
37.

This allows items that are more difficult to have a higher difficulty value. Note that this follows the work of Schlingman et al., which is opposite of the more conventional way of calculating item difficulty, where item difficulty is the fraction of correct responses.

38.
See supplementary material at http://dx.doi.org/10.1119/1.4945347 for histograms of the proportions of students choosing each answer choice for each of the 26 NGCI items, as well as interview notes from eleven “think-aloud” interviews with physics students.
39.

Interviews were with second-semester physics students just after they had finished first-semester physics. Audio recordings can be found in Ref. 42.

40.
R. N.
Bolton
and
T. M.
Bronkhorst
, “
Questionnaire pretesting: Computer-assisted coding of concurrent protocols
,” in
Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research
, edited by
N.
Schwarz
and
S.
Sudman
(
Jossey-Bass
,
San Francisco
,
1996
).
41.
R.
Weathersby
and
R.
Freyberg
,
Study Guide and SPSS Manual: To Accompany Statistics for the Behavior Sciences
(
Worth Publishers
,
New York
,
2008
).
42.
T. C.
Urdan
2010,
Statistics in Plain English
, 3rd ed. (
Taylor & Francis Group
,
New York
,
2010
).
43.
K.
Williamson
, “
Development and calibration of a concept inventory to measure introductory college astronomy and physics students' understanding of Newtonian gravity
,” Doctoral thesis, Montana State University,
2013
.

Supplementary Material

AAPT members receive access to the American Journal of Physics and The Physics Teacher as a member benefit. To learn more about this member benefit and becoming an AAPT member, visit the Joining AAPT page.