Valuable learning objectives of (experimental) physics education include developing in students the ability to design adequate methods and procedures, analyze data, and draw appropriate conclusions, including the specification of limitations to the validity.1 We have specified these learning goals as the understandings of evidence (UoE)2—insights and views that an experimental researcher relies on in constructing and evaluating scientific evidence. To build a foundation on which we can further develop these insights in my first-year physics lab course, I have redesigned an activity that is part of a teaching–learning sequence on scientific inquiry in secondary education.3 With this activity, deep questions about science, methodology, and validity are raised using simple means. I present the details pertaining to the intervention, the learning goals, and questions that can be addressed during this activity. Possibilities to adopt, adapt, and expand the activity are provided.
The intervention
The intervention—the Vitruvian man, symbolizing the ideal proportions of the human body according to Da Vinci—is carried out on the first day of the first-year physics lab course at our university as it (1) provides an outline of the course objectives and (2) lays a foundation for developing important insights in students. These insights, which we call understandings of evidence2 (Table I), are central in the course as they specify criteria to evaluate the quality of empirical evidence and explicitly explain not only what to do (e.g., repeat measurements) but also why doing so is essential in scientific inquiry (measurements will show inherent variation).
With this activity, we address a number of understandings of evidence2—insights and views that an experimental researcher relies on in constructing and evaluating scientific evidence.
| No. | The researcher understands that: | This understanding is demonstrated by: |
|---|---|---|
| 6 | It is important to choose suitable instruments and procedures to get valid data with the required accuracy and precision | Choosing appropriate measuring instruments and procedures that provide the required reliability and accuracy of the data set |
| 10 | It is important to use instruments and carry out procedures properly to obtain valid data with the required accuracy and precision | Intentionally carrying out measuring procedures and using instruments appropriately to optimally reduce measurement uncertainty |
| 12 | Data require appropriate methods for analyzing and describing them | Choosing data representation methods that reveal clearly and unambiguously the properties of, and patterns (or absence of these) in, the data set |
| 13 | An optimally informative answer to the research question requires a description of relationships in as much detail as possible. Quantitative descriptions are more detailed than qualitative ones | Describing the data by identifying salient and relevant patterns in detail and if possible their mathematical expression |
| 14 | A complete, clear, substantiated, and useful answer to the research question must be formulated | Formulating a clear, substantiated, and unambiguous answer |
| 16 | The validity of conclusions does not go beyond the data available. Therefore, limitations to the validity of the claim should be expressed | Specifying under what conditions the relationship/conclusion was established, discussing limitations |
| 18 | New questions may arise related to the inquiry | Proposing follow-up studies that stem from the outcomes of the inquiry |
| No. | The researcher understands that: | This understanding is demonstrated by: |
|---|---|---|
| 6 | It is important to choose suitable instruments and procedures to get valid data with the required accuracy and precision | Choosing appropriate measuring instruments and procedures that provide the required reliability and accuracy of the data set |
| 10 | It is important to use instruments and carry out procedures properly to obtain valid data with the required accuracy and precision | Intentionally carrying out measuring procedures and using instruments appropriately to optimally reduce measurement uncertainty |
| 12 | Data require appropriate methods for analyzing and describing them | Choosing data representation methods that reveal clearly and unambiguously the properties of, and patterns (or absence of these) in, the data set |
| 13 | An optimally informative answer to the research question requires a description of relationships in as much detail as possible. Quantitative descriptions are more detailed than qualitative ones | Describing the data by identifying salient and relevant patterns in detail and if possible their mathematical expression |
| 14 | A complete, clear, substantiated, and useful answer to the research question must be formulated | Formulating a clear, substantiated, and unambiguous answer |
| 16 | The validity of conclusions does not go beyond the data available. Therefore, limitations to the validity of the claim should be expressed | Specifying under what conditions the relationship/conclusion was established, discussing limitations |
| 18 | New questions may arise related to the inquiry | Proposing follow-up studies that stem from the outcomes of the inquiry |
At the start of the activity, students are told that they participate in a study in which the (fictitious) International Swimming League (ISL) investigates whether swimming competitions are fair: people with relatively longer arms may have an unfair advantage.4 This raises the question of whether swimming should have various categories, as is custom in many sports (e.g., boxing). To answer this question, it seems reasonable to first collect data and answer the following questions:
Is there, in general, a clear relation between body height and arm span?
How many people deviate from this ratio?
Students are asked to contribute to this study by measuring the body height and arm span of a fellow student and share the data using an online form. No further instruction on the procedures is given.
Students collect data
Without instruction on the procedure, students approach it differently, as can clearly be seen in Fig. 1. Some students stand against the wall, make use of additional tools, and sometimes take off their shoes. Others are standing in the middle of the classroom and do not take any measures to reduce uncertainties (and errors). Moreover, students respond differently to the problem that the given measuring tape is rather short (1.5 m).
Students measuring each other’s body height and arm span in various ways. Some considered ways of reducing uncertainties, whereas others clearly did not do so.
Students measuring each other’s body height and arm span in various ways. Some considered ways of reducing uncertainties, whereas others clearly did not do so.
As students only take a single measurement, they are finished within 5 minutes. This collaborative data collection approach5—where students engage in an experiment, but data-collection time is significantly reduced by sharing the data—leaves ample time for discussion. Once students are back at their seats, a plenary discussion is started. The first question I pose is whether the ISL can make use of the data collected. This question strongly relates to the validity of the students’ approaches: whether they actually measured what they want to measure. Students easily understand that their procedures differ, that some ways of measuring are better than others, but that there is (now) no way of telling which data points are trustworthy. The take-home message is that one has to choose suitable methods and procedures to get valid data (UoE 6). Students are then asked to think of an appropriate procedure to measure the two lengths with the available instruments.
Students analyze the data
The data are retrieved from the form and saved as a CSV file. I use the Python programming language to load the data and start the data analysis. As most students are not acquainted with Python yet, the script is explained step by step. The measurements are shown in a scatterplot (see Fig. 2). A discussion is started on whether a pattern in the data can be detected (UoE 12), and the idea of a line of best fit is introduced (UoE 13). Students then discuss whether the line of best fit should pass through the origin, i.e., whether the data should be fitted using a direct proportional or linear relation (see Fig. 3). A question that almost automatically arises is whether a linear relationship (y = ax + b) holds genuine physical significance (what does an offset imply?). The subsequent discussion addresses the issue that students frequently use a function devoid of any real-world interpretation.
Once the data have been retrieved, a scatterplot is produced and shown to the students.
Once the data have been retrieved, a scatterplot is produced and shown to the students.
A plot of the data set including two lines of best fit and an additional measurement of a 3-year-old. The two lines of best fit are so close that the red line is completely covered by the blue line here. However, the introduction of an offset provides a genuine different physical meaning in the context of body measures. The plot of the residuals is not shown during the introductory activity as it would probably lead to a cognitive overload.6 However, after a few weeks students should be able to conduct a residual analysis themselves.
A plot of the data set including two lines of best fit and an additional measurement of a 3-year-old. The two lines of best fit are so close that the red line is completely covered by the blue line here. However, the introduction of an offset provides a genuine different physical meaning in the context of body measures. The plot of the residuals is not shown during the introductory activity as it would probably lead to a cognitive overload.6 However, after a few weeks students should be able to conduct a residual analysis themselves.
The idea of additional data is then introduced: if we have measurements of body height and arm span available, can we combine these with our data without further thought? Or else, under what conditions may we include these? To further arouse a discussion, I show the measurements of my three-year-old daughter (see Fig. 3). Can we consider this datum part of the data set?
Finally, the data set is combined with a larger data set (growing each year). A new line of best fit is produced. Subsequently, various 2D histograms are shown, each with a different bin size. Reducing the bin size further and further results in the initial scatter plot. Students are asked which data representations make sense and could be used in presenting the results to the ISL (UoE 12).
Students discuss potential conclusions and the study’s validity
Once the interpretation of the data has been discussed extensively, I draw students’ attention to potential conclusions. Two obvious conclusions from Fig. 3 are that men tend to be taller than women and that, unfortunately, women are underrepresented in studying physics. Other conclusions, which do relate to the context, are presented with increasing precision (for conclusions 3 and 4, students are asked to use the data analysis to fill in the missing information), and their value is discussed:
Taller people have longer arms.
There is a relation between arm span and body length.
There is a direct proportional relation between …
The relation between arm span and body length is given by …
Finally, the students are asked whether they can draw even more informative conclusions.
Students come to see that multiple conclusions can be drawn but that more informative conclusions are to be preferred (UoE 14). Next, the validity of each conclusion is evaluated (UoE 16). As the data all stem from people studying physics, does the validity go beyond this sample? Or should we specify that this conclusion is mainly valid for white males from Western Europe, as physics has nothing to do with it? Considering the limitations of the study, particularly those arising from the sample (size), what data should be collected subsequently (UoE 18)? A quick Internet search provides ample ideas to direct the discussion further,7 if time allows.
To finalize the activity, I summarize the main take-home messages and outline how these elements of doing experimental work in physics are covered in other parts of the course.
Adopt, adapt, expand
Readers can readily adopt the intervention using the provided foundational data set and access the Python script in the supplementary materials.8 The activity offers ample opportunities for adaptation and future expansion. For instance, in learning to program, students can use the data to produce the graphs, determine the coefficients of the fit function and their associated uncertainties, and plot the residuals and verify whether they adhere to a normal distribution, as demonstrated in Fig. 4.
A residual analysis (direct proportional) shows that the residuals can be described by a normal distribution with an average of 0.0 cm and a standard deviation of 5.0 cm.
A residual analysis (direct proportional) shows that the residuals can be described by a normal distribution with an average of 0.0 cm and a standard deviation of 5.0 cm.
As mentioned, this activity has been adapted from a high school level activity.3 Although the initial steps remain similar at that level (except that I would use a spreadsheet program), a significant distinction lies in the focal point of the discussions. With younger students, the emphasis should revolve around fostering scientific literacy—the “general scientific awareness and understanding of science required to participate meaningfully in modern societal issues.”9 This activity may contribute especially to the development of the core ability: “being able to interpret data and evidence scientifically: Analyse and evaluate data, claims and arguments in a variety of representations and draw appropriate scientific conclusions.”10 Rather than reviewing a whole range of UoE, it would be pragmatic to concentrate on a specific facet, for instance, using the question “What conditions need to be met before accepting these data as evidence that can support a conclusion?”
In conclusion
In this activity, various important understandings are addressed using simple means. Students “discover” that the method of data collection is important and that the chosen method has a large bearing on the quality of data they get; this in turn has a large bearing on the validity of the potential conclusions they may draw. They begin to grasp the concept that in scientific research, only the best available answer to the research question is deemed satisfactory.2 Nonetheless, we cannot expect students to fully master these insights and apply these correctly in their own research projects yet. Moreover, being able to engage in experimental physics requires development of the full set of 19 understandings of evidence. Therefore, subsequent activities are required in which students expand their understanding. Still, in this activity, a foundation is laid that can be referred to in subsequent activities. By using Python to analyze the data, students come to see the benefit of learning how to program, i.e., where it leads to. Moreover, students make small steps in attaining a learning goal from the hidden curriculum: in physics experiments, it often takes more time to devise a proper method and process the data into adequate conclusions than it takes to collect the data.
REFERENCES
Freek Pols was a high school physics teacher for 10 years. Currently, he is coordinator of the first-year physics lab course and an assistant professor in physics education at Delft University of Technology.