Perspective on improving the quality of surface and material data analysis in the scientific literature with a focus on x-ray photoelectron spectroscopy (XPS)

X-ray photoelectron spectroscopy (XPS) is the most important method for chemically analyzing surfaces,^1–3 where its use has increased dramatically during the past two decades. Indeed, XPS is now employed for an extremely diverse range of applications. Its use extends far beyond materials science and for nano-objects^4,5 into areas such as geology and medical coatings. While these applications represent a success story for XPS, and, of course, for XPS manufacturers, they also indicate how difficult it has become to communicate with the (now) diverse user base about pitfalls and improved practices for XPS data analysis. For reasons that will be discussed herein, a great deal of incorrect XPS data analysis is currently entering the scientific literature.⁶ For example, a recent survey of three high-quality journals revealed that ca. 40% of the XPS peak fits in them are completely incorrect, and another 40% are suspect.⁷ Essentially all of this research was funded by governmental agencies. The peak fits in the papers evaluated in this retrospective analysis were classified as “red” (seriously incorrect), “orange” (questionable/suspect), “yellow” (containing minor errors, but otherwise quite reasonable and contributing to the message of the paper), and “green” (a high-quality fit). The levels of “red” errors observed in this study are not far from those reported by experts of other surface and material characterization techniques.^8–12 Indeed, current issues with XPS are compounded by the fact that most new surfaces and materials are characterized by multiple analytical methods. Statistically speaking, the odds of serious errors appearing in a study increase as the number of analytical techniques used in it increases. Ideally, results from complementary analytical methods should support each other and the claims of a study. Further complicating the matter, key experimental parameters and data collection and analysis protocols are often not reported in papers.¹³ For example, some authors do not even list the type of XPS instrument or x-ray source they used, let alone the instrument settings, e.g., pass energy or x-ray spot size, or details regarding their data work-up, e.g., the type of background and synthetic peaks used in peak fitting, their rationale for choosing them, and calibration of relative sensitivity factors. Reproducibility issues are also present in surface/thin film and material preparation, e.g., in the atomic layer deposition of multiple materials.⁸ 

The “red” errors commonly observed in XPS data analysis are major errors. At best, these errors render analyses meaningless. However, more seriously, they may distort or even invalidate the conclusions of studies. In some cases, it almost appears as if some fits are done more to support the thesis of a paper, or to maximize the goodness of a fit, than to try to understand a material. Sometimes, entire paragraphs, or even pages of text, “explain” how such analyses support the conclusions of papers. In general, these major errors are easily spotted by those with even an intermediate understanding of the technique. We believe they are avoidable. To illustrate the severity of this problem, we have recreated six examples of incorrect peak fits that we have seen in papers and presentations and borrowed one that was recreated by other XPS experts (see Figs. 1–6). The figures we made were produced from our own synthesized data. These figures are not exaggerations of what is now regularly found in the literature. This approach was taken with permission from this journal. As far as we can tell, the original data in these figures were collected with conventional, standalone XPS systems using monochromatic Al Kα x-ray sources. We now explain the problems with these fits/analyses.

1. The spectrum in Fig. 1 is dominated by noise, although it may show some low frequency undulations in its baseline, which are about the same size as the noise, and which may contain useful information.¹⁴ Superimposed on a questionable background (the baseline does not match the general undulations of the data), the data in Fig. 1 have been fit to a series of peaks that represent different chemical states. The chemical/oxidation states of the elements are poorly labeled, and the spin-orbit splitting is not correct. The areas of these peaks do not follow known 1:2 areas ratios for the spin-orbit splitting of the 2p signal. Basically, this analysis is completely incorrect. There are no features in the data that support the presence of these fit components. Furthermore, this fit completely ignores the standards for evaluation of signal significance in noisy spectra. No useful information regarding the amounts of different chemical states can be extracted from this analysis at any reasonable level of significance. Any inferences or conclusions drawn from this analysis would be at best misleading. If the sample was sufficiently stable during data analysis, and the authors felt that this spectral region contained useful information, they should have increased their data acquisition time to improve the signal-to-noise ratio of their data.

2. All XPS signals originating from p, d, and f orbitals show spin-orbit splitting, where (i) these signals appear as doublets, (ii) their relative areas are determined by quantum mechanics: ca. 1:2, 2:3, and 3:4, respectively, where the precise theoretical values are given by Scofield,¹⁵ (iii) in general, each different oxidation state for an element produces a pair of spin-orbit peaks at a different pair of binding energies, and (iv) the energy differences between spin-orbit doublets are quite constant (they only depend a little on sample chemistry).¹⁶ While, in some cases, the energy difference between the spin-orbit peaks of an element is quite small, e.g., the Al and Si 2p_1/2 and 2p_3/2 peaks overlap significantly,¹⁷ for some orbitals they are relatively large so pairs of fit components must be included for each chemical state. The three panels in Fig. 2 show XPS analyses of the S 2p region of sulfur and a sulfur-containing material.^3, Figure 2(a) shows a narrow scan of sulfur itself, i.e., sulfur in one chemical state. This data envelope is fit to a spin-orbit doublet (the S 2p_1/2 and 2p_3/2 peaks) with their correct separation (1.2 eV) and with their correct 1:2 intensity ratio. Figure 2(b) shows an inappropriate fit to an S 2p narrow scan of a more complex sulfur-containing material. Sulfur is present in multiple chemical states in this material. This narrow scan was fit with five peaks that are supposed to represent different chemical states. However, spin-orbit splitting has been entirely ignored. Any quantitative (or even qualitative) evaluation of the different oxidation states proposed here would be erroneous. In addition to this significant error, this fit shows considerably different peak widths for some of the proposed chemical states. While there are cases where the different chemical states of an element result in peaks with noticeably different widths, e.g., the signals from reduced metals are often conspicuously narrower than those from oxides,^18,19 these situations are usually more the exception than the rule. In Fig. 2(b), the significantly different peak widths do not appear to be justified. They appear to emerge out of an attempt to fit doublets with single components. Figure 2(c) shows a reasonable fit to this data with four spin-orbit doublets that represent four different chemical states.

3. It is common for series of related samples to be prepared in thin film growth and material synthesis studies. Because the resulting surfaces are often chemically similar, it is usually appropriate to use similar protocols to fit their XPS spectra. Figure 3 shows spectra of two closely related materials. However, the authors of this study have not used appropriately similar protocols to fit them. For example, it seems impossible to justify the presence of a very narrow fit component at ca. 720 eV in the upper spectrum and a very wide component in the lower spectrum that are supposed to represent the same chemical state. The significantly differing peak widths and peak positions in these fits of components that are supposed to represent the same chemical states render this analysis meaningless. A minor error in this analysis is that the data are plotted with binding energy increasing to the right, which is against convention. Finally, the baseline appears to have been subtracted from these spectra. We do not recommend this practice.

4. When peak fitting is performed appropriately, the concentrations of an element in its various chemical states are proportional to the areas of the synthetic peaks in a fit.²⁰ A major (red-type) error in Fig. 4 is the wide peak at 400.5 eV that extends substantially outside of the peak envelope. It also seems difficult to justify the presence of the smaller, narrower fit component under the larger, broader one. Two minor errors in this fit are that: (i) the baseline is too high on the high binding energy side of the peak envelope, and (ii) the spectrum is plotted backwards (against convention).

5. Figure 5 shows an alarming trend in published XPS peak fitting, which is the tendency to fit only a portion of a peak envelope and to call this a fit. This approach ignores the fact that other peaks corresponding to other chemical states and/or satellite signals may be present in the peak envelope, and/or that the synthetic peaks themselves inadequately represent the data. It would have been better to identify the possible positions of peaks in this peak envelope using vertical lines, if this is what was intended in this figure.

6. The major error in Fig. 6 is the labeling of a noise fluctuation as a chemical state. This “peak” is too low in intensity (not above the background, see the remarks above for Fig. 1 on signal significance), too narrow, and not different enough from the background to be a real signal. A conventional, monochromated Al Kα x-ray source has a peak width of 0.26 eV,²¹ which puts a lower limit on the breadths of peaks in conventional XPS (unmonochromated Mg Kα x-ray peaks are quite a bit wider). Factors that broaden real XPS signals include the instrument, e.g., convolution over its slits and detector elements, quantum mechanics (lifetime broadening), the sample chemistry, e.g., more amorphous materials generally show broader peaks than more crystalline ones, and differential charging. Finally, this “signal” may not even be Li. A survey of Li 1s peaks in the literature from both Li metal and Li⁺ containing materials shows them to have positions of 55.6 ± 0.7 eV.^22–35 Of course, in defense of the authors (on this point), their data may not have been charge corrected.

We hope that the spectra shown in Figs. 1–6 and accompanying discussion demonstrate that we are not quibbling or being pedantic about the problem of incorrect XPS data analysis in the literature. The errors in Figs. 1–6 are not subtle, advanced, or based on speculative interpretations of the data. These are major errors that are quite easily spotted by those with even an intermediate understanding of the technique. These errors may compromise the papers and presentations they are in. We believe it is reasonable to expect authors of scientific studies to be capable of avoiding these types of blatant errors. We think it is reasonable to expect scientists and engineers to be able to differentiate between signal and noise as in Figs. 1 and 6. Indeed, some of these errors seem to have little to do with XPS. They may point to something deeper. They suggest that critical thinking skills are not being taught and/or learned by some scientists. Also, the presence of seriously incorrect data analysis in the literature is not a benign phenomenon. It reduces its usefulness and credibility. Unfortunately, the types of errors in Figs. 1–6 are often propagated as authors copy and cite incorrect analyses in follow-up studies. Along these lines, one of us has found it difficult to convince students that what is in the literature is sometimes incorrect. In some cases, students insist on using and citing incorrect data fitting protocols. Nevertheless, instances of very good XPS data acquisition and analysis are present in the scientific literature. As examples, we recommend the XPS data fitting and analysis in the recent JVST guides on XPS,^{1,3,14,19,36–42} Strohmeier's determination of oxide thicknesses on aluminum alloys,⁴³ and Baer and co-workers' study of surface potentials during XPS of non-homogeneous samples.⁴⁴ 

In this Perspective, we explore six aspects of incorrect surface and material characterization, with XPS used as an example. In the first, we discuss whether surface and material analysis (and science in general) is in a state of “crisis” or “pre-crisis.” Second, we discuss why XPS is thriving. Third, we argue that the large number of surface and material characterization techniques presents unique challenges to those wishing to characterize their materials. Fourth, we consider the consequences of the increase in the number of papers showing XPS and the simultaneous decrease in the fraction of these papers published in surface journals. Fifth, we explain how material characterization is handled at many research centers, and implications of this approach. Sixth, we present a behavioral/game theory model (the Prisoner's Dilemma) that may apply to the problem of incorrect surface and material characterization. This model suggests that the current system may incentivize researchers not to characterize their materials well. Other Prisoner's Dilemmas may lead to other suboptimal outcomes in science. Finally, we consider how the different stakeholders in the problem of incorrect surface and material characterization may work to improve the current situation.

While the concerns raised in this work are related to the so-called “reproducibility crisis,”^45–47 some have questioned whether a “crisis” actually exists.^48,49 How is there a crisis when high-quality research is being performed at many places worldwide? We agree that the current situation probably does not constitute a full-blown crisis. Nevertheless, there is growing recognition that there are concerning issues in surface and material characterization, and even more generally in science. Indeed, these issues may be worsening. For example:

• There are often errors in papers that employ new experimental and computational techniques. Some of these errors compromise the messages of these papers.

• Significant errors are regularly observed in the analysis of data from some of the more established surface and material characterization techniques. As noted in this work, too much of this incorrect analysis is entering the literature. The existence of incorrect information in the literature presents a challenge for non-experts attempting to use these techniques.

• Many scientists who have specialized in surface and material characterization techniques such as XPS have retired or will do so soon. This anticipated loss of specialized knowledge was recognized two decades ago, along with the likely later lack of expert knowledge for acquiring and analyzing XPS data. A workshop in 2002, which addressed this issue, entitled “XPS: From Spectra to Results – Towards an Expert System” was sponsored by the International Union of Vacuum Science, Technique and Applications.⁵⁰ The participants at this workshop met in six groups to discuss issues such as instrument and specimen characterization, experimental objectives, instrument set-up and data acquisition, wide-scan interpretation, protocols for narrow scans, and analysis of chemical states and surface morphology. These discussions led to many recommendations for elements of an expert system on XPS.^51,52

Rather than declare that there is a “crisis” in surface and material analysis, and perhaps even in science in general, we suggest that we may be in a “pre-crisis” mode. We believe that if action is not taken to improve the quality of the surface and material characterization being reported in the literature, our current pre-crisis state may deteriorate further, making more of the literature unusable and perhaps discrediting surface and material analysis and even science in general.

Figure 7(a) shows a plot of Eq. (1). This plot shows the change in time in the population (⁠ $P ˙$⁠) as a function of the population (⁠ $P$⁠). It allows us to identify/propose possible populations (orange dots/points on the x-axis) and understand their dynamics (movement). The line/curve plotted above the x-axis in the graph indicates the direction (and speed) of the points on the x-axis. If the curve is above a point in question on the x-axis, the point moves to the right, where the height of the curve gives the speed at which the point moves. Similarly, if the curve is below the point in question, it moves to the left. This representation of the movement/dynamics of states in one dimension is referred to as “flow along the line.” In our model, only points at P = 0 do not move, i.e., spontaneous generation is not allowed in this model. Also, points with P < 0 are not allowed. Negative populations do not make physical sense. Importantly, all points with P > 0 move towards the carrying capacity, C. That is, in the logistic model, all populations below the carrying capacity are increasing and all populations above the carrying capacity are decreasing. By analogy, in a healthy scientific community and publishing enterprise, the literature should be self-correcting. No matter what its starting point is, the literature should proceed to a state of correctness. We see this state of correctness as analogous to the carrying capacity in the logistic model.

Much of the current success of XPS is attributable to (i) a community of scientists who helped make it into the technique it is today, and (ii) vendors who have done a remarkable job creating high-quality, user-friendly instrumentation. Indeed, the ease of use and reliability of modern systems allow large numbers of researchers in many fields to routinely use XPS in their work. XPS is booming. Instrument manufacturers have never sold more systems, and there are many new users in the field. Thus, a current challenge in XPS is to train all these new people. In addition, modern instruments collect data faster (with higher count rates) than earlier machines, so more data is being taken than ever before. Historically, data acquisition was slow, so scientists had more time to think critically about their results. Today, more materials are being characterized more thoroughly than ever before. For example, XPS images can now be generated rather quickly. These sometimes contain thousands of spectra. In contrast, in the past, a surface might have been characterized at a few spots and the results averaged. Chemometrics/informatics methods will be increasingly important in understanding large XPS data sets.^59–65 In summary, because of significant advances in instrumentation, there are many new XPS users, many of whom are not well trained, and there is much more XPS data.

Some areas of experimental science, e.g., organic chemistry, only require a small number of analytical methods to adequately characterize the materials made for their applications. For example, many organic molecules, even rather large ones, can be fully characterized by a combination of nuclear magnetic resonance (NMR), mass spectrometry (often electrospray), Fourier transform infrared spectroscopy (FTIR), and elemental analysis. Of these, ¹H and ¹³C NMR play the largest role. Accordingly, many graduate programs in organic chemistry do an excellent job training their students in NMR and other spectroscopic methods. Because of the small number of characterization methods used in this field, organic chemists generally become very familiar with them. They have a shared competence and language that is beneficial to the field.

In contrast to the situation with organic chemistry, surface and material analysis relies on many techniques for material characterization. The following is a partial list of useful, and often very important, surface and material characterization methods: amperometry, atomic force microscopy (AFM), atom probe tomography (APT), Auger electron spectroscopy (AES), BET (Brunauer-Emmett-Teller) surface area analysis (typically with N₂), confocal microscopy, dynamic secondary ion mass spectrometry (SIMS), electron probe microanalysis (EPMA), electron spin resonance (ESR, also known as electron paramagnetic resonance, EPR) spectroscopy, energy-dispersive x-ray spectroscopy (EDX or EDS), extended x-ray absorption fine structure (EXAFS), fatigue testing, fracture toughness testing, FTIR, hardness testing, helium atom diffraction, high resolution electron energy loss spectroscopy (HREELS), indentation hardness testing, laser-induced breakdown spectroscopy (LIBS), low-energy electron diffraction (LEED), low-energy ion scattering (LEIS), matrix assisted laser desorption ionization (MALDI) mass spectrometry, mercury porosimetry, Mössbauer spectroscopy, neutron diffraction/scattering, nuclear reaction analysis (NRA), polarimetry, profilometry, proton induced x-ray emission (PIXE), quartz crystal microbalance (QCM), Raman spectroscopy, reflection high-energy electron diffraction (RHEED), reflectometry, rheology (this is an entire field), Rutherford backscattering spectrometry (RBS), scanning electron microscopy (SEM), scanning tunneling microscopy (STM), scatterometry, size-exclusion chromatography (SEC), solid-state NMR, spectroscopic ellipsometry (SE), static time-of-flight (ToF)-SIMS, sum frequency generation (SFG) spectroscopy, surface plasmon resonance (SPR), temperature programmed desorption (TPD), tensile testing, terahertz spectroscopy, transmission electron microscopy (TEM), ultraviolet photoelectron spectroscopy (UPS), ultraviolet-visible spectroscopy (UV-VIS), voltammetry, wetting (contact angle goniometry), x-ray absorption near-edge structure (XANES, also called near-edge x-ray absorption fine structure, NEXAFS), x-ray diffraction (XRD), x-ray fluorescence (XRF), x-ray photoelectron spectroscopy (XPS), and x-ray reflectivity (XRR). Most of these more than 50 techniques operate in multiple modes. While there is some overlap in the information provided by some of the techniques just listed, they generally provide different, unique, complementary bits of information about samples. Indeed, focused training and experience are generally necessary for a person to become an expert in even one of them. It is simply not true that modern instrumentation and analysis software have removed the need for careful learning of a technique and data analysis.

Multiple characterization methods are often necessary to understand new surfaces, thin films, and materials. For example, in a thin-film deposition, e.g., by atomic layer deposition (ALD), one might monitor the film deposition by in situ SE or QCM to confirm film growth/material deposition. Subsequently, one might:

1. use AFM to measure the film roughness,

2. one or more forms of electrochemistry to understand the film's electrical properties,

3. FTIR and/or Raman to understand its vibrational structure,

4. LEIS to determine which atoms are at the outermost layer of the material (this is important in catalysis),

5. one or more forms of mechanical testing to understand the film's mechanical properties, e.g., hardness,

6. ex situ SE to determine the film thickness, optical constants, and roughness (ex situ SEs often have greater wavelength ranges and more angles of incidence that in situ instruments),

7. SEM to probe the quality, uniformity, and morphology of the film,

8. static ToF-SIMS to confirm film composition (atom/molecular connectivity) and to identify chemical impurities/contaminants on the film,

9. TEM to obtain a cross sectional, atomic picture of the film,

10. XPS to determine elemental composition, film stoichiometry, and the chemical states of the elements in the film,

11. XRD to understand its crystallinity,

12. and/or XRR to obtain a less model-based understanding of the film thickness and its electron density.

Obviously, the need to be familiar with and to use so many different characterization methods places a burden on researchers (and institutions) that make and modify surfaces and materials. Indeed, in addition to the need to become an expert in a particular area, e.g., in battery science, thin film deposition, or corrosion, a researcher working with surfaces and materials needs to know how to correctly collect and interpret data from multiple analytical tools. Further complicating this situation is the fact that new materials are regularly synthesized in research laboratories, which may require new characterization methods that researchers and research groups are unfamiliar with.

Figure 8 shows a summary of information about XPS taken from the literature. Two trends/sets of results are illustrated in this graph. The first (see the right y-axis) is the considerable increase in the number of “XPS” papers published per year since 1990, where the rate of increase here shows no sign of decreasing. Indeed, “XPS” is now being mentioned in ca. 13 000 papers per year, where the rate of increase in “XPS” is about 1.5 times that of the increase in the literature in general.³ These data are fit reasonably well with an increasing exponential function. The second trend/set of results (see the left y-axis) shows the fraction of papers with “XPS” in them that appeared in 13 established surface journals (Journal of Vacuum Science and Technology A, Journal of Vacuum Science and Technology B, Surface Science Spectra, Langmuir, Surface and Interface Analysis, Journal of Physical Chemistry C, Surface Science, Applied Surface Science, Surface and Coatings Technology, Thin Solid Films, Colloids and Surfaces B, Surface Review and Letters, Progress in Surface Science). In 1990, ca. 25% of all XPS results were published in one of these surface journals. However, at present, only ca. 7% of the XPS data in the literature is published in one of them. Admittedly, part of this decrease may be due to the presence of new journals; more of the XPS data than is suggested in Fig. 8 may still be published in surface journals. Nevertheless, we are intrigued by the correlation between these sets of results. If the total number of XPS publications is modeled with two straight lines, from 1990 to about 2010, there appears to have been a ca. linear increase in the number of “XPS” publications and a concomitant linear decrease in the fraction of XPS papers appearing in surface journals. In about 2010, the correlation continued, but the trends accelerated—both slopes changed. It is not clear to us what took place around 2010. (It is also not entirely clear whether the number of XPS papers in Fig. 8 should be fit with two straight lines or an exponential. Either way, the growth in the technique is significant.) Obviously, neither the (linear) decrease in the number of “XPS” papers appearing in surface journals nor the ca. exponential increase in the number of papers that mention XPS can continue indefinitely. We anticipate that the number of XPS papers published in surface journals will asymptotically approach a value below 7%. It is not clear how much more XPS will grow before its use plateaus.

The current and increasing number of “XPS” papers in Fig. 8 indicate that XPS is a prominent and growing technique. The apparent decrease in the number of papers with XPS data appearing in surface journals in Fig. 8 is probably related to both a decrease in the number of experts involved in XPS data acquisition and the fact that more incorrect XPS data analysis is entering the literature. These data underscore the difficulty of effectively communicating with a diverse user base regarding XPS data analysis and improved practices.

While there are well-run XPS centers in the world that provide excellent support for their users, there are multiple examples of centers, even at top institutions, where minimal training and support are provided. The following describes how XPS is performed at many places in the United States, including at some top research institutions. Often, graduate students and post-docs are given an hour or two of instruction on how to use the instrument and another hour or two of instruction on how to analyze/work up the data. They are then expected to collect and analyze data themselves. A few hours of instruction on XPS data collection and analysis are not nearly enough to know how to avoid common pitfalls and errors. Nevertheless, these graduate students and post-docs apply the little knowledge they have received, often combining it with protocols/peak fits that they find in the literature. However, they generally do not know enough to determine whether the protocols they have found in the literature make any sense, or how to reasonably modify them if their analyses are not the same as what they have found. They then take their results and analyses back to their advisor, who often knows little about XPS, but trusts the training procedures at their institution, and the work is written up and submitted for publication.

One would hope that errors in material analysis (and other aspects of a manuscript) would be caught when papers are submitted for review. Certainly, some are. However, as suggested in Fig. 8, most XPS results are currently published in non-surface journals, which means that, in many cases, neither the editors nor the reviewers of these journals know enough about XPS to be able to spot errors in XPS data analysis. Of course, ideally, each characterization technique used in these papers would be reviewed. However, as noted above, it is not uncommon for five or more analytical techniques to be used to characterize a new surface or material. The reviewing process would be more cumbersome and challenging if reviews were necessary for every technical aspect of a manuscript, including the surface and material analysis in it. Of course, such an approach would lead to a higher quality literature. However, given the general fatigue among many expert reviewers because of the large number of manuscripts they are regularly requested to review, the lack of incentive for some journals to produce high quality papers (vide infra), and the lack of incentive for reviewers to review in general (vide infra), it is not clear that many more reviewers could (or would) be found for many of the papers being published, although a suggestion below may help in this regard. Accordingly, because editors (i) are often not experts in material characterization (their areas of expertise are often entirely different), (ii) the manuscripts they handle may present results from multiple surface and material characterization methods, (iii) they trust the system, and (iv) they are limited by the resources available to them, e.g., the number of reviews they can reasonably obtain for a manuscript, they often base their decisions on the opinions of a few experts in the areas of their journals who may not be able to identify errors in XPS and/or other surface and material analyses. The papers are then published, and the problem is perpetuated. We are forced to conclude that many manuscripts today that contain surface and material characterization data are not thoroughly vetted before they are published.

XPS analyses can be rather expensive. At many academic institutions in the United States, they cost on the order of $100 per hour. Of course, many cost centers rely on these fees, and these fees do not always prevent good analyses from being performed. In addition, modern instrumentation allows scans to be taken more rapidly (at higher count rates) and (sometimes) at higher resolution than ever before, which reduces the cost per analysis. Nevertheless, high instrument fees can (and do) limit access to instrumentation. Constrained by finite budgets, scientists may not be able to (i) perform preliminary experiments that can be important for understanding their samples, including how to acquire good data from their particular samples, and/or (ii) fully characterize their materials. For example, prior to a full XPS analysis it can be very helpful to determine: (i) the appropriate pass energies for narrow scans, where the pass energy controls the resolution of an analysis (to a point), (ii) the number of scans to acquire to obtain an adequate signal-to-noise ratio for a spectrum, (iii) the parameters associated with optimal charge compensation, which is very important for moderately complex and/or multicomponent samples that may differentially charge, and (iv) whether sample damage has occurred, which can be evaluated simply by ratioing scans performed at the beginning and end of an analysis.^42,66,67 There are consequences to not obtaining this type of information prior to (and after) data collection. If the pass energy is too high and/or the spectra too noisy, there will be greater uncertainty in peak fitting. Incorrect charge compensation can lead to shifted and distorted peaks, which adds additional complexity (and sometimes even confusion) in chemical speciation. If sample damage is not identified, the spectrum from a damaged material may be incorrectly assumed to represent the undamaged material. Unfortunately, some are of the opinion that such optimization is unnecessary with modern instrumentation, i.e., that samples can simply be inserted into modern instruments and that meaningful data will immediately be produced. Good XPS data collection often requires the type of optimization described in this paragraph. Some trial and error is usually needed to find appropriate data acquisition parameters.

Another problem with high instrument fees is that they can limit student access to instruments. Graduate students, post-docs, and even undergraduates benefit greatly from time on instruments. Given the degree to which questionable peak fitting and data are currently entering the literature, we should be looking for ways to increase the amount of time students have with instruments. Many instruments are unused during a significant fraction of their lifetimes. This time might be better spent educating the next generation of scientists.

In summary, high instrument fees can limit the amount of data that can be collected in a study, including for the determination of appropriate data acquisition parameters, which can lead to suboptimal results. High instrument fees may also limit the time students have on instruments, which may affect their training.

The Prisoner's Dilemma^68,69 is a behavioral model in game theory that may apply to our current situation. It describes a simple interaction between two people who, despite high benefits to cooperation, choose to betray each other, leaving both people worse off. The model is commonly taught in introductory economics courses and has been depicted in many crime shows and movies.

In a classical embodiment of the Prisoner's Dilemma, two prisoners, P1 and P2, have been arrested for a crime. They are put into separate interrogation rooms and offered the same deal. Their choices are to betray their partner by confessing to the crime or remain silent. If they both choose to confess, they will be prosecuted for the crime, and each sentenced to a fairly long time in prison (eight years in the example in Fig. 9). However, since P1 and P2 are the only witnesses of this crime, if they both choose to remain silent, they will not be prosecuted for the crime. Rather, they will be prosecuted for a minor, ancillary crime, and each sentenced to only one year in prison. If, however, one prisoner chooses to remain silent while the other betrays their partner, then the betrayer is rewarded with a plea deal and serves no time in prison, while the silent one is punished with 12 years in prison. These choices and outcomes are summarized in the payoff matrix in Fig. 9. We are assuming here that there is neither a strong personal relationship between the two prisoners nor a credible threat of retaliation that might influence their decisions. Accordingly, each prisoner will act in their own self-interest by reducing the amount of time they personally spend in prison.

What should P1 and P2 do? Because they have been separated, they do not know what the other will choose to do. Accordingly, P1, as a self-interested agent, analyzes the situation as follows. P1 might first imagine a scenario in which P2 betrays him. From the matrix, P1 sees that he receives less time if he also betrays his partner (8 years versus 12 years). However, another scenario is that P2 chooses to remain silent. In that case, P1 observes that he again receives less jail time if he betrays his partner (0 years versus 1 year). Thus, no matter what his partner chooses to do, P1 comes out ahead if he betrays his partner. Similar logic applies to P2. The outcome, then, is that the prisoners betray each other, receiving eight years each in prison. This result is notable because both prisoners would have been better off if they had both chosen to remain silent. (The story just told is not the only situation in game theory that has this type of structure.)

In general, there are two key features of a Prisoner's Dilemma scenario.

1. Each player has one action that is best no matter what the other player chooses. Thus, a player choosing this action is part of an equilibrium outcome.

2. This equilibrium outcome is socially inefficient, in that all players are worse off compared to the outcome where both choose the opposite/cooperative action.

We now argue that some of the poor surface and material data analysis in the literature, in addition to other undesirable issues currently facing the scientific community, can be modeled as Prisoner's Dilemma scenarios. Whether or not they are fully aware of it, professors, post-docs, and graduate students researching surfaces and materials are often faced with the following two choices. Either they spend the time to become experts in the multiple characterization techniques needed in their research and thus risk publishing less (analogous to the cooperative strategy of remaining silent in the Prisoner's Dilemma game), or they spend more of their time on research that leads to papers in their field (analogous to the non-cooperative strategy of betraying one's partner in the Prisoner's Dilemma game). Given that the rewards of the system, e.g., permanent positions, grants, tenure, awards, and prestige, depend on publishing papers in high-impact-factor venues, researchers are incentivized to spend less time on meaningful material characterization as long as the quality of their work is not overly scrutinized. In fact, this rewards system incentivizes scientists not to engage in any activity that takes them away from their research, including reviewing manuscripts, teaching undergraduate and graduate classes, mentoring students at all levels, and doing committee work in their departments. These incentives exist independent of what other researchers do, and the resulting outcomes are socially inefficient because no one's relative standing would change if everyone published less, while the overall quality of research and benefits to society would improve.

Just because we have observed that Prisoner's Dilemma-type scenarios may exist in science does not mean that we are condoning selfish behavior or celebrating their suboptimal outcomes. We commend those who, for example, selflessly work to benefit the scientific community and society by doing and publishing careful science, teach their classes with excellence, reach out to their communities, write high-quality reviews of papers, take undergraduates into their research groups, edit journals, and write standards documents (there are generally no authors listed on ISO and ASTM standards, or their equivalents in other countries). In general, these people perform these activities knowing that most of them will not help them directly with their research. The virtuous and selfless behavior of many scientists has probably made the current situation in science much better than it otherwise would have been. We hope such behavior will continue.

However, expecting people to behave virtuously, but against their own interests, will probably have limited effectiveness. The problem is not so much with the morality of scientists, as with the incentive structure they face. A Prisoner's Dilemma scenario creates a situation in which self-interested agents are incentivized to choose socially inefficient outcomes. In general, to avoid these outcomes, we must change the reward structure of the situation itself. For example, the Prisoner's Dilemma is used effectively by prosecutors to extract confessions from criminals. Nevertheless, the Calabrian Mafia, known as Ndrangheta, which is one of the most powerful organized crime organizations in the world, has been a challenge to prosecute.⁷⁰ Their lack of vulnerability to the Prisoner's Dilemma has been a result of their choice to selectively recruit and admit close relatives of current members into their organization. Thus, for a member of Ndrangeta, confessing to a crime is tantamount to betraying their family of origin. One could never return to one's family after testifying against a brother or an uncle, or, for that matter, against any member of an organization filled with close relatives. The price of betraying and losing their family of origin is higher than the additional time in prison they might endure. Thus, changing the fundamental structures of Prisoner's Dilemmas, not just calling for more selfless or virtuous behavior, is generally necessary to prevent their outcomes. We now discuss suggestions for the various stakeholders in surface and material characterization. We believe that some of these suggestions may change the reward structures and incentives in science to help improve our current situation.

As noted by Park et al.,¹⁰ some parts of “Materials chemistry and related fields commonly report new materials with limited attention paid to reproducibility.” In addressing reproducibility and reliability challenges, we seek to change parts of the research culture and expectations of the research community. We now discuss the roles that authors, research advisers, subject matter experts, reviewers, scientists who observe poor quality surface and material characterization, editors, journals and publishers, funding agencies, scientific societies, leaders and administrators at universities and research centers, and instrument vendors can play in improving the analysis of surface and material characterization data. In some cases, the suggestions below may change the reward structures of underlying Prisoner's Dilemmas.

We believe that authors are primarily responsible for the content of their papers, not reviewers or journal editors. The burden of publishing high-quality data and data analysis ultimately rests with the authors. Authors should be careful regarding the work they publish because, whether formally or informally, everything in the literature is always subject to retrospective analysis. History will judge authors based on the quality and accuracy of the work they publish. While there appear to be Prisoner's Dilemmas that incentivize the rapid publication of incomplete or inaccurate surface and material characterization and data analysis, we believe authors are shortsighted if they succumb to these pressures. Authors should be aware that the current situation is serious and help move the community away from a crisis by publishing high-quality, well-referenced work. The analysis presented herein suggests that many papers with a focus on surfaces and materials (and other areas of science) will be incompletely reviewed. Authors should write their papers understanding this reality. Authors should provide sufficient detail in their papers about data collection and analysis so that their work can be reproduced and evaluated by others. We noted above the challenge for scientists of using multiple surface and material characterization techniques in their work. Researchers must take the time to become familiar with the characterization techniques and other methods they use in their studies, or they should collaborate with those who possess this expertise. When teams of authors work together, all team members should take at least some responsibility for the work as a whole, not just the part they contributed to. The principles of characterization techniques like XPS that are needed in studies range from elementary to complex. However, just because an XPS peak fit or analysis looks satisfactory to the eye does not make it unique or even chemically meaningful.^71,72 There are complexities and tradeoffs in the designs of XPS experiments, and there is often complexity in the analysis of XPS spectra.^{19,41,42,73–75}

How can students (undergraduates, graduate students, and post-docs) be trained to recognize that the spectra in Figs. 1 and 6 contain no useful signal? Advisors play a central role in student training. Advisors should regularly discuss students' research results with them. Advisors should have regular, ad hoc conversations with students about science, e.g., when they visit their laboratories. Research group meetings should be designed to train students to think like scientists. Research advisors should help students understand why they have made errors in their data collection and/or interpretation if they produce results like in Figs. 1–6. Research advisors should be open to collaboration with other experts when that expertise is needed. Students benefit from working with multiple experts during their training.

A great deal of high-quality tutorial material has been produced about XPS. For many years, XPS experts have written tutorial articles,⁷⁶ monographs,⁷⁷ and ISO (ISO/TC 201 Surface Chemical Analysis)⁷⁸ and ASTM⁷⁹ standards. Databases containing hundreds of carefully acquired and vetted spectra are available, e.g., from Surface Science Spectra and XPSSurfA (hosted by La Trobe University).⁸⁰ Websites filled with useful information about XPS can be freely accessed, e.g., from ThermoFisher, Surface Science Western, and the UK National Research Facility for Photoelectron Spectroscopy.⁸¹ Hundreds of informative videos on the technique are available for free from trusted sources on YouTube, e.g., by Neal Fairley on CasaXPS. Short courses and workshops on XPS are also presented. For example, for the past decade a group of French XPS experts has organized an excellent, multi-day workshop on XPS for both beginning and expert users.⁸² This workshop will next be held in Wales under the direction of David Morgan. John Grant has an online course on XPS and CasaXPS. Cedric Powell organized 10 Topical Conferences on Quantitative Surface Analysis on a two-year cycle beginning around 1987. These conferences were held for one or two days before the start of the AVS National Symposium and were well attended. (They were sponsored by the AVS Applied Surface Science Division, ASSD.) Several subsequent conferences were organized by Fred Stevie and Dave Simons, and later by Tony Ohlhausen. Experts in other characterization techniques have similarly produced large amounts of tutorial information about their methods, e.g., the discussion of Kim et al. on the use of vibrational spectroscopy in glass analysis.⁸³ As noted above, guides (championed by Don Baer) covering many aspects of XPS were recently written by experts and published in the Journal of Vacuum Science and Technology (JVST).^{19,40–42,73–75,84–88} A second set of guides on both XPS and other surface analytical techniques is in progress.^36,89 We hope these experts will continue to make these contributions. They have been invaluable.

The proliferation of papers in the literature appears to be a result of a Prisoner's Dilemma situation. A larger number of papers for a scientist often leads to greater recognition and funding, e.g., scientists cannot have high h-indices if they have not published a lot of papers. However, this additional publishing may be (i) encouraging authors to publish incomplete or less impactful studies,^90–92 and (ii) taxing reviewers, making experienced reviewers harder to find. Nevertheless, this additional publishing may not always be bad. It is sometimes advantageous to have shorter papers that focus on specific results. It can be challenging to find results in longer papers. In addition, modern instrumentation produces much more information than before, so, perhaps, the amount of publishing should increase so that this additional information can be disseminated. In any case, better reviewing would help reduce the amount of poor XPS (and surface and material) data analysis in the literature. However, this solution appears to be somewhat impractical at present because the number of papers employing XPS appears to exceed the number of competent and willing reviewers. The following suggestion made by Baer, Watts, and Herrera-Gomez at the ECASIA meeting in Dresden, Germany, 2019 may help with this situation. Many scientists do not have enough time to review all the manuscripts they are asked to examine. However, instead of turning down a manuscript, a subject matter expert may wish to review only the part of the manuscript that relates to their expertise. This type of selective reviewing takes much less time than traditional reviewing and allows key portions of more manuscripts to be evaluated by experts. An XPS subject matter expert would have immediately identified the errors in Figs. 1–6. Of course, such reviews should make it clear to the editor that only a portion of a manuscript has been evaluated. In addition, we hope that the discussion above on the many surface and material characterization methods will make reviewers more willing to admit to editors when they are not experts in every aspect of every manuscript they are asked to review. Editors need this type of feedback.

Some have proposed that the reviewers of manuscripts be listed on the papers they have reviewed. It has even been suggested that the potential for public shaming might encourage reviewers to produce better reviews. We question the wisdom of this approach for multiple reasons. First, authors should not be able to shift the blame for producing poor-quality work to the reviewers or editors of their manuscripts. A list of the reviewers who reviewed a paper and the editor who handled it would more easily allow this to happen. Again, we believe that authors are primarily responsible for doing high-quality, careful work. Second, the potential for public shaming will probably deter some scientists from doing any reviewing at all, and the pool of willing and capable reviewers is already too shallow. Why would a scientist risk being embarrassed or even discredited over a matter (reviewing a paper) that provides few external rewards? Third, given the large number of surface and material characterization techniques (see the discussion above on this topic), and also the many other technical topics that may be touched on in a paper on surfaces and materials, it seems likely that reviewers and editors will not be experts in every area covered in the manuscripts they review or handle. It hardly seems fair to penalize or embarrass reviewers and editors for not being experts in everything.

In summary, selective reviewing may be an alternative, efficient way for expert reviewers to help reduce the amount of poor surface and material data analysis entering the literature, reviewers should make it clear to the editor if there are parts of a manuscript they cannot fully evaluate, and it is not clear to us that listing/revealing the reviewers and editor of a manuscript is a good idea.

Collegiality among scientists is highly valued. However, this strength of a community may turn into a weakness if the community ignores incorrect data and data analysis (see our discussion above about being in a state of pre-crisis). Scientists who notice examples of faulty data analysis should apply the following suggestions with tact.

• Reviewers and editors should insist that only good surface and material data and data analysis are published. The best and easiest time to identify and correct poor data and data analysis is before it is published. Reviewers and editors are in a strong position to “get tough” with authors who try to publish poor quality work. If a reviewer believes that a study is of poor quality and that it should be rejected, this reviewer should recommend that the work be rejected and provide an appropriate rationale for this recommendation. It is not helpful for reviewers to recommend a “major revision” when they believe a manuscript should be rejected.

• After reading a published paper with an incorrect analysis in it, a scientist may wish to send an email to the authors alerting them to the problem.

• In some cases, scientists may wish to submit a “Comment” or “Short Communication” to a journal to explain issues associated with a faulty analysis. However, such action may be undesired by the journal, and some authors may respond to such criticism/correction in a less-than-friendly manner. Of course, we hope authors will graciously accept such feedback if it is appropriate.

• Scientists may wish to point out that an analysis they observe in a presentation is incorrect. This might be done in the question-and-answer session of a talk, in a conversation with a presenter after an oral session, directly to the presenter of a poster, or as a follow-up email to the authors.

The large number of surface and material characterization methods (and other technical areas in surface and material research) affects journal editors in the same way it affects authors and reviewers. Most editors will probably only be deeply familiar with one or two material characterization techniques and one or two technical areas in their subdiscipline. Accordingly, editors rely on expert reviewers for many of their decisions. As suggested above, it is not clear to us that potentially shaming editors by listing them on the papers they handle will improve the current situation. The scientific community needs good journal editors, and such an approach may deter some qualified people from taking on this role. Nevertheless, we believe editors can do the following things to help improve the current situation.

• Editors should exercise their authority to make sure that high-quality work is published (see Sec. II H 5).

• Editors can watch for trends related to poor surface and material data analysis in the manuscripts they handle and communicate these issues to the other editors of their journal, their editor-in-chief, and the community at large.

• Editors can consider using subject-matter reviewers to selectively review portions of manuscripts.

• Editors should be selective in the papers they edit. Many journals have multiple editors with different areas of expertise. Editors should be able to transfer a manuscript to a different editor if they feel they do not possess the expertise needed to handle it.

• Editors need to stay up to date in their areas by attending conferences and learning about the characterization methods relevant to their journals. Of course, many editors are active researchers who already do this.

Journals can play a valuable role in reducing the amount of incorrect data and data analysis entering the scientific literature. In particular, journals and publishers can:

• Choose editors and reviewers who are experts in their fields, including in the material characterization techniques that are relevant to their journals.

• Ask reviewers to verify that specific figures and tables in manuscripts are correct (perhaps by clicking boxes in the review form), or that they do not possess the expertise needed to evaluate them.

• Commission and publish guidelines for authors on the characterization methods that are most relevant to their journals. A lack of adherence to these standards could be grounds for a manuscript to be rejected.

• Modify the software their editors use so that requests for specific/targeted reviews on portions of manuscripts can be more easily generated and sent to potential reviewers.

• Be realistic about the amount of time editors have for their work. “High-visibility” scientists and scholars often become editors of journals. However, some of them may not have the bandwidth to do high-quality editing of large numbers of manuscripts on top of their responsibilities to teach and run large research groups. Perhaps an editorship should be a full-time job, instead of a prestigious service to the community. In some ways, program officers at funding agencies and editors do similar work, but program officers generally work full time for a funding agency.

• Provide training to their editors to better understand their fields, including in material characterization, where relevant.

• Be open to the possibility that more papers with blatantly incorrect data analysis in them need to be retracted.

Of course, journals and publishers are limited in their abilities to improve the current situation. There are multiple journals with different standards, including so-called “predatory,” open-access journals. The existence of the latter journals reduces the control that journals from a non-predatory publisher have over the rewards in science. Indeed, some of us have been surprised (and even dismayed) at how easy it has been to publish in some of these open-access journals. Certainly, prior to the open access movement (and even now), there were significant problems associated with the high (some would say exorbitant) fees charged by publishing companies and professional societies for their content. Nevertheless, at least the previous model incentivized publishers and professional societies to create high-quality content that they could sell for many years to come. Copyrights last a long time. A publishing company that it is paid upfront when papers are published has less incentive to encourage excellence in editing and reviewing. Minimal editing and reviewing of open access papers will probably make more money for a publishing house in the short term. Nevertheless, we think that such behavior is short sighted. Ultimately, we do not see much difference between poorly or minimally reviewed papers and papers that are placed in a repository without any review at all. Why have journals, whether open access or not, if they are not going to create a high-quality product for the scientific community?

Because funding agencies largely control the funding of science, they essentially control the rewards of the scientific endeavor. That is, of all the stakeholders and influencers in science, funding agencies may be in the strongest position to change the reward structures of the Prisoner's Dilemma models that may be influencing behavior. We suggest that funding agencies consider the following ways of improving the surface and material characterization in the work they fund.

• Material characterization should be specifically addressed in proposals dealing with material synthesis. Proposals to make materials should demonstrate that newly synthesized materials can be appropriately characterized. How can progress in advanced materials take place if researchers are misled regarding their composition?

• Experts in material characterization may need to be co-investigators in large-scale materials synthesis efforts, especially when complicated materials are prepared.

• When deciding to fund major research instrumentation, funding agencies should give preference to institutions that can show/project the most student and post-doc use of the equipment. Lower instrument fees may be helpful in making this case.

• Funding agencies should support workshops in surface and material characterization that are directly related to their missions. For example, the XPS workshops in France mentioned above were supported by CNRS (The French National Center for Scientific Research).

• Audits are performed on essentially all aspects of all organizations. Audits are used to ensure that organizations perform their prescribed work in appropriate and effective ways and that they are accountable to their stakeholders. We find it strange that funding agencies receive large amounts of money from governments, i.e., politicians and voters, but that an important product of their efforts (the papers published by the research groups they fund) is not audited. We ask whether they should be audited. Perhaps, funding agencies and the researchers they fund have convinced their stakeholders that such audits are unnecessary, i.e., that those doing the work can be trusted to publish high-quality results. Poor quality data and data analysis in the literature proliferate incorrect data analysis and reduce the credibility of the scientific endeavor. The scientific enterprise must be careful in this regard. Many non-scientists are already very critical of it.

• Funding agencies may need to be more deliberate in letting research groups know that their ability to obtain future funding depends on them publishing good science, including good surface and material characterization.

• Funding agencies should look for ways to reward cooperative behavior that benefits students and society.

Scientific societies can play an important role in improving the current situation. These organizations should monitor the content and quality of the work published in their journals. Scientific societies may be more effective in improving the current situation than commercial journals because of their need to create high-quality content for their members. If the quality of surface and material characterization published by members of the society is questionable, scientific societies can (i) sponsor short courses on surface and material characterization at their meetings, (ii) organize sessions at their meetings that focus on surface and material characterization, (iii) invite experts in material characterization techniques to speak at their meetings, (iv) encourage their members to work with surface and material characterization experts, rather than relying on poorly trained students or post-docs (vide supra), and (v) commission committees of members to monitor the quality of the data appearing in their journals and make recommendations on how to improve it. Overall, scientific societies are in a strong position to positively influence authors, reviewers, editors, subject matter experts, and even funding agencies and journals.⁴⁶ 

Department chairs, deans, and directors of colleges and institutes are responsible for creating environments where high-quality research can be performed. They must ensure that necessary instrumentation is purchased and maintained, and that the expertise needed to run it and analyze its data are present (see the discussion above about the inadequate/limited amount of training given to graduate students and post-docs at many institutions). We now discuss some issues that these leaders should be aware of.

• The microscopists at one of our institutions shared the following. These staff members, who have PhDs and extensive experience in their fields, report that graduate students, often with very little experience in microscopy, regularly come to them with difficult problems from their research. These problems usually require the microscopists to search the literature and devise creative solutions to solve them. However, once a microscopy problem has been resolved and the necessary data are collected, the graduate student returns to their research group after having “solved” it, and no credit is given to the microscopists for the significant intellectual contributions they have made. We are concerned that people will not take, keep, or be fully satisfied in their jobs as staff scientists if they are not treated like professionals and fairly given authorship or acknowledgments on papers they have contributed to. These people are on the front lines of surface and material characterization. We need them.

• No professor or research group can possess expertise in every area. However, some institutions discourage collaboration and multiple authors on papers. We believe that collaboration with other experts needs to be viewed more favorably at some institutions, including in their rank and status evaluations. Policies or views disfavoring collaboration discourage faculty members at all ranks, and especially junior faculty members, from obtaining the technical assistance they need from experts in other areas, including from experts in surface and material characterization. Statements of attribution can be used to ensure that faculty members and their groups make strong intellectual contributions to the papers they publish.

• Administrators can change the way faculty members are evaluated at their institutions by rewarding, to a greater extent, behavior that benefits students and the community, i.e., they can change the reward structures of Prisoner's Dilemmas. However, the reward structures of these Prisoner's Dilemma models are widespread, both within a university and at a broader scale between universities. Universities that require more teaching and outreach from their professors put their faculty members at a disadvantage in seeking external funding compared to faculty members at universities that have fewer expectations. Meaningful changes along these lines may need to come through funding agencies.

• While XPS instruments are expensive to purchase and maintain, and most organizations are constrained by their budgets, per the discussion above, administrators should understand that high instrument fees may (i) contribute to incomplete and/or poor data collection and analysis and (ii) reduce the quality of student education.

• Graduate courses on surface and material analysis should be created and maintained. They provide in-depth training for students beyond what is possible in most short courses.

Thanks to decades of development, XPS instrument quality and reliability are very high. In addition, because of advances in automation, operation of modern instruments can be relatively simple. However, anecdotally, XPS instrument vendors report there has been a significant shift in the knowledge (related to XPS) of those purchasing their instruments. Ten years ago, systems were largely sold to experts. Now, they are mostly sold to newcomers in the field. Ten years ago, the training at an installation focused on advanced features of an instrument, whereas today it focuses on the basics. This anecdotal information from instrument vendors may or may not be related to the change in the slopes about 10 years ago in Fig. 8. Also, it is likely these trends pertain to other surface analysis techniques as well. In any case, there now appear to be many more XPS systems in the world than experts. It would be a shame if poor-quality surface and material data analysis in the literature discredited a surface characterization technique such that the need to buy or use the equipment was diminished. Accordingly, we believe it is in the best interest of instrument vendors to ensure that their customers understand both how to properly collect and analyze the data from their instruments and how to maintain and calibrate their instruments. XPS instrument vendors can help improve the quality of XPS data being collected and XPS data analysis by:

• Making sure their technical employees are well trained in the technique. Instrument manufacturers can be important repositories of know-how about surface and material characterization.

• Holding short courses and seminars to train their customers. (We hope that their customers will regularly request training to create a demand for it.)

• Supporting workshops and short courses run by professional societies and experts.

• Collaborating with other scientists—both with experts to support their efforts and non-experts to help them perform their data analysis correctly.

• Being clear about the limitations of the data analysis approaches/software they include with their instruments.

• Providing instruction and checks/safeguards in their analysis software for users. For example, their software could warn a user if the full width at half maxima (FWHM) values of the peaks in their fits were too small, too large, or too varied, or if the baseline in a fit was cutting through the data.

• Clearly recording and presenting parameters used for data acquisition and analysis in a way that encourages users/analysts to report them.

Finally, although XPS data can often be obtained quickly, and sample introduction and removal, and also instrument control, can be straightforward, instrument manufacturers should not imply to future customers that the technique is simple or trivial. XPS data acquisition (please refer to the discussion above on finding appropriate parameters for data collection) and data analysis are often challenging.

We believe that improving the current situation will require the efforts of multiple stakeholders. We are optimistic that these suggestions will help improve it. They may help change reward structures that incentivize poor outcomes.

Much incorrect data analysis has been entering the scientific literature. Indeed, there are so many flawed analyses in some areas that it is becoming difficult for non-experts to differentiate between good and poor work. Incorrect analyses are being copied and cited.⁹³ The surface and material characterization community appears to be in a “pre-crisis” state that may escalate into a full crisis if action is not taken. The presence of thresholds in biological systems, e.g., in the modified logistic model, below which populations of organisms collapse, makes us ask whether the concentration/abundance of good data analysis in the literature could fall to a point that the literature would be useless to non-experts. Advances in XPS instrumentation have made the technique available to a broad community of scientists. The large number of surface and material characterization techniques presents a challenge to scientists that synthesize and analyze surfaces and materials. The falling number of “XPS” papers in surface-focused journals suggests that fewer experts are involved in the collection and analysis of XPS data. The training of graduate students and post-docs in surface and material methods is inadequate in many places. High instrument fees may limit both the quantity and quality of XPS data that are collected. Various parts of the current scientific publishing system suffer from short-term driving forces that motivate behaviors that are often against the general interests of the community. Such behaviors can be described using a Prisoner's Dilemma model. Accordingly, there need to be appropriate incentives and reward structures to encourage both rigor and appropriate benefits to society. We believe the actions of multiple stakeholders of this problem, including authors, research advisers, subject matter experts, reviewers, scientists who observe poor quality surface and material characterization, editors, journals and publishers, funding agencies, scientific societies, leaders and administrators at universities and research centers, and instrument vendors, can improve the current situation. We all have a role to help set a high bar for quality.

We believe it is appropriate to acknowledge the countless hours devoted by members of the surface and material community to deal with the concerns raised in this work.

The authors have no conflicts to disclose.

G.H.M. and J.W.P. contributed equally to this paper.

George H. Major: Conceptualization (equal); Investigation (equal). Joshua W. Pinder: Investigation (equal); Writing – original draft (equal). Daniel E. Austin: Investigation (equal); Writing – original draft (equal). Donald R. Baer: Investigation (equal); Writing – original draft (equal). Steven L. Castle: Investigation (equal); Writing – original draft (equal). Jan Čechal: Investigation (equal); Writing – original draft (equal). B. Maxwell Clark: Investigation (equal); Writing – original draft (equal). Hagai Cohen: Investigation (equal); Writing – original draft (equal). Jonathan Counsell: Investigation (equal); Writing – original draft (equal). Alberto Herrera-Gomez: Investigation (equal); Writing – original draft (equal). Pavitra Govindan: Formal analysis (supporting). Seong H. Kim: Investigation (equal); Writing – original draft (equal). David J. Morgan: Investigation (equal); Writing – original draft (equal). Robert L. Opila: Investigation (equal); Writing – original draft (equal). Cedric J. Powell: Investigation (equal); Writing – original draft (equal). Stanislav Průša: Investigation (equal); Writing – original draft (equal). Adam Roberts: Investigation (equal); Writing – original draft (equal). Mario Rocca: Investigation (equal); Writing – original draft (equal). Naoto Shirahata: Investigation (equal); Writing – original draft (equal). Tomáš Šikola: Investigation (equal); Writing – original draft (equal). Emily F. Smith: Investigation (equal); Writing – original draft (equal). Regina C. So: Investigation (equal); Writing – original draft (equal). John E. Stovall: Formal analysis (supporting). Jennifer Strunk: Investigation (equal); Writing – original draft (equal). Andrew Teplyakov: Investigation (equal); Writing – original draft (equal). Jeff Terry: Investigation (equal); Writing – original draft (equal). Stephen G. Weber: Investigation (equal); Writing – original draft (equal). Matthew R. Linford: Conceptualization (lead); Formal analysis (lead); Investigation (lead); Methodology (lead); Project administration (lead); Resources (lead); Supervision (lead); Validation (lead); Writing – original draft (lead); Writing – review & editing (lead).

The data that support the findings of this study are available from the corresponding author upon reasonable request.