Introduction to topical collection: Reproducibility challenges and solutions with a focus on guides to XPS analysis Free

This article introduces a collection of papers that has been assembled with a focus on Reproducibility Challenges and Solutions (RCSs) related to the science and technology interest of the American Vacuum Society (AVS). Because of the importance and growing rate of use, many of the papers provide information related to surface analysis using X-ray Photoelectron Spectroscopy (XPS). The purpose of the RCS collection is to describe causes of nonreproducible data and analyses, to help analysts, researchers, and reviewers recognize problematic and erroneous data, and to provide guides and related information regarding best practices with the aim of assisting analysts in conducting and reporting reliable and useful information.

Surface analysis has become increasingly important in many areas of science and technology during the past few decades, possibly due to a growing awareness of the relationships between interfacial properties and the performance of materials. The availability of equipment to measure surface chemistry has increased and the most used technique for this purpose is XPS. Over the past twenty years, the growth in the use of XPS has been phenomenal, and this is a testament to the utility of the method and the unique insights it provides in the analysis of materials. While the method is used in many large and innovative industries for product development and the identification of causes of failure, it is also used in an increasing number of academic studies to test hypotheses and provide information to support conclusions. The graph shown in Fig. 1 demonstrates this academic trend, the data are drawn from a Web of Science search for papers that mention “XPS.” This search does not capture all papers that are concerned with, or use, XPS and it captures some papers that have nothing to do with the technique. Nevertheless, a series of additional searches and inspection of the fraction of unrelated papers demonstrates that the figure provides a broadly correct picture of exponential growth. The average annual growth rate in papers mentioning XPS is 8%, and this has led to the number of papers increasing from approximately 2000 in the year 2000 to over 10 000 per annum in the past few years. This is an amazing achievement for a technique that was already relatively mature in the late 20th century. Along with increasing awareness of the important surface and interface information it provides, this success has been enabled by the efforts of manufacturers who have made the instruments easier to use, more reliable, more adaptable, and automated.

Reproducibility and nonreproducibility are both important to scientific and technological advances. Issues and concerns associated with nonreproducible investigations have received increasing attention in the past few years.¹ Measurements and studies that are not reproducible are not necessarily of no value, they indicate that there are important unknowns or unidentified variables that when understood might stimulate scientific or technology advances. In a National Academies study,² these studies were identified as having “helpful nonreproducibility.” The study also observed that there is “unhelpful nonreproducibility” that may be “due to shortcomings in the design, conduct, and communication of a study. Whether arising from lack of knowledge, perverse incentives, sloppiness or bias, these sources of non-replicability reduce the efficiency of scientific progress; time spent resolving non-replicability issues that are found to be caused by these sources is time not spent expanding scientific understanding.” Within the AVS, some specific efforts are being undertaken³ to address items of unhelpful nonreproducibility that are appearing in many areas of science.^1,4–7

The papers in the RCS collection evolved from multiple threads of activity within the AVS: anecdotal concern about the quality of some of surface-related data and analysis appearing in the literature; discussions and presentations associated with Quantitative Surface Analysis Topical Conferences; a survey of AVS members related to quality and ways it might be improved;⁸ and a focus topic on New Challenges to Reproducible Data and Analysis at the 66th AVS International Symposium. In a 2018 survey of AVS members,⁸ 65% of those responding identified reproducibility as a problem in their research and indicated that guides, tutorials, and standards would be useful in helping address reproducibility challenges. To address this need, the RCS collection of papers includes identification of challenges and solutions associated with specific application areas such as semiconductor materials, nanoparticle characterization, and film deposition and guides, tutorials, and perspectives related to surface spectroscopy, primarily XPS.

To facilitate discussion, the 27 papers in the collection have been given a topic related identifier or label that will appear along with the reference number in this introduction to help organize and identify the topics covered as shown in Table II in Sec. III on solutions.

Each of the papers in the collection addresses solutions in the context of the challenges relevant to the paper topic. However, three of the papers are highly focused on identifying practices that contribute to nonreproducibility. These papers address non-reproducibility in atomic layer deposition (ALD),⁹ long term repeatability that is needed for development and manufacturing of semiconductor materials,¹⁰ and the frequency and types of errors in published XPS data.¹¹ 

Difficulties associated with reproducibility of ALD film growth are discussed by Sønsteby et al. (APP-2).⁹ In contrast to common expectations, ALD film growth can in practice be difficult to reproduce with results varying substantially between ALD reactors and among laboratories. The authors foresee nonreproducibility increasing as ALD is adopted by more researchers and integrated into new applications. Their paper highlights some of the major sources of variations, errors, and misconceptions with a focus on issues related to precursors, substrates, and deposition tools. The challenges are illustrated by examples from the literature. Their hope, consistent with the purpose of the RCS collection of papers, is that by educating newcomers and advocating for consistent reporting of deposition conditions, it will be possible to minimize nonreproducibility and enable ALD practitioners to realize the full potential afforded by self-limiting surface chemistry.

Conard et al. (APP-1) described the importance of surface analysis and the challenges of long term repeatability and reproducibility in a characterization facility supporting process development and high volume manufacturing in the semiconductor industry.¹⁰ They note three sources of repeatability and reproducibility problems: instrumental variation, sample variability, and operator decisions. Effectively, a laboratory needs to deal with instruments and operating systems that change with time, samples that can become contaminated or damaged, and operator decisions that may include how the data are analyzed, peak fitting methods, and the statistical treatment of data. Consistent with another theme of this collection, they show that an understanding of the characterization techniques used is paramount to understanding and dealing with these challenges.

Careful examinations of data in publications have identified problems with data quality and analysis in multiple areas of research, including those associated with materials analysis. It is not uncommon to find that 20%–30% of the data or analysis is identified as faulty to the extent that conclusions stated in the publication may be conpromised.^12–14 Major et al. (PLN-5)¹¹ have assessed the quality of XPS data appearing over a six month time period in three major journals. They find that the frequency of major errors is roughly 30%, consistent with reports for other types of measurements.

They also looked in some detail at the nature of the problems that were occurring and found that peak fitting is a major source of significant errors. Roughly 60% of the papers showing XPS spectra included peak fitting and major errors were identified in 40% of these papers. Issues included how the background was handled, inconsistent or incorrect peak widths, and inadequate display of the fitting results. It was found that peak fitting was most common for C 1s and O 1s photoelectron peaks and that, although such peaks seem relatively simple to interpret, inappropriate fitting was common.¹¹ To assist analysts, editors, and reviewers in judging XPS analysis the authors have included a list of common errors in XPS analysis as an appendix. The observation that C 1s and O 1s curve fitting was a significant common error source that led to the development of a guide for interpreting the C 1s spectrum (IDQ-7).¹⁵ One of the frequent errors is inadequate consideration of the physics and chemistry of the photoelectron peaks. These include mistreating the doublets created by spin–orbit splitting, as demonstrated in Fig. 2.

The three problem-focused papers just described identify several high-level challenges as summarized in Table I. Our purpose here is not to review the growing literature discussing the multiple drivers or causes of reproducibility issues,^{2–4,14,16–18} but observe that most researchers want to produce high quality research results and that because of the need for use of multiple techniques in many areas of advanced materials (and other) research, research teams often do not have expertise in all of the needed analysis tools and can benefit from the types of guides and tutorials recommended in the AVS survey.⁸ The purpose of the RCS paper collection is to provide useful information that can assist and encourage publication of high-quality, informative and reproducible data so that any nonreproducibility is of the useful type.

The use of XPS has grown dramatically in the past couple of decades to become the most widely used surface analysis tool, producing highly important information needed in many areas of science and technology.¹⁹ The success of XPS is based, in part, on the development of high quality highly stable instrumentation that can produce highly precise and reproducible data for samples that have been appropriately prepared and are stable in vacuum and under x-ray beam exposure. However, even for stable instruments properly set up, the conversion of XPS data to the desired information is usually not as simple as often assumed, and operator and analyst decisions play a major role in the quality and reproducibility of the results and conclusions. As confirmed by the study of XPS data in the recent literature (PLN-5),¹¹ along with the growth in its use, there is increasing appearance of misinterpreted or erroneous data. The guides, tutorials, or perspectives in the reproducibility collection are intended to enhance the ability of XPS users with different backgrounds and interests to recognize and produce good quality XPS data that are relevant to their areas of research. Even papers in the collection that are not focused on XPS include some type of surface characterization.

To assist the reader in accessing the paper most relevant to their questions or interests, the papers in the RCS collection have been organized by their main topical areas as shown in Table II. These areas address stages of an XPS experiment that are discussed in the first steps guide (PLN-1).²⁰ Readers learning XPS may want to start with the first steps (PLN-1) and introduction to XPS (PLN-2) papers as they will provide important background information and enable selection of the specific topics of relevant interest in other papers. As would be expected, papers with introduction or guide in the title start with useful introductory information to the topics discussed.

The papers address the general challenges identified in Table I as well as the specific topics listed in Table II. Because most papers also address related topics, an expanded table (Table III) in the  Appendix identifies both primary and secondary topics addressed in each paper. Most of the papers address issues at the introductory level, usually starting at the basic level but often progressing to some discussion of issues or nuances of the technique important for some types of analyses. The majority of papers also provide guidance and show examples, some examples of good practices, and others of inappropriate practices. The latter are clearly indicated. Using the material in Table II and the  Appendix Table III, it is anticipated that readers can identify the paper or papers that address their specific needs.

As noted in the first steps guide to applying XPS (PLN-1),²⁰ before doing an XPS measurement, it is important to identify the analysis objective and determine if XPS can provide the required information. An XPS measurement involves several stages including planning, making the measurements, analyzing the data to extract the information, and reporting. The first steps guide (PLN-1) focuses on sources of information and identifying questions to be addressed during the planning stage. It is not uncommon for researchers to consider XPS measurements knowing little about the technique and the first steps guide, along with the introduction to XPS, can help them ask the right questions and avoid wasting time if XPS is inappropriate. The introduction to XPS (PLN-2)²¹ provides a basic introduction to XPS for prospective or novice users. In addition to providing a basic introduction it identifies, as cautionary or warning notes, issues or topics that when looked at in detail are more complex than simple introductory explanations can convey.

Sample preparation and mounting are both critical to successful XPS measurements because of the surface sensitivity of XPS and must be part of planning before studies are initiated. Methods to appropriately handle, prepare, and mount samples are described and illustrated in a paper by Stevie et al. (PLN-3).²² Standards committees from ASTM International (ASTM Committee E42 on Surface Analysis) and the International Organization for Standardization (ISO Technical Committee 201 Surface Chemical Analysis) have developed standards and guidelines for many of the topics important for reproducible surface analysis measurements and these documents are useful for planning, measurement, and analysis and are described in PLN-4.²³ Finally, information about common errors observed in XPS data in publications provides useful knowledge for users so that they can avoid such errors in experimental measurements, analysis, and reporting XPS results (PLN-5) as well as when they review work that contains XPS data.¹¹ Learning how to recognize issues is an important aspect of enhancing the reproducibility of XPS results.

It is not possible to make reproducible measurements without ensuring that the measurement system is set up and operating properly. For XPS, this includes determining that the energy scale and signal sensitivity are appropriately calibrated and consistent. A simple, easy instrument check is described by Wolstenholme (ISU-1)²⁴ built around an ISO standard 19830. Supplementary material to this paper includes a working spreadsheet that can be used to analyze instrument performance data described in the paper and keep a record of that performance. Consistent instrument transmission and sensitivity can be verified as described by Wolstenholme, but additional understanding and information is needed to enable comparison with “ideal” spectra or spectra collected for different instruments or different angular configurations or the same instrument or different instruments.

The papers by Reed et al. (ISU-2)²⁵ and Shard and Reed (ISU-3)²⁶ describe a new approach for determining spectrometer response functions based on the comparison of a survey spectrum of low density polyethylene to a model survey spectrum. Paper ISU-2 reports part of the effort to establish the reliability of this process for consistent and traceable intensity calibration across the community of XPS users. It contains a method by which the intensity scale of an instrument can be validated and is therefore useful in a series of articles on XPS reproducibility. Paper ISU-3 contains some essential information for ISU-2 but also contains a lot of detail about topics such as photoemission distributions, x-ray polarization, elastic scattering, and how these affect XPS spectra. This detail may be of interest to some readers and is worth reading if a comparison of data between different instrumental setups, for example, a laboratory-based instrument and a synchrotron beamline, is being considered. The information provided enables meaningful comparison of data taken using different settings of the same instrument, using a different instrument, or using theoretical calculations. Without such calibration, the application of “standard” relative sensitivity factors can lead to serious errors and inconsistencies in determined elemental concentrations.

Increasingly, XPS is used to analyze small regions of materials, and for such measurements, it is important to set up the instrument appropriately with adequate spatial or area resolution. The concepts and background for this is described by Unger et al.²⁷ in ISU-4. It is important to understand, for example, that most measurements of spatial resolution give values much smaller than the area from which signals are accepted. Approaches to control charge build up on samples during the measurement and ways to appropriately adjust the binding energy scale are important for insulating materials. Approaches to identify sample charging, methods to control or minimize it, and binding energy scale adjustments are described by Baer et al.²⁸ in ISU-5. Here, it is important to note that the common approach of adjusting the binding energy scale to set the C 1s peak position to a defined value, assuming the presence of adventitious carbon, has serious shortcomings in relation to absolute energy scale determination (ISU-5, IDQ-7). Regardless, in the day-to-day workings of an analytical laboratory, this approach has its place because it can be a straightforward and effective method, occasionally the only method available, for useful energy scale adjustment, allowing identification and interpretation of other peaks collected during an experiment. It must always be remembered that this “adventitious carbon reference” approach is neither precise nor accurate and is utterly inappropriate for samples that are electrically conductive.

Because of the importance and associated challenges, many of the papers in the RCS collection deal with analysis issues. Included are a tutorial on peak identification and guides for quantification, backgrounds in XPS spectra, curve fitting, and interpreting C 1s spectra. Additional papers related to quantitative accuracy and peak fitting uncertainties provide additional perspectives on these important topics.

Through a series of questions to the readers, a tutorial on peak identification for survey spectra works through processes that an analyst might follow to identify major and minor peaks (IDQ-1).²⁹ The guide to quantification (IDQ-2)³⁰ provides an introduction and overview of all of the components essential for quantification starting with instrument setup and the nature of the sample and moving through other essential issues such as sensitivity factors, peak fitting and, highly important, what needs to be reported. With modern instruments, XPS data can be robust and highly precise. In a perspective paper, Brundle and Crist (IDQ-3)³¹ examine some of the challenges and issues that can limit the accuracy in turning spectral signals into quantitative elemental concentration. They note the impacts of peak complexity, determining the intensity of satellite peaks, and the differences between theoretical and experimentally derived sensitivity factors. A follow on paper demonstrating some of these issues with the seemingly simple example of LiF as a specific example has been published.⁴⁵ 

Three papers deal with spectral backgrounds and aspects of fitting XPS photoelectron peaks. The introductory guide to backgrounds (IDQ-4)³² introduces the types of backgrounds observed in XPS spectra, the background models used when determining peak intensity and for peak fitting and then uses examples to demonstrate how the use of different background models impact the relative intensity of photoelectron peaks. Some of the example spectra used in this paper are published in Surface Science Spectra and available in a digital form so that analysts can process the example data and compare their analysis approaches to those discussed in the paper.⁴⁶ 

The practical guide to curve fitting (IDQ-5)³³ addresses issues associated with several literature problems including the physics and chemistry involved in generating XPS spectra and good practices for peak fitting. It also provides examples of appropriate use of peak fitting information along with tools for avoiding mistakes. A second fitting paper discuss uncertainties in peak fitting (IDQ-6).³⁴ 

Because of the importance of peak identification and fitting of C 1s spectra described in the common errors paper (PLN-5),¹¹ a guide to the interpretation of C 1s peaks was prepared (IDQ-7).¹⁵ It is hoped that this guide will be used to avoid many of the most frequent errors found in the fitting of C 1s spectra. This paper is derived from long experience in working with these peak features and provides many useful examples. The final paper in this topical area focuses on extracting quantitative information from the Auger electron spectra from Cr containing oxides for which interference between the O KLL and Cr LMM Auger electron peaks complicate quantification (IDQ-8).³⁵ 

The surface sensitivity of XPS arises from the distances that electrons travel in a sample before losing their characteristic identify. Formalisms for quantitative analysis are impacted by the distances that electrons travel.³⁰ The inelastic mean free path (IMFP) is the term that is most often identified as the analysis depth of an XPS measurement. However, as Powell (DPH-1)³⁶ notes, IMFP does not appropriately include the impacts of elastic scattering on electron spectroscopy and three additional terms have been defined to identify concepts and information needed to address the impacts of elastic scattering in different analysis situations. The IMFP is a material dependent quantity, while other parameters, which rely on IMFP values [effective attenuation length (EAL), the mean escape depth (MED), and the information depth (ID)], vary with the instrument configuration and applications. Powell notes that these terms are often applied incorrectly, and his guide is designed to clarify the distinctions and appropriate applications.

Accurate measures of the appropriate path lengths are important for quantitative analysis. Although they can be calculated theoretically, accurate measurements are important to test and identify the limits of the models. Chambers and Du (DPH-2)³⁷ describe how attenuation lengths can be obtained with high accuracy through the use of the well-defined film thickness and abrupt interfaces of epitaxially grown films.

As noted in the introductory guide to backgrounds³² in XPS, an important component of background signals arises from inelastically scattered electrons. Standard XPS quantitative analysis assumes, usually incorrectly, that the elements are uniformly distributed along the surface and within the analysis depth. In a guide to the use of background signals in quantitative XPS, Tougaard (DPH-3)³⁸ shows how background signals can be used to quantitatively provide information about elemental concentrations including their distribution below the surface of a sample.

The recording, reporting, curation, and storage of data are of increasing importance in all areas of science. Optimizing the process of recording data and related information, analyzing data in a systematic manner, and making the data available in a findable manner are all important aspects of reproducibility. Suzuki et al. (D&R-1)³⁹ describe a system under development to accomplish these objectives. Tools have been developed that can convert raw data into a structured data package that consists of mandatory and measurement-characterization metadata, primary and raw parameters, and formatted numerical data (FND). The converted data can be linked with important information such as specimen details, process information, specimen handling records, and an electronic laboratory notebook.

Using this raw-to-repository (R2R) conversion flow, the authors have demonstrated that they can generate and store interoperable data files of XPS spectra and depth profiles, powder x-ray diffraction patterns, (scanning) transmission electron microscope images, transmission electron diffraction patterns, electron energy-loss spectroscopy spectra, and calculated electron inelastic mean free path data. By combining R2R conversion with a high-throughput data collection system and automated data analysis routine, highly reproducible data acquisition and data analysis could be achieved, where human interaction is minimized.

The consistent use of terminology is critical for enabling published work to be reproduced. A paper by Baer and Shard (D&R-2)⁴⁰ provides examples of terms that are commonly misused or confused in the literature. Clear definitions also provide a common basis for reporting and comparing instrument parameters and performance and new terminology is often necessary for clarifying concepts that evolve due to development and advances of understanding. The paper provides examples in each of these areas.

Six papers in the collection focus on reproducibility challenges and solutions in relevant application areas including: high volume semiconductor manufacturing (APP-1),¹⁰ consistence and reproducibility in ALD film growth (APP-2),⁹ and four guides focus on promoting good practice in the XPS analysis of polymers (APP-3),⁴¹ catalysts (APP-4),⁴² epitaxial films and heterostructures (APP-5),⁴³ and nanoparticles (APP-6).⁴⁴ 

Repeatability and reproducibility in surface analysis in the semiconductor industry are key for supporting efficient process development and high-volume manufacturing (HVM) (APP-1).¹⁰ Long-term repeatability is critically important when comparing to historical data, while reproducibility is also required to support technology transfers when HVM of specific devices is to be carried out at multiple sites. As already noted, the authors highlight the importance of instrument setup and calibration, sample handling and preparation, and operator/analyst consistency. They show examples how problems with each of these can lead to wildly varying results.¹⁰ 

Some of the major sources of variations and errors and common misconceptions related to ALD highlighted in (APP-2)⁹ focus attention on issues related to precursors, substrates, and deposition tools. The authors illustrate these problems through examples from the literature, and they present results from numerical simulations that describe how nonidealities would manifest in thickness profiles in a typical cross-flow reactor. They also describe how reproducibility in ALD is linked to consistent experimental practice and reporting between laboratories.

XPS is highly valuable for determining the elemental composition of polymers films and layers and has proven to be an invaluable tool in the development and translation of different polymers to market. Easton et al. (APP-3)⁴¹ note that thorough XPS analysis of these material is not trivial. They provide a summary of the issues and present practical examples of how to address them. Similarly, XPS has become increasingly important in catalysis research (APP-4).⁴² Davies and Morgan describe the approaches that have been developed to obtain reliable XPS on catalytic materials including sample mounting, binding energy referencing, peak identification, peak fitting, particle size determination, and more.

Epitaxial films and heterostructures (APP-5)⁴³ present some unique challenges and opportunities for XPS analysis. Although “standard” XPS analysis approaches can provide important information, the very thin nature and the crystalline order in these materials create both challenges and opportunities for analysts including understanding and using the effects of photoelectron diffraction and extracting information about interfacial charging. Nanoparticle analysis using XPS also presents challenges and opportunities as described by Baer (APP-6).⁴⁴ Major complications due to the tendency of nanoparticles to change in response to their environment, their susceptibility to damage and the impact of size and shape significantly affect quantitative analysis. However, with careful modeling, XPS can be used to get precise information about coatings and coating layers.

A wide range of topics are covered in the collection with many papers focused on significant individual important topics. This has been done so that researchers may seek information on the topic of concern to them at a given time. However, there are some important topics that are not addressed by specific papers and other topics that are only minimally covered. Both are noted here. The areas highlighted briefly below include sample damage, data and parameter reporting, and sputter depth profiling.

Inadequate reporting of information related to instrumentation and instrument setup, data collection and analysis has been identified as a significant contributor to the quality of information in the literature.⁴⁷ The lack of reporting may impact the reproducibility of XPS results in two related ways. First, researchers may be unaware or not understand the importance of parameter selection on their results, a possible indication that they were not selected with the needed care. The lack of reporting also deprives readers of information needed to assess and possibly replicate or reproduce the analyses reported. Thus, lack of adequate reporting represents a serious problem and the importance of careful reporting has been noted in more than a dozen of the papers in the RCS collection as noted in Table III in the  Appendix. The importance of reporting is further highlighted by the series of reporting standards developed by ISO TC201 on surface chemical analysis related to peak fitting,⁴⁸ charge neutralization,⁴⁹ thin film analysis,⁵⁰ and sample handling.⁵¹ 

The importance of looking for sample damage also appears in eight papers in the collection (Table III in the  Appendix). Sample preparation, x-ray sources, charge neutralization methods, and ion sputtering are all possible sources of sample damage that can contribute to unwanted sample alteration that, at a minimum, can confuse the analysis, at worst can lead to serious misinterpretation and can potentially be a source of nonreproducibility. In addition to what is discussed in eight of the papers in the RCS collection, there is an ISO standard dealing with estimating and correcting for unintended degradation of samples during XPS analysis.⁵² A relatively simple way to assess the extent and sensitivity to x-ray damage is to collect a quick XPS survey scan before detailed analysis and, after additional x-ray exposure, to collect another survey scan. If there are significant differences, which can be identified by examining the ratio of survey scan intensities, the sample is changing due to x-ray exposure and appropriate care must be taken.

The importance of composition and chemical state as a function of depth into a sample has been mentioned earlier. Such information can be obtained in multiple ways, some of which are covered by the papers PLN-1 (First Steps Guide),²⁰ DPH-1(Path Lengths Guide),³⁶ DPH-2 (Guide to Backgrounds in Quantitative Analysis),³⁸ APP-3 (Guide to Polymers),⁴¹ and APP-6 (Nanoparticle Guide).⁴⁴ Sputter depth profiling is an important method for obtaining depth distribution information for depths exceeding the relevant electron path lengths/information depths. A guide specifically focused on this topic is being prepared, but not complete in time for inclusion in this collection. It is an important element of discussion in the XPS of polymers guide (APP-3),⁴¹ and references to some useful papers on the topic are provided therein. The development of several types of cluster ion sources has significantly expanded the types of materials for which useful sputter profile information can be obtained using XPS.⁵³ 

As noted in the National Academies study, the existence of significant amounts of nonuseful nonreproducible data and analysis is a deterrent to scientific progress. It wastes time, effort, and resources and may also be a detriment to public acceptance of scientific results. The RCS collection of paper is just one part of several efforts needed to address this issue. It is critically important for researchers to recognize that problems exist and papers such as the assessment of XPS data (PLN-5),¹¹ along with the problems identified in the semiconductor (APP-1)¹⁰ and ALD (APP-2)⁹ areas, inform and, we hope, motivate efforts of the research community to address the issue in their own work and as well as the work of others.

Members of the surface analysis community have indicated that guides, tutorial, and standards would be helpful in addressing the problem by providing information about best practices useful to experience or less experienced researchers using XPS data. The guides, tutorial, and perspectives in the RCS collection are intended to at least partially meet this need. To date many of them appear to be well received and we hope they are useful and used throughout the community. It is also well recognized that, as described in the first steps guide,²⁰ that there are many resources of useful information, including ISO TC 201 Surface Chemical Analysis and ASTM International Committee E42 on Surface Analysis standards and guides for surface analysis, books, website and short courses. All of these can be valuable, and we encourage their use.

Recognition of the problem and the availability of informative tools are not adequate to move toward solving the nonuseful reproducibility problem. Efforts by researchers and research teams are core to any solution. There are multiple aspects to this but two seem particularly important. Appropriate self-review and peer-review of research products are essential. Because of the increasing use of many techniques in research projects, the breadth of the analytical techniques used can extend beyond the expertise of the core research team—as well as that of those reviewing the work—leading to less than full rigor in the analysis of some of the data. Researchers must be willing to seek expertise to make sure their data and reporting is appropriate if they do not have it in their research team.

Reviewers of many papers are assigned based on their expertise in the focus area of a paper. They may not have the expertise required to evaluate all or even most of the types of data used in a paper. One approach to this challenge is to have technique experts review selected parts of a paper, contributing to the review process but not requiring full expertise in the main subject of the paper. In different words, be willing to review more papers, but feel free to review only the parts relevant to your expertise (letting the editor know where other reviewers are required).

Instrument vendors can play a critical role in addressing reproducibility and data quality challenges. Modern technology and significant efforts of instrument vendors have produced a range of high-quality and highly reliable instruments. Unfortunately, the detailed information about processes used in collecting and processing data along with indications of the uncertainties of the analysis is not always readily accessible and often not reported, making data and analysis assessments difficult. It will be useful for vendors to publish detailed instrument characteristics that can be referenced in papers, to develop smarter software that follows and reports workflows and records important parameters. The use of artificial intelligence and expert system approaches can be used to aid and records of data analysis.

The RCS collection of papers has been a true community effort toward addressing the growing problem of erroneous XPS and other data in many publications. We would like to thank many authors who have contributed to the collection. As one author noted, writing an introductory guide is a good deal more challenging than writing many other types of papers. A special thank you also goes to the very responsive anonymous reviewers who have made this collection possible, were unusually prompt in their reviews and, in many cases, significantly enhanced the quality of the papers. We want to thank Nancy Schultheis and Tonya Yandle for their tireless efforts in moving papers forward and dealing with the many different issues associated with creating such a collection of papers. D.R.B. would like to acknowledge the inspiration of MT “Tom” Thomas, Cedric Powell, Martin Seah, and James Castle for their contributions to his interest in quality XPS and thank the members of the ASTM and ISO surface analysis related standards committee for their efforts in developing the standards that make some of this collection of papers possible. G.E.M. thanks Robert E. Clausing for helping launch his career in electron spectroscopy. In 2019, several instrument companies that produce surface sensitive electron spectroscopies Auger electron spectroscopy and x-ray photoelectron spectroscopy were celebrating 50th anniversaries. The high-quality instruments that they have developed have enabled the rapid growth and high impact of these surface analysis methods.

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

The papers in the RCS collection address many topics in addition to the primary focus of each paper. Many of these topics are cross referenced in Table III which lists each paper as well as the topics covered in a matrix. Primary topics addressed in each paper are indicated by xx, while others are indicated by x. Many of the guides have examples of both correct and incorrect protocols and procedures, sometimes in an appendix. We note again, some of the data used as an example in the Background Introduction IDQ-4)³² are published in Surface Science Spectra⁴⁶ so that analysts may download the spectra for examination and comparison of various approaches to background removal or analysis and that a functioning spreadsheet for instrument status recording is available as the supplementary material to Instrument Performance Check paper (ISU-1).²⁴