Understanding the origins of enhanced reactivity of supported, subnanometer in size, metal oxide clusters is challenging due to the scarcity of methods capable to extract atomic-level information from the experimental data. Due to both the sensitivity of X-ray absorption near edge structure (XANES) spectroscopy to the local geometry around metal ions and reliability of theoretical spectroscopy codes for modeling XANES spectra, supervised machine learning approach has become a powerful tool for extracting structural information from the experimental spectra. Here, we present the application of this method to grazing incidence XANES spectra of size-selective Cu oxide clusters on flat support, measured in operando conditions of the methanation reaction. We demonstrate that the convolution neural network can be trained on theoretical spectra and utilized to “invert” experimental XANES data to obtain structural descriptors—the Cu–Cu coordination numbers. As a result, we were able to distinguish between different structural motifs (Cu2O-like and CuO-like) of Cu oxide clusters, transforming in reaction conditions, and reliably evaluate average cluster sizes, with important implications for the understanding of structure, composition, and function relationships in catalysis.
Metal oxides as heterogeneous catalysts have received considerable attention in both fundamental research and industrial applications.1–4 For instance, metal oxide catalysts (MOCs) possess high catalytic performance and robustness in the water oxidation reaction.1,5,6 In industry, MOCs are crucial for the asphaltene adsorption to enhance the oil discovery.2 The metal oxide nanocatalysts, in particular, display unique electronic properties due to their non-bulk-like coordination geometry and redox properties.7–9 To understand the activities of metal oxide nanocatalysts, identification of the active sites of the catalysts10–13 and, importantly, the size and shape of the particles14–16 are required. The geometric properties of nanoparticles play greater role in their activity mechanisms because of the larger surface-to-volume ratio,17 compared to bulk-like particles. Due to the large range of possible structures, the catalytic activities of nanocatalysts exhibit large variation with different geometry.14–16,18 Besides, the nanocatalysts can undergo agglomeration under reaction conditions,19,20 affecting their catalytic activity.
In a toolbox of characterization methods tailored for understanding catalytic mechanisms, a prominent place is taken by the operando method, in which the structure of the catalysts is analyzed in real time, during the reaction, and the reaction products are detected simultaneously with the structural measurement to build the structure-reactivity relation.21–23 Due to the formidable challenge that low metal loadings and high reaction temperature and/or pressure present to many techniques, extended X-ray absorption fine structure (EXAFS) spectroscopy,24–26 the workhorse method for catalytic studies, is limited in its applicability to MOCs. X-ray absorption near-edge structure (XANES) is measured in the same X-ray absorption spectroscopy experiment and has better signal-to-noise ratio than EXAFS; hence, it can be advantageous for use in the in situ/operando catalytic experiments.27 XANES is also sensitive to the arrangements of atoms and electronic characteristics28–31 and is less affected by structural disorder compared to EXAFS.24,26 For some model catalysts, such as size-selective clusters supported on single crystal surfaces, for which EXAFS data cannot be obtained due to their ultra-low weight loadings, grazing incidence (GI) XANES becomes a unique tool to monitor the transformations in the oxidation state, structure, and/or size of the cluster.32–34 However, GI XANES has been rarely employed for structural characterization due to the limitations in its quantitative analysis. Recently, we demonstrated that a supervised machine learning-based method enables the establishment of relation between XANES spectral features and structural descriptors of monometallic nanoparticles.35,36 By employing an artificial neural network (NN) trained on the large set of theoretical XANES, we were able to obtain metal-metal coordination numbers (CNs) and investigate the structure of monometallic nanoparticles and size-selective clusters.35–39 In all prior cases, we deliberately selected well reduced systems to eliminate metal-nonmetal bonding that would have complicated neural network training and applications. That limitation precludes the broad applicability of our NN-based XANES analysis for operando studies, in which the changes in chemical states of the catalysts may occur in real reaction conditions.
In this work, we report the application of the convolution neural network-based method to analysis of the structure and chemical state of size selective copper oxide clusters measured by XANES during their catalytic reaction process. Copper oxide catalysts are known for their good reactivity and selectivity in numerous oxidation and reduction reactions.40–47 One of the important reactions is CO2 methanation, which can assist the conversion of CO2 to chemical feedstock and benefit the inhibition of CO2 emission.48,49 We used GI XANES spectra collected for Cu size-selective clusters in the operando experiment during the process of catalytic CO2 methanation to extract information about the oxidation state and size of the clusters. In what follows, we present our method for training and validating NN, describe the experimental data chosen to illustrate its application to MOCs, and demonstrate the applicability of this approach to extract their structural descriptors in operando conditions.
NEURAL NETWORK TRAINING AND VALIDATION
The common route for NN construction is preparing training sets, training the NN, and validating the NN. From our previous works, it is known that for the construction of training set, we need hundreds of thousands XANES data with unique and a priori known labels (that is, structural descriptors). It is not feasible to obtain such a large number of labeled data from experimental measurements for this purpose. Ab initio XANES simulations could be a good alternative to the experimental spectra, as demonstrated in our prior work.35,36,39 Before planning the training with theory-generated spectra, it is important to verify that a given method or code used for simulations reliably reproduces standard compounds. For example, FEFF950 is adequate for reproducing experimental XANES of bulk Cu oxides, as illustrated in Fig. 1. The details of the XANES simulation are given in Note I of the supplementary material.
Verifying the sensitivity of XANES to the size and structure of the nanoscale oxide is the necessary first step for any regression-based method, in general, and NN-based method, in particular, to work. Following the strategy, first implemented in Ref. 36, we first examine the absorption site effect on XANES spectra, as illustrated in Figs. 2(a) and 2(b). Each spectrum in Figs. 2(a) and 2(b) is labeled with two Cu–Cu CNs (for the first coordination shell and second coordination shell) to represent the structure of copper oxide nanoparticles. By comparing the theoretical XANES (calculated by FEFF9) on different sites of CuO and Cu2O models (indicated by their respective CNs of the 1st and 2nd nearest neighbors), more pronounced features are captured by XANES for the copper atom with the larger Cu–Cu CNs [Figs. 2(a) and 2(b)]. The XANES spectrum for the copper atom in the inner shell of copper oxide model has greater resemblance of the XANES spectrum for the bulk of copper oxide. In contrast, XANES calculated for the copper atom on the surface has relatively more smooth features. After establishing the absorption site dependence, we examined the cluster size effect on XANES by averaging the site-specific spectra over all atoms in the simulated CuO-like and Cu2O-like clusters of different sizes and stoichiometries [Figs. 2(c) and 2(d)]. That procedure is described in greater detail below. The simulated XANES spectra reveal that particles with larger sizes have more pronounced features compared to the smaller particles, as expected from their difference in the surface to volume ratios. As shown in Fig. 2(e), experimental XANES spectra measured in the Cu oxide clusters show a similar trend to have sharper features for larger sizes as obtained for simulated clusters of the same motif (CuO).
Similarly to our previous work,36 to build the training set for NN, we first constructed several sets of Cartesian coordinates for atoms residing in the sites that correspond to the crystal structure of bulk CuO and Cu2O, and truncated the lists of coordinates to simulate clusters with various shapes (tetrahedral, octahedral, and cubic) and sizes. This was accomplished by creating the cluster surfaces using (100) and (111) planes of bulk CuO or Cu2O. The details of lattice structure information for CuO and Cu2O are listed in Table S1 of the supplementary material. To generate more models, additional CuO and Cu2O models were constructed by truncating the previous regular models with (100) and (111) planes. Furthermore, we also constructed the planar structures with one or two layers of (111) plane of CuO and Cu2O to describe the active structural motif, thin film, which has been reported as an active phase for catalysis.51–53 As a result, we created 25 CuO models and 30 Cu2O models to capture the diversity of CuxO nanostructures, which are relevant to catalysis.
In the nanometer-scale nanoparticles and subnanometer clusters, the interatomic distances can deviate from those in their respective bulk compounds42,54 due to the effects of size, adsorbates, and support. For example, the nearest Cu–Cu distance for the bulk of CuO is 2.93 Å. However, the Cu–Cu distance of the CuO cluster was reported to be longer54 or shorter42 compared to the bulk. The shortening of the Cu–Cu distance in size-selected reduced Cu clusters was reported by us earlier.39 To allow for this effect to be recognized in the process of NN-based analysis, we isotropically stretched or compressed the structures in our theoretical models to generate more training sets. The distance between nearest copper atoms varied from 2.784 Å to 3.077 Å for CuO models and from 2.879 Å and 3.182 Å in Cu2O models. These ranges bracket the reported Cu–Cu distances for copper oxides available from EXAFS analyses or crystallography data.42,54,55 To represent the size and shape of the clusters, we choose the first few Cu–Cu coordination numbers as structural descriptors for each unique model. We preferred to rely on the Cu–Cu CNs rather than on Cu–O CNs for this purpose because the latter parameter is not a good descriptor of the cluster size, geometry, and oxidation state in those cases when all Cu atoms are terminated by oxygens. The copper atoms on the surface have smaller Cu–Cu CNs compared to the inner copper atoms, thus providing the desired sensitivity to the size and shape of the copper oxide clusters. We illustrate the sensitivity of Cu–Cu CNs to the different size and shape of the copper oxide models in Figs. S1 and S2. For different copper oxide models (e.g., Cu2O vs CuO), the Cu–Cu CNs exhibit unique values. With the increase of the size of the models, the Cu–Cu CNs also get larger. For the models with same Cu–Cu CNs of the first shell, the values of the Cu–Cu CNs of the second shell provides additional sensitivity to the task of classification of different models.
In order to construct a required (large) number of spectra in the training set using FEFF9, we adopted a combinatorial approach, first developed in Ref. 36, relying on randomly mixing several site-specific XANES calculations for CuO and Cu2O models prepared above. Each spectrum was labeled with first and second Cu–Cu CNs as structural descriptors. The total size of the training set was 100 000 spectra for each of the CuO- and Cu2O-type models. In order to compensate for the unknown X-ray energy shift between theoretical and experimental XANES spectra for each type of oxide clusters (CuO- or Cu2O-like), we shifted all the theoretical XANES spectra by ΔE (obtained from the difference in energy between experimental and theoretical XANES spectra for the respective bulk oxides). Such an approach is reasonable because no visible shift was observed in the XANES spectra between different experimental copper oxide clusters. Furthermore, the convolution neural network we used for machine learning has the advantage of shift invariance,56 which means that the results will not depend strongly on the possible, small (shown to be within a ±1 eV range, as tested in this work) mismatch in the X-ray energy origins used in theory and experiment. An alternative approach, relying on random energy shift between different spectra from the training set, was also recently proposed.57 After the shift was applied, the spectra were interpolated to the same energy scale from Emin = 8981.5 eV to Emax = 9059.3 eV. The step size for the energy scale is 0.15 eV near Emin and increases to 1.5 eV near Emax. Following this step, all spectra were represented as multidimensional vectors, containing 94 data points. Each data point corresponded to the value of absorption coefficient at specific energy.
The NN used in this work was a nonlinear function , where μ represents the preprocessed XANES spectrum (a vector with 94 points) as input and represents the first two CNs as output. The parameter space θ consists of the weights and biases in the NN models.58 The purpose of the training process is to optimize the parameter space θ to accurately correlate input with output. Once the optimal parameters are found, the training process is finished. More details of NN construction and training are described in the supplementary material.
The accuracy of our NN was demonstrated by the theoretical XANES calculated by FEFF9 for particles with different sizes and shapes. Unlike the data set we used for the training, the spectra for validation are particle-averaged spectra (averaged XANES for the particle) corresponding to the real copper oxide models and not used in the NN training process. In Fig. 3, we compare the true Cu–Cu CNs on the first coordination shell with the predicted Cu–Cu CNs for CuO and Cu2O models. The validation for the Cu–Cu CNs on the second coordination shell is given in Fig. S3 of the supplementary material. According to the comparison result, NN can predict accurate CNs from the theoretical XANES for a large range of particle sizes.
APPLICATION TO EXPERIMENTAL DATA
After the validation of our NNs, we applied them to the unknown structures of supported ultra-small size-selected clusters used in a recent work59 These copper-based clusters can be used, for example, as catalysts for conversion of CO2 with hydrogen. The data discussed in this paper were extracted from in situ grazing incidence XANES (GI XANES) spectra collected on samples of 4-, 12-, or 20-atom Cu clusters deposited on zirconia support prepared by atomic layer deposition and by supersonic cluster beam deposition, as exposed to CO2 and H2 under elevated temperatures reaching 375 °C. More experimental details are given in Note III of the supplementary material.
The data extracted from the in situ XANES data were collected and analyzed by multivariate curve resolution with the alternating least squares (MCR-ALS) method to obtain the mixing fraction of clusters of different oxidation states (CuO, Cu2O, and Cu).59 Because our NN method is designed for an idealized, pure metal oxide phase (either CuO- or Cu2O-like), its test required access to the corresponding phase-pure clusters, which were not found in the series of spectra obtained in our operando experiments. We thus selected those spectra collected and analyzed in Ref. 59, which had the highest fractions of CuO or Cu2O phases based on the results of the MCR-ALS analysis. The fractions of individual copper components (CuO, Cu2O, and Cu) for the spectra are listed in Tables S2 and S3. The sampling chosen for testing our NN prediction corresponds to those temperatures, for which the XANES data, as analyzed by MCR-ALS, indicated the presence of at least 70% of either CuO or Cu2O phase. As a justification of validity of this approach, we note that their XANES spectra have similar features with either CuO or Cu2O bulk XANES spectra, thus validating their designation as tests for NN validation purpose.
In Table S2, combining the spectra found (by MCR-ALS) to correspond to the CuO-like clusters, we show the application results of the CuO-trained NN model for extracting the first and second Cu–Cu CNs from those XANES spectra. To interpret the results, the correlation between the number of Cu atoms and the first Cu–Cu CNs for the CuO models is shown in Fig. 4(a). All models shown there were selected from the NN training and validation steps. Such a correlation demonstrates that our method can be used for measuring the cluster size, as evident here from the correct detection of the number of atoms (which was known a priori from the cluster deposition experiment60,61). To check the capability of our NN to distinguish between CuO and Cu2O motifs, we applied our trained Cu2O NN model to the XANES spectra and showed the predicted CNs in Table S2. Not surprisingly, the predicted CNs from Cu2O NN have larger error bars for the first nearest neighbors. It means that the result is unstable. By comparing the predicted CNs from two NN, we obtained that the oxidation state of the sample is consistent with that obtained by an independent chemometric approach, as reported in Table S2.
For the samples with a large fraction of Cu2O, we performed similar NN-XANES analysis, this time by using our Cu2O NN model. Table S3 lists samples that are mainly composed of Cu2O. The first and second Cu–Cu CNs are extracted by our Cu2O NN model. To validate the results, we present the correlation between the number of Cu atoms and the first Cu–Cu CNs for Cu2O models in Fig. 4(b). The results demonstrate a correlation between predicted CNs from Cu2O NN with the sizes of the cluster. Similar to the prior example, we used the CuO-trained NN to check the capability of our method to distinguish between the CuO and Cu2O motifs. The predicted CNs from CuO NN give much larger error bars for the first nearest neighbors than those obtained from the Cu2O NN for the same experimental spectra Table S3. Thus, by combining the predicted CNs from two NNs, and comparing them with the known CN values that correspond to the known cluster sizes, we demonstrated that the oxidation state and structural information can be extracted from the spectra for Cu2O-like clusters (Table S3).
After validating the NNs using experimental spectra for clusters with the known sizes and structures (dominated by either Cu2O or CuO clusters, as described above), we applied the NNs to analyze the spectra for samples of a nominal size of Cu20 and unknown structure and oxidation state. Our trained NNs were applied to answer the question whether the sample was mainly composed of a CuO-like or a Cu2O-like phase. In Table S4, we present the first and second Cu–Cu CNs extracted by our CuO NN and Cu2O NN. To analyze the results and determine the oxidation states, we correlate the predicted CNs with the sizes of the cluster and examine the relation between Cu–Cu CNs and the number of copper atoms in Fig. 4. The predicted CNs from CuO NN follow the size-dependent trend in Fig. 4(a). However, the predicted CNs from Cu2O NN show large error bars when compared with the CuO NN prediction [Fig. 4(b)]. Based on the results for Cu4 and Cu12 clusters, described above and summarized in Tables S2 and S3, larger error bars were always obtained when the incorrect NN models were applied. Therefore, by comparing the predicted CNs and taking into consideration the difference in the error bars (that were demonstrated to be an important factor in discriminating between two possible phases of the copper oxides) from CuO NN and Cu2O NN, we conclude that the Cu clusters containing 20 atoms were dominated by the CuO phase.
In summary, a neural network method was utilized to build the relationship between the XANES spectra and structural parameters for copper oxide cluster systems. This method enabled the determination of the average particle size and the oxidation state of metal clusters during the catalytic reaction by “inverting” their XANES spectra. Since the metal clusters acted as important catalysts in many reactions, this method is poised to have many applications. For instance, for the carbon dioxide and nitrogen oxide related reduction reaction where the metal cluster acts as catalyst and gets oxidized,62 the NN method can be soon utilized to analyze the structure of these metal oxide clusters and thus help decipher the reaction mechanism. At this stage, while our method is an improvement compared to the previously developed NN-based XANES analysis approach, because it is applied, for the first time, not to pure metallic clusters but to the metal oxides, the present NN method still has several important limitations. For example, it relies on the CNs as descriptors and thus cannot distinguish isomers with the same CNs. It is also favoring those speciations where one (CuO, Cu2O, or Cu) phase dominates and would not be helpful when these phases coexist with similar fractions during a particular reaction step. We envision that such recently developed techniques for XANES analysis as MCR-ALS, linear combination, and principal component analysis will be used in combination with our approach for accurately obtaining both the mixing fractions of different types of clusters and structural characteristics of each type. Our method, after the required training and validation, is also applicable to a wide range of metal oxide cluster catalysts and for the understanding of structure, composition, and function relationships in catalysis.
See the supplementary material for additional details on the XANES calculation, neural network implementation and training, experiment, and results of speciation analysis of clusters used in this work.
A.I.F. acknowledges support by the U.S. Department of Energy, Office of Basic Energy Sciences under Grant No. DE-FG02-03ER15476. A.I.F. acknowledges support by the Laboratory Directed Research and Development Program through Grant No. LDRD 18-047 of Brookhaven National Laboratory under U.S. Department of Energy Contract No. DE-SC0012704 for initiating his research in machine learning methods. The computational work used resources of the Center for Functional Nanomaterials, which is a U.S. DOE Office of Science Facility at Brookhaven National Laboratory under Contract No. DE-SC0012704. This work was supported by resources provided by the Scientific Data and Computing Center (SDCC), a component of the Computational Science Initiative (CSI) at Brookhaven National Laboratory (BNL). The work at the Argonne National Laboratory (A.H., B.Y., L.K., M.J.P., and S.V.) was supported by the U.S. Department of Energy, Office of Science, BES-Materials Science and Engineering, under Contract No. DE-AC-02-06CH11357. This research used resources of the Advanced Photon Source (beamline 12-ID-C—S.S., and 12-BM), a U.S. Department of Energy (DOE) Office of Science User Facility operated for the DOE Office of Science by Argonne National Laboratory under Contract No. DE-AC02-06CH11357. S.V. also acknowledges support from the European Union’s Horizon 2020 research and innovation program under Grant Agreement No. 810310, which corresponds to the J. Heyrovsky Chair project (“ERA Chair at J. Heyrovský Institute of Physical Chemistry AS CR—The institutional approach towards ERA”) during the finalization of this paper. The funders had no role in the preparation of the article. P.L. was supported by the U.S. Department of Energy (DOE), Office of Science, Office of Basic Energy Sciences, Division of Chemical Sciences, Biosciences and Geosciences, Chemical Sciences program, under Contract No. DE-SC0012704.