Visual explanations of machine learning model estimating charge states in quantum dots

Semiconductor quantum bits (qubits) that utilize electron spins in semiconductor quantum dots are expected to be good candidates for future qubits because of their high operation fidelity and excellent integration properties. Basic operations, such as single-qubit^1,2 and two-qubit operations,^3,4 have been demonstrated, and quantum error correction has also been demonstrated¹² with improved operation fidelity.^5–11 Furthermore, attempts are being made to construct large-scale quantum systems using the semiconductor integration technology.^13–20 It is essential to trap one electron in each quantum dot to construct semiconductor qubits. Charge state tuning in semiconductor quantum dot devices is required for this purpose. In previous experiments, this has been mainly accomplished by manually adjusting the gate voltages to achieve the desired charge states. However, this approach requires a significant amount of time to learn and execute. This will make tuning large-scale semiconductor quantum systems difficult in the future.

There is a movement to solve such a difficult problem automatically.²¹ The main types of algorithms studied in these methods are script-based algorithms^24–27 or machine-learning (ML) methods.^28–39 In particular, ML methods are expected to be more easily applicable to different experimental environments and more compatible with different devices. Efforts to develop ML methods for auto-tuning quantum device parameters are currently underway.^22,23 In ML methods, the amount of labeled data required for network training is enormous, making the preparation of labeled experimental data as training data a labor-intensive task. Consequently, simulations have been utilized to generate training data. This approach has led to the development of a method that utilizes the Thomas–Fermi model and deep convolutional neural networks (CNNs) for recognizing charge states.²⁸ In addition, by adding various types of noise that could be observed in real experiments to the training data, the noise-robust charge state recognition has been demonstrated.³⁹ 

Thus far, charge state recognition has primarily been demonstrated in double quantum dots.^28,31,39 Looking toward the future large-scale integration of quantum dots, there is a need to explore scalable methods. In addition, while the Thomas–Fermi model with added noise is highly useful as a physical model, the necessity of such physical accuracy in a data generation model for machine learning has not been extensively debated. For instance, humans tend to estimate charge states by focusing mainly on charge transition lines, without giving much consideration to background noise. Therefore, generating training data with a physical model that reflects these characteristics could lead to the realization of an estimator with judgment criteria similar to those of humans. Such an estimator would likely be more user-friendly for humans, as it would share the same criteria for judgment. To achieve this, it is important to clarify the criteria for judgment, which is usually a black box.

In this study, we analyze the operation of the ML model that estimates the charge states in double quantum dots. We propose pre-processing the data through binarization, which enables us to use a simple simulation model to generate training data. We also visualize the interest of the model by gradient-weighted class activation mapping (Grad-CAM)⁴⁰ and confirm that they are on the charge transition lines. By incorporating feedback from the Grad-CAM results, we demonstrate improvements in the ML model.

Figure 1(a) illustrates the process flow for generating a charge state estimator (CSE). Initially, the preparation of training data is needed to train the ML model. We utilize a constant interaction (CI) model in the simulation for simplicity, where quantum dots are modeled as a capacitor circuit.^41,42 Using this CI model and treating the electrostatic coupling between the double quantum dot and the charge sensor, we can obtain the charge stability diagrams with the size (30 × 30) pixel measured using a charge sensor by changing the parameters in the model. This size is chosen to achieve a sufficient resolution to distinguish the charge transition lines. The next step is to prepare the charge state label, corresponding to the presence and relative locations of trapped electrons: no quantum dot, which means an empty state (ND), single left (LD), center (CD), right (RD), or double quantum dot (DD), for each stability diagram [Fig. 1(a), step 0]. Note that the CD state is formed by strong coupling between both dot sites, including a large single dot, which in the CI model is expressed by setting a large value for mutual coupling capacitance between both quantum dots C_m. Here, the diagrams are automatically categorized based on the largest number of quantum dots in the region.

Next, pre-processing is performed on these images. All images are differentiated in the x axis to make the charge transition lines clear [Fig. 1(a), step 1]. Then, the images are binarized by adaptive thresholding to simplify them into black and white images [Fig. 1(a), step 2]. Here, the charge transition line is set to white, while the background is set to black. In this process, we use adaptive thresholding by the OpenCV module.⁴³ 

To adapt to noisy experimental data, random noise is added to these images. Due to binarization in the previous step, noise can be realized by simply flipping the black and white pixel values at random locations [Fig. 1(a), step 3]. We adjust the amount of noise such that the number of white pixels after this process does not depend on the charge states. Images labeled DD will have more white points than others without this adjustment because there are many charge transition lines, and the CNN will be trained to estimate the state by counting the number of white pixels. We adjust the number of white points in an image between 30 and 110, combining the noise and the charge transition lines. Next, a 3 × 3 Gaussian filter is applied to the images [Fig. 1(a), step 4]. The same filter is applied to the experimental data in the pre-processing step. After applying the Gaussian filter, the image is again binarized by adaptive thresholding, as shown in Fig. 1(a), step 5.

Here, 84 325 images, each measuring (30 × 30) pixel, are undersampled from a total of 180 000 images in the simulation and trained by CNN. Images for each charge state are prepared in the same quantity. The details of the CNN are presented in the supplementary material.

In this section, we perform the charge state recognition of the real experimental data with the obtained CSE. The experimental data are a radio-frequency reflectometry-measured charge sensing signal from a GaAs double quantum dot.^44,45 The size of the experimental data is (30 × 30) pixel [Fig. 1(b), step 0]. Note that the gate voltage range is adjusted to include several charge transition lines in the images. This prevents the situation where the (30 × 30) pixel is small compared to the interval of the charge transition lines, which could lead to erroneous recognition of the ND state for other charge states.

Prior to inputting the experimental data into the CSE, it undergoes pre-processing using almost the same methods as the training data. This includes differentiation, application of a Gaussian filter, and binarization. These steps correspond to steps 1, 2, and 4 in Fig. 1(b). Adaptive thresholding can binarize even with a non-uniform background because the threshold is calculated for each small region. It is useful for the actual experiments since charge stability diagrams often have a nonuniform background due to the shift of the charge sensor condition. Here, the adaptive thresholding parameters are set to the same as those in the training process.

We now discuss the performance of the charge state recognition using the obtained CSE on the experimental data pre-processed by the steps in Fig. 1(b). The experimental data (step 1 in pre-process) are presented in the background layer of Fig. 2 using gray-scale representation. The color in Fig. 2 represents the result of the charge state recognition by the CSE. Here, each recognition is performed by extracting each (30 × 30) pixel area from the original image. Subsequently, CSE loads each area and returns the output as the maximum number of quantum dots in the area. Orange, deep blue, green, light blue, and yellow dots indicate the points classified as the ND, LD, CD, RD, and DD regions, respectively. Note that the CD state is not obtained in the used experimental data. By comparing the results with experimentalists’ classification per pixel, the accuracy is evaluated as (93.3 ± 0.8)% with 30 CSEs using the same network architecture. This means that the model even trained by the simple binarized data can recognize the charge states of noisy experimental data. We also apply the CSE to the simulation data with noise, and an accuracy of (97.0 ± 0.2)% is achieved. Therefore, the experimental data are estimated with a degree of accuracy comparable to that of the simulation data. To verify that the CSE developed in this study is not exclusively optimized for only our experimental results, we apply it to publicly available experimental data, including the CD state from previous studies.⁴⁶ The CSE demonstrates a (86.0 ± 1.2)% accuracy in recognizing charge states, even when applied to data from other group.

We analyze the explainability of the operation of the CSE on the experimental data using Grad-CAM.⁴⁰ This method identifies class-discriminative regions for each estimation by the model from an input image. The results of the Grad-CAM analysis are demonstrated in Fig. 3. The images in the left column are the input images, while those in the right are the outputs from Grad-CAM. The brighter regions in the right panels of Fig. 3 correspond to the regions that affect the result of the CSE. We can see bright regions around the charge transition lines in these images. In the case of the simulation data with noise, the CSE estimates charge states by focusing on these lines as in the case of the operation on the experimental data. These demonstrate that the CSE estimates charge states by focusing on these lines, indicating that human-like recognition can be realized.

Here, it is important to note that in Fig. 2, there are several instances where regions labeled as ND are incorrectly classified as RD and LD. Such misclassifications outside the boundary regions of charge states pose a problem in the auto-tuning of the quantum dots. Examining the Grad-CAM results at these misclassification regions, it appears that areas where a few noise pixels are connected are identified as charge transition lines, as shown in Fig. 4(a). In the training data of the first CSE, all five classes have the same number of data by undersampling. In this case, the data without charge transition lines are less than the data with these. This makes the trend of the CSE to find charge transition lines even if there are no lines, which leads to detect the connected noise pixels as the charge transition line. To address this issue, we increased the ND training data by fourfold to balance the data with and without charge transition lines, further teaching the CSE that noise pixels can occasionally connect in the ND state. As a result, as shown in Fig. 4(b), the misclassifications in the ND region have notably decreased. This demonstrates that the feedback from the Grad-CAM is useful to improve the model. The accuracy are also evaluated as (92.8 ± 1.3)%, (96.1 ± 0.3)%, and (85.7 ± 1.2)% for our experimental, simulation, and other group’s data, respectively. Note that these accuracy values are primarily influenced by misclassifications near the boundary regions of the charge states. Then, modifying the training data resulted in negligible effects on the overall accuracy, even though it improved classifications outside these boundary regions.

In this paper, we analyze the explainability of the machine learning model estimating charge states in quantum dots. To generate the training data, we employ the CI model, which is a simple simulation model and clearly shows the charge transition lines we typically focus on to recognize the charge states. The CSE recognizes five charge states in both our experimental data and that of other group, and its interest highlighted by Grad-CAM is on the charge transition lines, the same as humans. This allows for the tuning of CSE parameters, such as training data, based on human insights and has been empirically demonstrated to result in improvements. This approach eliminates the need for excessive parameter tuning, thus reducing the cost associated with generating an optimal ML model. We also thoroughly detail the pre-processing steps for both training and experimental data, offering sufficient information to enable replication of our method.

Toward large-scale semiconductor qubits, the method is needed to recognize the charge state in the additional multiple quantum dots. One of the approaches is utilizing a virtual gate,²³ which can translate the charge stability diagram of the multiple quantum dots into that of the double quantum dots. By utilizing virtual gates, the method demonstrated for double quantum dots in this study can also be applied to multiple dots, expanding its potential applicability. Additionally, our pre-processing steps, including the CI model, facilitate the expansion of the quantum dot system without incurring significant additional simulation costs. This approach is also suited for future scaling up of quantum dot systems.

See the supplementary material for additional details regarding the CNN model.

We acknowledge the RIEC Fundamental Technology Center and the Laboratory for Nanoelectronics and Spintronics for the technical support. Part of this work was supported by MEXT Leading Initiative for Excellent Young Researchers, Grants-in-Aid for Scientific Research (Grant Nos. 20H00237, 21K18592, 23H04490, 23H01789, 23H05455, and 23KJ0200), the Fujikura Foundation Research Grant, the Hattori Hokokai Foundation Research Grant, the Kondo Zaidan Research Grant, the Tanigawa Foundation Research Grant, FRiD Tohoku University, and Cooperative Research Projects of RIEC. Y.M. acknowledges the WISE Program of AIE for financial support.

The authors have no conflicts to disclose.

Yui Muto: Data curation (lead); Investigation (lead); Methodology (equal); Software (lead); Visualization (lead); Funding acquisition (equal); Writing – original draft (lead). Takumi Nakaso: Conceptualization (equal); Investigation (equal); Methodology (lead); Software (equal); Writing – review & editing (equal). Motoya Shinozaki: Data curation (equal); Investigation (equal); Methodology (equal); Visualization (equal); Resources (lead); Writing – original draft (equal); Writing – review & editing (equal). Takumi Aizawa: Investigation (equal); Methodology (equal); Resources (equal); Writing – review & editing (equal). Takahito Kitada: Investigation (equal); Methodology (equal); Resources (equal); Writing – review & editing (equal). Takashi Nakajima: Methodology (equal); Resources (equal); Writing – review & editing (equal). Matthieu R. Delbecq: Methodology (equal); Resources (equal); Writing – review & editing (equal). Jun Yoneda: Methodology (equal); Resources (equal); Writing – review & editing (equal). Kenta Takeda: Methodology (equal); Resources (equal); Writing – review & editing (equal). Akito Noiri: Methodology (equal); Resources (equal); Writing – review & editing (equal). Arne Ludwig: Methodology (equal); Resources (equal); Writing – review & editing (equal). Andreas D. Wieck: Methodology (equal); Resources (equal); Writing – review & editing (equal). Seigo Tarucha: Methodology (equal); Resources (equal); Writing – review & editing (equal). Atsunori Kanemura: Conceptualization (equal); Methodology (equal); Software (equal); Writing – review & editing (equal). Motoki Shiga: Conceptualization (equal); Methodology (equal); Software (equal); Writing – review & editing (equal). Tomohiro Otsuka: Conceptualization (lead); Methodology (equal); Resources (equal); Funding acquisition (lead); Supervision (lead); Writing – review & editing (lead).

The data that support the findings of this study are available within the article and at https://en.qd.riec.tohoku.ac.jp/database.