Meta-photonics: A bridge between physical association and digital models in photonics

Metaverse, an enormous innovation with many recent applications in ubiquitous domains, has become a powerful tool to converge virtual and reality, whose impact on these areas is still unfolding.^1–3 Based on this insight, we interfaced traditional photonics with modern technologies, such as deep learning models, and proposed a nascent concept of meta-photonics. Notably, it has taken physicists decades to just reveal a fraction of the photonics, because without known or partially known associations, even a simple device may remain enigmatically complex.⁴ Although an upper limit for the physical associations in photonics is established, we can still shorten the path to discovery of deep associations between different concepts by introducing diversified digital mapping.^5–14 The proposed concept of meta-photonics gives the traditional photonics a considerable acceleration for opening more avenues to gain more flexible optical responses. One critical bottleneck which impedes the realization of such devices is that natural materials have a remarkably limited range of electromagnetic responses. Therefore, in this work, we employed metamaterials with more potent electromagnetic responses as anchor points to demonstrate the feasibility and contribution of meta-photonics.^15–20 However, there are numerous challenges in the design of metamaterials, such as excessive reliance on specialist knowledge and Maxwell's equations in the design processes, significant time costs due to the lack of intuitive understanding of the relationship between metamaterial structure and its optical responses, and the necessity for an optimized starting point when numerical design methods require the identification of specific material structures or device types.^21–25 

Traditionally, metamaterial design methods rely mainly on simulation techniques, such as finite-difference time-domain (FDTD), which suffer from limitations, such as the need for physical insight and intuitive reasoning, the use of laborious and resource-consuming structure design methods, and a highly restrictive selection of geometric patterns.^26,27 To address these limitations, researchers are looking to use the computational capabilities of artificial intelligence (AI) to accelerate the design process and reveal unknown relationships between a large number of variables.^28,29 Nonetheless, most existing work has only analyzed amplitude or phase separately, with relatively few data sample points and insufficient accuracy in neural network model prediction, particularly in the terahertz band. Furthermore, once the desired optical response requirements are altered, the previous dataset and network architecture become challenging to reuse, impeding the transferability of data and methods.

To overcome these challenges, we comprehensively utilized the convolutional neural network (CNN), long short-term memory (LSTM), gated recurrent unit (GRU), and transformer that have excelled in computer vision (CV) and natural language processing (NLP) for building our neural network models.^30–34 Different network models bring forth diverse approaches for data extraction and analysis, thereby better catering to the diverse requirements of various optical problem scenarios. This approach, to some extent, mitigates the limitations imposed by the lack of interpretability of underlying logic, which can restrict design effectiveness (additional details are provided in the supplementary material). We also conducted an analytical study of various photonics concepts, such as polarization conversion ratio (PCR), extinction ratio (ER), unwrapping phase, surface electric field, and connected domain, in addition to previous phase and amplitude analyses, through the establishment of innovative digital model relationships. This enhanced concept recognition and device design, even though the physical relationships remain partially unknown at present. Moreover, we selected a broader bandwidth for analysis in the model prediction and increased the data sample points beyond 1000 (a few dozen times the number of previous works). When the selection of predicted data sampling points transitions to quasi-continuous curves, the difficulty in model design and learning significantly increases. However, our design approach not only addresses this challenge but also enhances the accuracy of network predictions. Additionally, we introduced the unwrapping phase to decrease the prediction mean squared error (MSE), effectively mitigating the difficulty of model prediction caused by sharp curve changes. Furthermore, we achieved a significant increase in device efficiency by optimizing the loss function module while ensuring the functionality of the device, further improved the design by considering the connected domain parameters in device fabrication, and provided a corresponding fitting analysis of the surface electric field and network structure weights.

Here, we demonstrate the concept of meta-photonics through a remarkably simple low-resolution (25 × 25) metamaterial structure, which yielded exceptional predictive results for both forward and inverse networks. This modeling approach functions as a “screen” that can generate structures with completely different shapes based on diverse optical response requirements. It eliminates the significant constraints imposed by the binding relationship between specific shapes and their corresponding functionalities. Additionally, we postulate digital models for previously unknown physical phenomena, proving the tremendous potential of this approach toward the versatility and scalability of various concepts in photonics and opens up more avenues for optical metrology, optical neural chip, image classification, and other photonics fields.^35–40 

The concept encompassed in photonics is exceedingly broad, which gives rise to a diverse range of digital models that urgently need to be constructed. Therefore, in order to demonstrate the concept of meta-photonics, the metamaterials selected should satisfy the demand for a broad range of optical responses. Additionally, when faced with other optical target demands, there is no need to change the dataset or redesign the network architecture. As such, we considered the substrate as a screen, and the individual unit where a metal patch can be placed as a pixel [Fig. 1(a)]. This grants us a screen of 25 × 25 pixels, with the ability to generate as many as 2^25 × 25 distinct patterns where “1” represents a metal patch and ‘0’ does not. We have constructed a dataset consisting of no less than 5000 sets of all target parameters by randomly assigning “0” or “1” to 625 arbitrary units on the structural surface. In fact, this model is not entirely unique, as the arrangement of regular geometric patterns of metal patches, such as the split-ring resonator, has been explored before.^41–43 However, with this arrangement, there is more potential for previously unexplored geometric distributions that bring about richer and more exciting optical responses. Thus, for the metasurface structure arranged in a periodic array, we chose a high-resistance silicon substrate with a side length of 200 μm and a thickness of 500 μm, an individual pixel with a side length of 8 μm and a thickness of 200 nm of aluminum flake for each unit (Fig. S1). With the incident x-polarized light in the terahertz frequency band ranging from 0 to 2 THz, we explored the optical responses generated by transmission of the structure and have achieved a higher level of accuracy in reproducing the curve distribution trend of original optical response through the use of over 1000 sampling points. Although the significant increase in quantity of data, a few dozen times more than previous works, tremendously elevates the prediction complexity of deep learning models while ensuring prediction accuracy, we were able to achieve high-precision prediction by this approach in both forward and inverse models through our continuous optimization process.

In predicting the optical responses of unknown devices, traditional methods primarily rely on classical interpolation effects, numerical calculations, or a series of trial-and-error processes to find relative suitable values of various parameters in the device, even if they are not optimal. While these methods can achieve the desired optical response results to some extent, the time expended is enormous. Moreover, for structures that are extremely complex and have the same few types of configuration for reference, there is a lack of auxiliary verification methods for the highly nonlinear relationship between input and output data, making it difficult to ensure absolute accuracy with a single simulation result.⁴⁴ Therefore, we first constructed a forward design from the metasurface structure to its optical responses. In this work, the dimension of the spectral data is 1 × 1001, which allows for a more precise representation of the overall curve trend of the transmission spectrum. For photonics, the digital models among different concepts are still nascent and need to be continuously explored and optimized. The lack of explicit underlying logic makes it challenging for a single deep learning model to adapt to complex optical response analysis, so we conducted forward network design using CNN, GRU, LSTM, and transformer, and correspondingly analyzed the different optical responses of the constructed metasurface. This includes not only the phase and transmitted amplitude with the same incident polarization state but also the transmitted amplitude orthogonal to incident polarization state, unwrapping phase, and the two parameters PCR and ER which are frequently involved in optical devices. These six different parameters, as shown in Figs. 1(b) and 1(g), respectively, represent the results of y-polarization amplitude, PCR, phase, x-polarization amplitude, ER, and unwrapping phase (the detailed information for each individual figure can be found in the supplementary material). All prediction results achieved a highly consistent fitting with the simulation results in our forward network, with an MSE as low as approximately 0.008. In previous similar works, the MSE for task prediction was approximately around 0.1. In this work, the prediction performance has been improved by over ten times compared to previous works in terms of accuracy. Compared with traditional electromagnetic simulation methods, such deep learning models greatly reduce the calculation time of metasurface optical responses. The time required for a single prediction is less than 0.1s. Although the model training process requires some time, this cost is only one-off, and the trained model can be reused in future work, which is a negligible and highly worthwhile investment. The detailed forward network results and other detailed data for these six different parameters are explained in the supplementary material.

After constructing the forward network design, we conducted research and analysis on the inverse design process from the desired optical responses to the structural pattern. Inverse design is the most challenging part of traditional photonics design because researchers can only analyze and calculate based on their own past design experience after the target optical response is proposed. Without this experience or relevant expertise, trial-and-error is the only way to gradually find suitable structure parameters, which is undoubtedly an arduous and protracted process.⁴⁵ 

In order to alleviate this issue, we focused on the concept of amplitude in meta-photonics and once again utilized the powerful data processing capabilities of deep learning to analyze the highly nonlinear mapping relationship in inverse design. We selected four most classic optical responses in traditional photonics for verification: narrowband band-stop filter, wideband band-stop filter, narrowband bandpass filter, and wideband bandpass filter. However, there is an issue concerning the real existence of input data in the inverse design process. Although such a metasurface screen can display various types of optical responses, such as bandpass filter and band-stop filter, it will inevitably lose many fine results due to the limitation of the smallest step in the structural parameters being 8 μm.

Nevertheless, this does not affect the purpose of this work in validating the feasibility of the concept of meta-photonics, as the design can still be achieved solely through this simple screen with 25 × 25 pixels, accomplishing the inverse design of these four most classic optical responses in photonics. As shown in Fig. 2, although we have no previous experience of such a metasurface simulation type and are uncertain about the optical responses of this structure, we can still analyze data according to the target spectrum and input different ideal spectrum data for completing the inverse design process.

Although the input spectrum data (blue solid line) may not find a set of completely matching structural parameters among the 2^25 × 25 possibilities under this scheme, it does not prevent us from continuously optimizing the model to obtain the target structural parameters with high fitting accuracy. After getting hold of the inverse design results by using the deep learning model, these parameters are input into the electromagnetic simulation software CST for calculation, and the simulation results of four devices (black dashed lines) are attained. From Fig. 2, we can see that the inverse design results obtained by the deep learning model have high consistency with the target spectrum.

After completing the inverse design of the concept of amplitude, we also analyzed another classic concept in photonics, phase. Compared with amplitude which has relatively stable data curves, the inverse design of phase has always been difficult due to its large data curve fluctuation. Prior research often considered predicting the real and imaginary parts of the phase separately to achieve a certain degree of accuracy improvement which inevitably increased both the complexity of the network and the workload required by network optimization. In this work, we attempted to use the unwrapping phase instead of phase for the inverse design. In terms of physics, the unwrapping phase only adds different cycles (360°) to varied phases without actual changes. However, it can transform the traditional phase data curve with violent fluctuations into a smooth or even close-to-linear one. As shown in Fig. 3(a), yellow bars represent the MSE of traditional phase under different deep learning network model parameters, and blue bars represent that of the unwrapping phase method. In this way, we effectively reduced the MSE by tens or even hundreds of times in the inverse design of phase, achieving a highly effective optimization.

The inverse design of phase profiles for metasurface structures ultimately aims to achieve modulation of light beams according to target requirements by arranging different phase distributions on a two-dimensional plane. If this structure is evaluated from the perspective of an optical device performance, it can be divided into two parts: one is the accuracy of each unit phase and the overall distribution of the two-dimensional plane, which determines the type of function of the device; the other is the amplitude size of each unit, which dominates the efficiency of the device. If the phase accuracy is insufficient, the modulation effect of light beams will be reduced. However, if either the phase accuracy or amplitude is prioritized at the expense of the other, practicality and industrialization of the device will be constrained.

In previous optical device design methods using deep learning, amplitude and phase were often considered independently and calculated separately. That is, previous research efforts have primarily focused on optimizing the performance of single objectives, leaving multi-objective optimization tasks in the field of optics largely unresolved. In this work, we used a unique method by adjusting the loss function, aiming to output the structure parameters of metasurfaces with higher amplitude values while maintaining the MSE of the inverse design network as stable as possible.

As depicted in Figs. 3(b) and 3(c), yellow bars represent the MSE after refinement, while blue bars represent the MSE of traditional methods. Figure 3(b) illustrates the comparison of MSE between the optimized method and traditional methods under different deep learning model parameters within the overall data sampling frequency range. Additionally, the comparison of MSE between the optimized method and traditional methods at a single operating frequency (1 THz) is presented in Fig. 3(c). It is evident that the optimized MSE is only about 1/5 or 1/6 of the traditional methods under certain model parameters, demonstrating its excellent performance.

Based on the aforementioned approach, we designed two metasurfaces shown in Figs. 4(a) and 4(c), respectively, where the yellow square represents “1” pixel with a metal patch and the blue square represents “0” pixels without a metal patch. Their inverse design input data and simulation results are shown in Figs. 4(b) and 4(d), respectively, where the red dashed line illustrates the input to the network model during the inverse design process and the black solid line demonstrates the simulation optical response results based on the parameters of the metasurfaces acquired from the inverse design. It can be seen that even in a wide frequency sampling range with denser data sampling points, the deep learning network model used in the inverse design still shows a satisfactory computational performance.

At the center operating frequency (1 THz), the two curves nearly overlap, and the prediction error is even less than 1°, which is rare among similar works that utilize deep learning for computation. In order to intuitively verify the two metasurfaces extracted from the inverse design, we arranged them in a two-dimensional plane according to the pattern of “0101…” as shown in Fig. 4(e), where white represents phase “0” and black represents phase “1.” The two metasurfaces have a constant phase difference of 45° at the operating frequency of 1 THz [as shown in Fig. 4(f)], and the resulting metasurface has a beam splitting effect at an angle of approximately 48.6° due to the coding metasurface theory, which is a classic, extensively validated concept in traditional optics and, therefore, not elaborated upon in this work.^46–48 

In inverse-designed networks, the output structural parameters for individual input data are often not unique, and “one-to-many” mappings frequently occur. Therefore, taking manufacturing process and data post-analysis into account, we ulteriorly proposed the consideration of connected domains. For optical devices with same optical response requirements, we selected ‘1’ region as the benchmark. If they have the same number of “1” pixels, we will choose the structure with lower connected domains. This is because photolithography is mainly used in the current manufacturing process, and a large pattern with high convergence is easier to process than sparsely distributed patterns. For the derived structures in inverse design, the fewer connected domains of “1” area, the closer the result is to the classical geometric patterns gained by traditional optical design ways, which is more conducive to summarizing, analyzing, and increasing the design experience of researchers in relevant fields. Conversely, if the connectivity domains of “1” area are numerous, the pattern tends to be more chaotic, making it difficult for humans to learn different design concepts contained therein, which is not conducive to the increase in photonics design experience.

We also conducted some interesting potential mapping relationship analyses. For example, we selected two different states of the metasurface structure corresponding to a classic physical phenomenon known as electromagnetically induced transparency (EIT) within the metasurface domain (see Fig. 5). Although the precision is restricted by the minimum pixel length discussed earlier, this does not affect the qualitative demonstration of the potential mapping relationship that may exist. Unlike from physical parameters to physical parameters in the forward or inverse network design, digital relationships not only serve as a link in the network but also as an important role in the mapping relationship. In all regions situated on the same substrate of a specimen, the proximity to structured areas composed of metals or alternative materials exhibits an escalating significance in generating optical responses. However, there is also a digital weight system when such a metasurface physical model enters the deep learning network model calculation.

Therefore, we visualized the weight of this surface and obtained the results in Figs. 5(e) and 5(f). Compared with the surface electric field in Figs. 5(c) and 5(d), an obviously intuitive corresponding relationship can be observed. The potential development brought by the existence of such mapping relationships is enormous, but the underlying mechanisms still need to be further elaborated by the development and improvement of photonics and deep learning theory.

In summary, we demonstrated the versatility and scalability of the concept of meta-photonics by the metasurface screen that can exhibit a variety of optical responses. Through the construction of forward and inverse network models using different deep learning models, such as CNN, GRU, LSTM and transformer, we creatively analyzed a wider range of photonic concepts beyond the classical concepts of amplitude and phase, including the unwrapping phase, PCR, ER, connected domain, and surface electric field. Despite the challenge of achieving more precise spectral sampling of over a thousand points, we have made significant advancements in accuracy compared to previous works. After undergoing dual validation through simulation and experiment, it exhibited a satisfactory fitting performance with theoretical values. Moreover, the proposed modeling approach eliminates the need to change the dataset or redesign the network architecture when switching target tasks. It has also demonstrated excellent performance in multi-objective optimization tasks. With increased resolution, our method holds the potential for designing complex optical devices. While future development of this concept requires more digital associations to be met, our research showcases the possibility of expanding the scope of conventional photonics through diversified development and opens up more avenues for image processing, integrated photonics, optoelectronic neural chips, and other photonic fields.