Generative design, manufacturing, and molecular modeling of 3D architected materials based on natural language input

Directed materials design is an exciting frontier for science and engineering.^1–5 The quest to develop new tools that intersect directly with human cues such as language, however, has remained challenging and largely limited to database mining approaches. While these are powerful, we envision new material models and design approaches that directly engage with human forms of communication. With the emergence of deep learning and the use of the generative adversarial network (GAN), transformer methods, and natural language processing (NLP), we can now generate quantitative and rigorous models toward that goal⁶ and provide a computationally sound realization of earlier category theoretic approaches.^7–10 

Reflecting these developments, neural networks have been used in recent studies to develop novel new classes of materials, such as materials developed from fire or musical expression,^10–13 offering a systematic approach to translate information across manifestations.^14–19 More recently, in earlier work,²⁰ we reported a general concept to use a combination of the Vector Quantized Generative Adversarial Network (VQGAN)—a variation of the Vector Quantized Variational Autoencoder (VQ-VAE) model²¹—and Contrastive Language-Image Pre-training (CLIP)²² to realize a word-to-matter paradigm. However, earlier work was limited to quasi-2D materials. Here, we expand the concept to 3D architected materials and provide a detailed mechanistic analysis of failure under extreme mechanical deformation using molecular dynamics (MD) simulation.

In recent years, massive foundational neural network models have been built,²³ thanks to the broad availability of computational resources and the development of “attention” mechanism neural networks. The “attention” mechanism, as the name suggests, originates from the idea of human attention, which directs neural networks to focus on key components of the input data.^24,25 Such attention to certain details of a process is exquisitely suited to describe physical phenomena and, more broadly, design approaches, as it reflects key factors, such as those displayed in singular mathematical events or rare occurrences of events.

The transformer neural network model is a model built based on an attention mechanism,²⁶ which shows great success in natural language processing (NLP) and also in computer vision (CV).²⁷ Following the transformer model concept, numerous large pretrained DL models have been developed including, but not limited to, “Bidirectional Encoder Representations from Transformers” (BERT)²⁸ and “Generative Pre-trained Transformer” (GPT)²⁹ for NLP tasks and “DEtection TRansformer” (DETR)³⁰ and “Vision Transformer” (ViT)³¹ for CV tasks. Those pretrained models, due to their outstanding performances, benefit both direct applications and adapted domain learning via transfer approaches.

Within the context of applications in the physical sciences and specifically materials research, the transformer concept has been applied to generate molecular fingerprints,³² predict organic chemical reactions,^33,34 and design de novo drugs.³⁵ In our previous work, by combining the attention-based CLIP and VQGAN models, we showed how we can generate images of 2D architected materials that reflect text prompt driven designs.²⁰ The “words-to-matter” approach reported in that work opened up a new perspective of materials design enabled by both generative networks and pretrained foundation models and illustrated how they can be generalized toward broader applications in materials science.

With the rapid development of ML approaches, and especially generative models, the materials-by-design paradigm has evolved using generative ideas proposed by artificial intelligence (AI). Additive manufacturing (AM) enables the realization of those conceptual ideas of materials design, bypassing the traditional manufacturing techniques using a bottom-up approach. The combination of AM and generative designs has proved successful in soft robot optimization,³⁶ composite device development,³⁷ and bio-inspired design.³⁸ However, earlier studies have not examined a generative design framework for architected 3D materials using human-readable and parameterizable text input. Such input, especially including the concept to systematically vary the text using a mix of mathematical and language-based parameterization, is a straightforward and natural way for human beings to input and output certain ideas for materials design.

Indeed, materials-by-design has been motivated by human ideas that are often initialized based on written text or artisan work, especially considering its historical relevance and context over thousands of years of civilization.^17,39 Words and sentences are our species’ natural way of communication to spread and describe our ideas. However, the gap between the original thoughts described by human language and the final materialization is not easy to bridge. The text-to-material paradigm usually involves a modeling process with the assistance of experimental manufacturing techniques or computational simulations and/or mathematical models. However, the possibility of utilizing a deep learning based approach for the paradigm remains little explored. Given the rapid advances of DL architectures in NLP and the development of massive generative neural nets, we are now able to link the text to generative designs through an iterative process combining the approaches from these two fields.

Here, we propose an approach that combines transformer neural nets with generative models to enable a text-to-material translation for 3D architected materials that feature fully periodic, tileable unit cells. We extend the approach from 2D to 3D and develop an atomistic model based on the continuum structures using MD simulations along with 3D printed geometry, which enables the modeling of generated designs across different levels of scales, leading to various insights about extreme mechanical deformation mechanisms via atomically precise models. Compared to continuum-level modeling we performed in our previous work,²⁰ MD simulations can reveal atomic-level phenomena, such as dislocation motion, providing multiscale insights including atomistic details about the generated designs.

In this paper, we demonstrate the use of a transformer neural network in the design of architected, 3D materials. 3D hierarchical architected materials find numerous applications in a variety of industries,^40,41 ranging from healthcare to structural engineering, and offer significant advances to enable multifunctional properties at low weight.^42–45 

Serving as an outline of this work, Fig. 1 depicts a flowchart of the approach reported here, translating words—human readable descriptive text—toward 3D physical material designs. We proceed with various examples generated based on the approach, describe how we convert the predicted images into 3D models, and use multimaterial additive manufacturing to manufacture them. We then report experimental and computational analysis of mechanical properties to assess the viability of the designs generated.

Figure 2 shows example images generated from various text inputs using mathematical parameterization of the word cues provided using a variable X that offers distinct weight to specific words. In this example, we use the text cue “hexagonal lattice|hollow circles,” whereas we systematically vary the weight of the two terms in the image generating using the algorithm reported in Ref. 22. This is achieved by adding the variable X to the text cue, as in: “hexagonal lattice X | hollow circles (1 − X)” where X is between [0, 1]. One can clearly see the variation from a focus on a “lattice” toward “round” objects as the weights are varied.

Figure 3 depicts further processing of one of the generated images, showing how they are processed into a periodic, 3D structure. Figure 3(a) shows how periodicity is achieved in the x–y plane, whereby the image is ultimately repeated four times through a series of mirroring operations. Figure 3(b) depicts the chosen image that was picked for transformation into a 3D architected material, following the concept described in panel (a). Figure 3(c) shows the two-step transformation of the image into a pixel intensity map (left). The intensity map is then used for generating the 3D model, as described in Sec. IV.

Figure 4 displays the process by which a 2D image (i.e., the x–y plane) is used to construct a 3D representation by generating a stack of images (z-direction) for volumetric reconstruction. Each layer in the stack is represented by a different threshold value of pixel intensity, whereas the brightest spots achieve the largest height while the least bright spots the lowest height. The construction is pursued in both positive and negative z-directions, leading to a symmetric structure that can be periodically stacked in the z-direction, and repeated multiple times, yielding a fully periodic and hence tileable architected material. Since the x–y plane is already periodic via the process described in Fig. 3, the resulting architecture represents a fully periodic unit cell. Threshold values for the minimum and maximum values are used to achieve a continuous 3D model, such that the material is present when transitioning between multiple repeats.

Figure 4(a) shows the image used for this 3D material reconstruction, showing the original image (left) and the pixel intensity map (right) (the detailed process by which this was obtained is described in Fig. 3). Figure 4(b) depicts how a continually varying threshold is used to generate layers for the architected material. The process is repeated four times in the z-direction to match the four copies in the x–y direction needed to achieve full periodicity, as explained in Fig. 3. The supplementary material, Movie M1, shows a traverse through the layers in the z-direction. These layers are then processed into a 3D mesh representation, as shown at the bottom of Fig. 4(b).

Figure 5 shows the results of additive manufacturing result of the architected material described in Fig. 4. The architected material is printed using a multimaterial printer, whereas PLA is used for the material phase and PVA as a water-soluble support material (to realize the complex 3D structures with internal holes). A gyroid infill is used in this example to illustrate the capacity to yield hierarchical designs. The final material is depicted in the lower right panel of the image. Figure 6 shows various snapshots of the resulting architected material, from different angles. The bottom image shows a macro-view of the printed material, revealing the individual printed layers (approximate length-scale of each layer is on the order of tens of micrometers). Figure 7 illustrates prints generated from soft materials, printed using black TPU filament. Using such flexible, soft material, novel architected materials can be fabricated that allows for large deformation. Figure 7 (left) depicts a full 3D architecture, showing one periodic layer in the z-direction (as opposed to four layers as in Fig. 6). Figure 7 (right) shows examples of deformation of a soft architected material, featuring a hollow architecture as shown in the top visual. Such architected materials could find applications in the biomedical field or soft robotics, for instance.

The resulting 3D model cannot only be used to generate 3D physical samples using 3D printing but can also serve as the basis for atomistic modeling, for instance, by simulating how an architected material with such a geometry but made out of metal (e.g., aluminum) would behave. Figure 8 shows an atomistic version of the architected material, modeled using an embedded-atom method (EAM) potential.⁴⁶ The atomistic structure is generated using a perfect FCC aluminum crystal based on the continuum-level architected materials with atoms removed in the region of voids (details in Sec. IV: Atomistic model of architected materials). We aim at investigating the atomic-level behaviors of the generated structure using molecular dynamics (MD) simulations. A compression test at a high strain rate is performed on the atomistic model [Fig. 8(a)]. To better visualize the atomic porous structure, surface meshes are constructed based on atomic positions.

Figure 8(b) displays the structural evolution of the atomic structure during compression. The corresponding strain–stress curve [Fig. 8(c)] shows the typical three-stage evolution of porous materials under the compression test. At a small compressive strain (<3%), the structure is in the linear elasticity region. When the strain is between 3% and around 45%, the stress increases slowly, showing a plateau region. After the porous structure reaches increasing densities above a threshold, a densification process leads to a significant increase in stress.⁴⁷ 

To analyze the compressive behaviors, we compute local dislocation lines and lattice types,⁴⁸ which are visualized in Fig. 9(a). The crystal structures are colored based on the local lattice type and the dislocation lines are colored based on either the local character that is classified into edge and screw dislocation or the dislocation type. As the compressive strain increases, we observe more and more dislocations and shear bands in the structure. In order to quantify the structural evolution, we calculate the lengths of different types of dislocations and the number of particles given local crystal types during compression. Figure 9(b) confirms the visual observation in Fig. 9(a), showing the overall increasing trend of dislocation lengths. During strain between 45% and 60%, the lengths decrease in some dislocations because of the densification. Based on Fig. 9(c), we find that the shear bands shown in Fig. 9(a) are mostly HCP lattices. More details about the evolution of dislocations and crystal structures are visualized in the supplementary material, Movie M2.

The work reported here provides a path to translate words or language input more broadly into 3D architected material designs, based on a generally applicable algorithm that transforms 2D image data into hierarchical architected materials. While a general algorithmic variation of the produced image can be accomplished (see Fig. 2), the method also provides us with the capacity to translate a single 2D image into an architected material, as described in Figs. 3 and 4. Future work could explore the use of gradients, where the constructed architecture can smoothly vary in spatial directions x, y, and/or z. This offers yet another way to develop material functionality.

The designs generated here were 3D printed (Figs. 5–7), from both stiff and soft materials, to demonstrate the generation of physical prototypes. In another example, we generated a 3D aluminum molecular model and exposed the material to extreme compressive loading. This enabled us to assess the detailed molecular mechanisms of deformation including dislocation activity (Figs. 8 and 9).

Future work could focus on better mechanical characterization during the design process, for instance, via the use of genetic or other optimization algorithms. This can not only yield static functional materials but also provide access to multidimensional 4D printed materials^45,49,50 that would originate from neural network synthesis and incorporate tunable and hence time-dependent material properties. Also, from the perspective of Fused Deposition Modeling (FDM) 3D printing, the thermal–mechanical effects may be an important factor that may contribute to mechanical performance.⁵¹ Since our aim is to build architected materials with a high variety of changes in wall thickness and shape in real space, such effects may be important and could be addressed in future work when more comprehensive experimental studies are performed.

More broadly, research into the materialization of language can be of great interest to the physics community, especially the intersections of physics and philosophy.^52–54 The availability of mapping models as presented in this paper can form a foundation for future research into such relationships, especially when explored with massive NLP models and how they relate with physical models.

The methods used in this paper include the following:

• deep neural networks (a combination of VQGAN and CLIP for an integrated NLP-transformer image generation approach that can be mathematically parameterized);

• a method to translate a 2D image into a periodic 3D architecture in x-, y-, and z-directions, fully tileable;

• additive manufacturing using multi-material FDM, to construct complex materials with internal architectures and voids; and

• mechanical analyses using molecular dynamics (MD), to elucidate fundamental deformation mechanisms of metal-based nanoarchitected metamaterials.

The Contrastive Language-Image Pre-training (CLIP) model⁵⁵ is a large pretrained neural network model that was trained on a variety of (image-to-text) pairings. The model was originally built for general image classification tasks. In our work, the CLIP model is utilized to evaluate the generated images from VQGAN given a specified input text, realized in an iterative optimization process. The detailed architecture and hyperparameters of CLIP model are the same as our previous work.²⁰ 

The VQGAN model is a neural net that combines convolution operation with transformers to generate images at super high resolutions. The only difference of the pretrained model we use in this work compared to the previous work²⁰ is the size of the codebook. The codebook size is 128 × 128 in this work, which can produce higher reconstruction quality compared to the former size of 16 × 16 in earlier reports. The VQGAN model serves as a generator that produces new candidates guided by CLIP model.

The integration of CLIP and VQGAN model was first proposed by Komatsuzaki to generate art images from text prompt.⁵⁶ Here, we leverage the integrated CLIP and VQGAN model for 3D architected materials design,²² and specifically the creation of 3D material architectures as described in Sec. IV B.

The primary objective for image processing is to transform a complex image with multi-channel data into a printable geometry. To that end, we use colormap operations, smoothing functions (cv2.GaussianBlur and cv2.bilateralFilter), and other image processing methods to remove small image parcels, and/or islands, to ultimately generate printable mechanically functional designs based on continuous material distributions. All image operations are performed using the OpenCV computer vision package implemented in Python.⁵⁷ 

We use an algorithm to convert an image into a 3D representation that is periodic in all directions after transforming it into a grayscale image to enable algorithmic processing using pixel intensities. The approach is described in detail in Figs. 3 and 4. There are several steps involved.

• Step 1: To create a periodic topology from a 2D image, we rotate and mirror the original unit and then concatenate the results together as shown in Fig. 3(a). By doing so, the expanded image is now periodic in both x- and y-directions and can be extended by itself one after another. Also, we are able to apply smoothing functions to find the original image to meet the desired precision during this process.

• Step 2: To further expand an image from a 2D plane to a 3D printable geometry, we need extra information to build the geometric shape toward the normal direction of the plane. This information is “hiding” in these images and is extracted using several processing steps. First, by applying different colormaps, the pixel-by-pixel intensity map of an image is calculated, which is then transformed into only a single channel. The value at each pixel can now be considered as an indicator to decide its position in the corresponding 3D structure subjecting to a directional light, that is, the height perpendicularly to the original 2D plane. With such spatial information, we apply a scanning function to pick up pixels in an image and then pin them into a 3D binary array according to its assumed position. The above construction will go through the original image not only once but reversely again to expand a mirror image for the geometry to be periodic in the z-direction. Stacks of such periodic 3D structures are assembled into higher level architectures as those shown in Figs. 5 and 6.

With this algorithm, any 2D image with multiple channels can be transformed into a 3D geometry periodic in all directions, and during the process, the expanded geometry can be directly stored as a stack of images layer by layer. Gradients in the z-direction can be realized by slowly varying the source image, whereby Step 2 is conducted with slowly changing input data, which is then tiled in the z-direction. Gradients in x and y can be realized by repeating Step 1 with distinct source images that slowly vary as periodic images are tiled in such directions.

While not done here, multimaterial designs can be easily realized by using different pixel intensity thresholds (for, say, stiff vs soft materials) or different layers in the final makeup of the architected material. That way, complex plywood-like material geometries can be designed and manufactured.

With the above approach to translate a 2D image into a stack of images in order, it is straightforward to place them into a 3D model to form a 3D array, in which any nonzero value can be considered as a cube, or more precisely, a voxel, to be printed out afterward. In this study, a binary 3D array, in which 1 represents the entity and 0 denotes the void region, can form a 3D porous material. After a 3D geometry is generated, we then apply two python open-source libraries, skimage⁵⁸ and trimesh,⁵⁹ to render the 3D arrays into STL format that can then be read by 3D printing slicing software.

This voxel-based approach using a 3D array enables us to solidify our stacks of images and to modify the details in a model before it is rendered into STL format. As mentioned before, we avoid generating parcels or islands in a 3D geometry because they hardly have a contribution to the mechanical performance and sometimes are not even printable with certain 3D printing techniques.

Overall, this two-step method not only can translate a 2D image into a 3D printable model but also gives us the potential to explore additional mechanically functional designs based on information within just one image.

We employ additive manufacturing to generate 3D models of the materials designed from words. 3D files are sliced using Cura 4.9.1 to obtain GCODE and printed using a Ultimaker S3 multi-material printer. The process is shown in detail in Fig. 5. A water-dissolvable Ultimaker PVA support material is used to realize the complex geometries, whereas the primary material component is printed using an Ultimaker White PLA filament. Similarly, soft specimens are printed using TPU filament (both NinjaTek NinjaFlex and Ultimaker TPU95 are used). PVA is also used as a water-dissolvable support material.

We investigate the behavior of architected materials at the atomic level using the unit cell in Fig. 4. We generate the initial configuration of the atomistic structure based on a stack of images showing the continuous distribution of 3D architected materials. The detailed procedure to generate the atomistic structure is as follows: We first create a perfect FCC aluminum crystal containing 250 unit cells of aluminum in all three directions (with a total of more than 15 × 10⁶ unit cells in total). Then, we remove atoms based on the voids in the image stacks to create pores in the crystal to reflect the architected material design.

Specifically, we find the corresponding pixel of each aluminum atom in the perfect crystal based on the spatial position and remove the atom if the color of the pixel is white. We use a relatively large atomistic system to avoid surface instabilities (which can occur if the void dimension reaches atomic vacancy dimensions or that of crystal unit cells and hence coalescence of pores so that the initial atomistic structure). With this constraint, we find that the metal-based architected nanomaterial structure is maintained after equilibration in MD simulations. The force field we utilize for aluminum nanocrystal modeling is a many-body interatomic potential developed for monoatomic metals based on the Embedded Atom Method (EAM).⁴⁶ 

With the initial atomistic structure of the architected material, we implement non-equilibrium molecular dynamics (NEMD) simulations using a Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS)⁶⁰ to perform a compression test to the atomistic architected material. Before the deformation is exerted, the initial structure we obtain from the image slack is equilibrated with both energy minimization (10 ps) and relaxation process in NPT ensemble (50 ps). During the compression test, an NVT ensemble is applied to the system and the strain rate is set to be −7.5 × 10⁻³/ps. The final compressive strain is −0.75.

The visualization of the MD results is performed in OVITO.⁶¹ To better visualize the porous structure, we utilize the “construct surface mesh” modifier that generates a geometrical description of the outer and inner boundaries of an atomistic solid using the alpha-shape algorithm.⁶² In terms of visualization of dislocation lines and crystal structure evolution during the compression, a “dislocation analysis” modifier is leveraged, which implements a so-called dislocation extraction algorithm⁶³ to calculate burger vectors and generate line representation for dislocations. The dislocation lengths and crystal types are output from the “dislocation analysis” modifier.

See the supplementary material for Supplementary Movie M1: Traverse across image stacks in the z-direction, illustrating the individual x–y layers that form the basis for 3D architected material construction (https://www.dropbox.com/s/mgo4yi5knxyqf05/Movie_M1.mp4?dl=0) and Supplementary Movie M2: MD simulation snapshots of compression tests of the atomistic-level architected material made of aluminum (https://www.dropbox.com/s/x59n309nkrchvc3/Movie_M2.MP4?dl=0).

The authors acknowledge support from the MIT-IBM AI Lab, MIT Quest, ONR (Grant Nos. N000141912375 and N000142012189), AFOSR-MURI (Grant No. FA9550-15-1-0514), and ARO (Grant No. W911NF1920098).

The authors have no conflicts to disclose.

M.J.B. performed the neural network calculations, developed the image processing, and performed the 3D printing and experimental tests. Y.-C.H. developed the 3D rendering algorithm, and Z.Y. developed, carried out, and analyzed the MD simulations. All authors wrote and edited the paper, analyzed the data and conclusions, and contributed to the scientific research design and interpretations.

Y.-C.H. and Z.Y. contributed equally to this work.

The data that support the findings of this study are available from the corresponding author upon reasonable request.