Deep neural network (DNN) has made remarkable achievements in many research and business environments. But DNN is a black box model, and you don't know how or why they work. Some limitations of traditional DNN can be alleviated by using basic physical knowledge to guide or constrain the training process or network architecture of DNN in different ways.
Recently, researchers from Duke University have proved that integrating known physical knowledge into machine learning algorithms can help the mysterious black box gain new transparency and have a deeper understanding of material properties.
Researchers have constructed a modern machine learning algorithm, the Lorentz neural network (lnn), a feedforward neural network, which uses the physical constraints of causality to directly learn the electrical and magnetic response functions that completely determine the electromagnetic scattering characteristics of metamaterials.
This method not only enables the algorithm to accurately predict the properties of metamaterials, but also is more effective than previous methods, and provides new insights.
The research results, entitled "learning the physics of all dielectric metamaterials with deep Lorentz neural networks", were published in advanced optical materials on May 9.
Willie Padilla, Professor of electrical and computer engineering at Duke University, said: "by directly incorporating known physics into machine learning, the algorithm can find solutions in less training data and shorter time. Although this study mainly shows that the method can reproduce known solutions, it also reveals the internal working principles of some non-metallic metamaterials that have not been known before."
*Metamaterials * are a kind of man-made materials with special properties, which are not available in nature. They have some special properties that traditional materials cannot achieve. There is nothing special about the composition of metamaterials. Their peculiar properties come from their precise geometry and size.
The metamaterial consists of a grid of large silicon columns similar to LEGO floor. Depending on the size and spacing of the cylinders, metamaterials interact with electromagnetic waves in various ways, such as absorbing, emitting, or deflecting specific wavelengths.
Figure 1A shows the all dielectric metamaterials (ADM) considered in this work. The inside of the cylinder is shown in Figure 1b.
Figure 1: Adm.
Here, researchers try to build a machine learning model called neural network to find out how a series of heights and widths of a single cylinder affect these interactions. The physical properties that researchers impose on neural networks are called Lorentz models - a set of equations that describe how the inherent properties of materials resonate with electromagnetic fields. Instead of directly jumping to predict the response of the cylinder, the model learns to predict the Lorentz parameters, which are then used to calculate the response of the cylinder.
The physical information lnn consists of a feedforward neural network with four fully connected hidden layers (100-250-250-100). The input layer receives the geometric vector g of size 4 and outputs the parameters of the Lorentz oscillator. Its architecture is shown in Figure 2.
Figure 2: lnn architecture.
Omar Khatib, a postdoctoral researcher at Padilla lab, said: "when you make neural networks more interpretable, in a sense, this is what we do here, and fine-tuning may be more challenging. We do have difficulties in optimizing training to learn patterns."
However, once the model started working, it proved to be more effective than the previous neural networks created by the team for the same task. In particular, the team found that this approach could significantly reduce the number of parameters required for the model to determine metamaterial properties.
They also found that this physics - based approach was self - discovery.
When an electromagnetic wave passes through an object, the way it interacts with it at the beginning of the journey is not necessarily the same as at the end. This phenomenon is called spatial dispersion. Because researchers had to adjust the spatial dispersion parameters to make the model work accurately, they discovered physical processes that they did not know before.
Compared with traditional DNN, lnn and other information-based neural networks should have higher data efficiency and achieve better extraterritorial generalization. Figure 3 shows the comparison between lnn and two traditional DNN models. The selection of the two DNN models makes the first network (represented as dnn1) have the same hidden layer structure as lnn, while the second model (represented as dnn2) aims to achieve the similar performance as lnn, and the size is much larger.
Figure 3: comparison between lnn and two traditional DNN models.
It is found that the performance of lnn is significantly better than that of dnn1, which is likely to have enough complexity to fully approximate the underlying physics, and achieve a performance similar to that of dnn2, which has two orders of magnitude of learnable parameters. In addition, compared with dnn2, the generalization ability of lnn outside the training boundary is improved. Finally, it is found that the performance of lnn is variable in different regions of geometric space.
The results show that the causal physical lnn, which is supplemented by frequency dependent dielectric constant and permeability when predicting complex scattering parameters, shows a significantly enhanced ability to learn the complex physical characteristics of ADM, while the training data is much less, and the model parameters are much less than the traditional DNN.
"Now that we have proved that this can be done, we hope to apply this method to systems with unknown physics." Padilla said.
"Many people are using neural networks to predict material properties, but it is a great pain to obtain enough training data from simulation," malof added. "This work also points out a way to create models that do not need too much data, which is very useful."
Paper link: