Smartphone based indoor localization using permanent magnets and artificial intelligence for pattern recognition

Localizing people inside buildings using smartphone sensors draws increased interest. Common methods use RFID¹ or Wi-Fi.² Magnetism recently attracted considerable attention due to its pervasiveness and autonomy.³ There are two main positioning approaches utilizing magnetic sensors: identifying magnetic field characteristics of the infrastructure signatures as locations fingerprints,⁴ and using active magnetic flux transmitters as markers.⁵ 

When using fingerprints, a magnetic map is built considering magnetic anomalies as landmarks.⁶ This scheme does not require additional hardware deployment, while interferences and anomalies from indoor building objects greatly improve magnetic field discernibility.³ Yet, this approach requires known magnetic maps or massive calculation to build simultaneous localization and mapping (SLAM). Moreover, in strong electromagnetic interference environments, magnetic fields may differ from the fingerprints causing potential localization errors.⁷ The second method uses external magnetic field transmitters, producing fields located by the smartphone magnetometer, regardless of existing building magnetic fingerprints.⁸ Generating the magnetic flux is energy consuming and expensive transmitter units are required. Both approaches depend either on active magnetic devices, or suitable mapping surveys of the deployed areas, making them sensitive to local magnetic field clutters.

We present a simple, low-power, robust smartphone localization in unknown-magnetic-fingerprint locations by deploying an array of permanent magnets in pre-known locations. By detecting its unique magnetic signature using machine learning-based algorithms, a passive localization method is obtained.

As the proposed method is landmark-based, we draw comparisons to three competing state-of-the-art approaches for indoor localization with landmark-based magnetism mechanisms are covered in Ref. 3. Wang et al.⁸ presented UnLoc, established on landmarks matching. MapCraft, introduced by Xiao et al.,⁹ is rooted on the conditional random fields method. The IODetector from Li et al.¹⁰ used joint thresholds. For completion, we also refer to localization methods based either on spatial-temporal sequence matching or on fusion with motion.³ 

Consider commercial buildings where many people carry uncontrolled ferrous materials. The magnetic field is measured by the cell-phones’ vector magnetometer. Magnetic clutter is present, e.g., stationary magnetic gradients of the construction infrastructure, distorting the natural magnetic flux and causing spoof localizations. The infrastructure may contain large-scale electrically conductive loops acting as sources of magnetic alternating transmissions.¹¹ Furthermore, moving ferrous objects create interference either from inside the building (people carrying ferrous objects) or from outside (cars, trains). Finally, the cellphone itself generates clutter, both by moving in the environmental field and from its uncontrolled internal electrical currents.¹¹ 

Ferromagnetic bodies create three-dimensional distortions of the ambient Earth magnetic field. Usually, dipole approximation is acceptable. Here, the anomaly magnetic flux decreases with inverse proportion to the third power of the distance. The decay for the magnetic field B is:

where K is a constant with range of 0.1-0.2, M the total magnetic moment in Am², and r is the dipole-to-sensor distance. Typical values of indoor background clutter (e.g., corridor surroundings) lie around 10-20 μT, easily detectable by smartphone magnetometers with common sensitivity of about 1μT. Generating patterns having good signal-to-interference-ratios (SIRs), an induced field of 40-50 μT is required. By Eq. (1), for r between 0.5-1m, a magnet of 125Am² is required. Such devices exist: fixed neodymium magnets, powerful mainly due to the tetragonal Nd₂Fe₁₄B crystal structure, having exceptionally high uniaxial magneto-crystalline anisotropy. The neodymium alloy is composed of microcrystalline grains aligned in powerful magnetic fields during manufacture. The resulting magnetic energy value is 18 times greater than “ordinary” magnets by volume. This allows using small but very prominent magnets as passive beacons or markers, producing the required signals. The magnetic field is three-dimensional and the phone sensor can measure each one of its components.

We embed small-volume-large-moment permanent magnets in given locations inside the building and arrange them in specific geometric configurations. This results in super-structure patterns of supervised magnetic signatures, constituting unambiguous magnetic environments. The localization algorithm learns these unique patterns during a training stage and detects them from ongoing data streaming during real-time localization: the test stage. Localization is based on smartphone motion rather than on static positioning of the magnetometer.

Each permanent magnet creates a dipole or a quadrupole, depending on its geometry. The dipole creates non-uniform magnetic flux, resulting in magnetic fields that differ from the source at every azimuth, and used to code the reference magnet. By placing an array of magnets in different orientations, a coded magnetic field is created. A magnetic sensor passing along a path of coded magnets can determine its unique position by matching the pre-learned code.

Let B_x[n], B_y[n], and B_z[n] respectively hold the nth sample of the x, y, and z magnetic field components, processed with moving time frames of N samples. Note $B̲xi,B̲yi,B̲zi∈R1×N$ the ith time frame of the x, y, and z components. Denote the concatenated ith frame as $fi=B̲xi,B̲yi,B̲zi∈R1×3N$⁠. Assume two hypotheses for each frame f_i: $H0$ if the embedded magnetic pattern is absent and $H1$ if it is present in that frame. Define the following indicator $I(fi)$⁠:

The goal is to correctly classify (f_i) ∀i.

We implement six AI-based algorithms with different modeling mechanisms: two variations of support vector machine (SVM),¹² with and without principal components analysis (PCA).¹³ To efficiently perceive non-linear trends, a fully-connected deep neural network (FC-DNN)¹⁴ was used with 4 hidden layers of 400 neurons each. Optimization was carried using Adam.¹⁵ These methods lack feedback information in their mechanisms, instructive when dealing with time series modeling. Thus, a recurrent neural network (RNN),¹⁶ a gated recurrent unit (GRU),¹⁷ and long-short term memory (LSTM)¹⁸ networks are implemented.

Measurements were collected using Xiaomi-mi4 smartphone and the Sensor Kinetics Pro sensor acquisition application, which recorded magnetic sensing information from the BOSCH magnetic field sensor with sensitivity between 0-1600μT, resolution of 0.3μT, and power consumption of 0.5mA.

Experiments were conducted in four environments of different intrinsic magnetic patterns. In each, the user strolled with the smartphone in-pocket and recorded 30 minutes of data, comprising 10 passes by the embedded magnetic pattern. To generate randomness, walking pace differed between 0.8-2 m/s and walking distance ranged between 0.5-1m (Fig. 1).

The walking heading ranged between 0° and 360° while elevation remained unchanged. The magnets were positioned at height of 1.5m and at distance of 3m from one another. The average SIR was 8 dB.

The data contained five signals: 3-axial magnetic signals, pitch, and roll, notated B_x, B_y, B_z, P, R, respectively. The magnetic norm, B, was calculated to enhance the prominent energy of instilled magnets (Fig. 2):

We aimed to reduce dependency of measurements on the smartphone orientation. Thus, we generated the horizontal and vertical magnetic components, B_h and B_v, correspondingly:¹⁹ 

All magnetic signals sense prominent magnetic anomalies around the magnets relative to the intrinsic magnetic environment. While B expectedly captures the explicit target pattern, B_h follows the same behavior while being user-orientation-independent.

During pre-processing, rectangular windows of 12.5s with 80ms shift divided the data into frames, capturing the entire embedded pattern in a single frame, while containing each target signature multiple times. The database contains 72000 data frames.

We perform feature extraction from the time series measurements to feed the AI with a compact and informative data. The extended ith time frame is:

Each signal component in $f̃i$ is mapped to 20 features with physical and statistical orientations. To facilitate temporal behavior, each feature vector is concatenated to 2 feature vectors affiliated with 2 adjacent past time frames, resulting in 360 features per time frame. Past look of 2 frames enhanced performance without overconsuming computational load. Let FE(·) hold the feature extraction operator and denote $x̃i∈R1×360$ as the feature vector extracted from $f̃i$⁠. For i > 2:

Each feature vector is linked with a binary label $yi∈0,1$⁠, indicating magnetic signature.

We construct the training and testing datasets with a leave-one-out methodology and perform four training-testing procedures. The reported results correspond to the statistical mean of test accuracy over this 4-fold process.

The localization performance (ROC) of the six AI algorithms is shown in Fig. 3, allowing analysis of the trade-off between true positive rate (TPR) and false-positive rate (FPR) in various operation points.²⁰ The Long Short-Term Memory (LSTM) network leads. Following the LSTM come GRU and RNN. Even though they show degraded performance relative to the LSTM due to less advanced temporal modeling, their architecture is leaner regarding memory and computational load, being more adequate for on-device applications. The least performant methods are the DNN and SVM. LSTM shows 95% accuracy (based on Ref. 21), while GRU and RNN reach 93% and 90%, respectively. DNN shows 85%, SVM-PCA method reaches 80%, and SVM produces 75% accuracy. The superiority of the LSTM over competing methods is explained as follows. LSTMs are able to learn optimal time series context via feedback-based learning for temporal classification, this context being pre-specified and fixed in the DNN and SVM methods. Even though the RNN employs feedback, it suffers short-term memory, preventing it from modeling temporal context between early and late periods. To mitigate this gap, the GRU and LSTM architectures contain gate mechanisms that regulate information flow through the network. While the GRU contains fewer parameters that require less memory and training data, the LSTM embeds an additional memory unit inside its gate allowing enhanced control over information flow.²² 

For performance evaluation, we employ the key metrics of mean localization error (MLE) and classification accuracy.³ These metrics are sensitive to changes in test environments, so the following values are listed for reference only. UnLoc achieved MLE between 1-2m and outperformed MapCraft. Both approaches show lower performance compared to the proposed approach, localizing at maximal error of 1m. The IODetector was able to obtain localization accuracy as high as 82% indoor, which falls short to the 95% presented by the LSTM-based algorithm covered in this study. Additional methods, rooted on spatial-temporal sequence matching or on fusion with motion,³ also show lower MLE and classification accuracy compared to our study.

Additional experiments were conducted using the LSTM algorithm to inspect generalization and robustness. First, localization accuracy with respect to the walking pace of the carrier was analyzed, to verify our approach as applicable to real-life scenarios. Walking paces were taken from the ensemble {0.8, 1.2, 1.6, 2} m/s. Performance analysis showed respective average localization accuracies of $94.5,96.2,95.6,94$%. This evaluation stresses high robustness of the proposed approach for slow and fast paces. The optimal walking pace, in terms of maximal accuracy, is expected to vary depending on the smartphone sampling rate, distance between magnets, and carrier walking distance. Future work will cover this extended analysis.

Localization should apply in intrinsic environments of low SIR levels. While experiments dealt with 8dB SIR, we synthetically inspected the performance in SIRs ranging from 6dB to 0dB. Results have shown that 6dB SIR barely affects the performance with average degradation of less than 2% in accuracy, while 4dB SIR leads to an accuracy of 88%, and 0dB SIR reaches 82%.

Another experiment synthetically decimated the resolution of recordings to simulate CPU load. While measurements were taken at 120Hz, 60Hz and 30% lead to 88% and 75% accuracy respectively.

In practical scenarios, it is beneficial to record fewer shots of the magnetic pattern during training to avoid inefficiency and fatigue. Experiments showed that instead of crossing the magnetic pattern 30 times in each of the 4 environments, 20 times were sufficient for 90% accuracy, and 10 times provide accuracy of 80%.

These experiments point to the robustness of the LSTM algorithm to dominant magnetic interference and lower amounts of training data. It also emphasized the highly generalized modeling the LSTM performs that enables it to identify the target signature with missing information derived from impeded data resolution.

A new low-power dynamic indoor localization method combined with AI is presented. Passive permanent magnets are deployed instead of active magnetic transmitters. The localization was performed based on smartphone motion rather than on the static positioning of the magnetometer. Among various AI schemes, LSTM network showed leading performance of 95% accuracy. Additional experiments point on the robustness of the LSTM algorithm to dominant magnetic interference and lower amounts of training data. These results are very promising regarding applying passive localization of a moving person in indoor environment. For instance, it may be used for locating potential customers passing by a targeted area in a shopping mall or a store aisle. Further refinement of this method will address specific magnets orientation, pre-defined pattern positioning, and improved AI models to reach higher accuracies.

The data that support the findings of this study are available from the corresponding author upon reasonable request.