The rapid development of computation power and machine learning algorithms has paved the way for automating scientific discovery with a scanning probe microscope (SPM). The key elements toward operationalization of the automated SPM are the interface to enable SPM control from Python codes, availability of high computing power, and development of workflows for scientific discovery. Here, we build a Python interface library that enables controlling an SPM from either a local computer or a remote high-performance computer, which satisfies the high computation power need of machine learning algorithms in autonomous workflows. We further introduce a general platform to abstract the operations of SPM in scientific discovery into fixed-policy or reward-driven workflows. Our work provides a full infrastructure to build automated SPM workflows for both routine operations and autonomous scientific discovery with machine learning.
REFERENCES
1.
K.
Bian
, C.
Gerber
, A. J.
Heinrich
, D. J.
Müller
, S.
Scheuring
, and Y.
Jiang
, “Scanning probe microscopy
,” Nat. Rev. Methods Primers
1
(1
), 36
(2021
).2.
P. K.
Hansma
, J. P.
Cleveland
, M.
Radmacher
, D. A.
Walters
, P. E.
Hillner
, M.
Bezanilla
, M.
Fritz
, D.
Vie
, H. G.
Hansma
, C. B.
Prater
et al, “Tapping mode atomic force microscopy in liquids
,” Appl. Phys. Lett.
64
(13
), 1738
–1740
(1994
).3.
G.
Binnig
, C. F.
Quate
, and C.
Gerber
, “Atomic force microscope
,” Phys. Rev. Lett.
56
(9
), 930
–933
(1986
).4.
G.
Binnig
, H.
Rohrer
, C.
Gerber
, and E.
Weibel
, “Surface studies by scanning tunneling microscopy
,” Phys. Rev. Lett.
49
(1
), 57
–61
(1982
).5.
F.
Ohnesorge
and G.
Binnig
, “True atomic resolution by atomic force microscopy through repulsive and attractive forces
,” Science
260
(5113
), 1451
–1456
(1993
).6.
G.
Meyer
and N. M.
Amer
, “Novel optical approach to atomic force microscopy
,” Appl. Phys. Lett.
53
(12
), 1045
–1047
(1988
).7.
F. J.
Giessibl
, “The qPlus sensor, a powerful core for the atomic force microscope
,” Rev. Sci. Instrum.
90
(1
), 011101
(2019
).8.
M. F.
Crommie
, C. P.
Lutz
, and D. M.
Eigler
, “Confinement of electrons to quantum corrals on a metal surface
,” Science
262
(5131
), 218
–220
(1993
).9.
Y.
Martin
, C. C.
Williams
, and H. K.
Wickramasinghe
, “Atomic force microscope–force mapping and profiling on a sub 100-Å scale
,” J. Appl. Phys.
61
(10
), 4723
–4729
(1987
).10.
A. J.
Heinrich
, J. A.
Gupta
, C. P.
Lutz
, and D. M.
Eigler
, “Single-Atom spin-flip spectroscopy
,” Science
306
(5695
), 466
–469
(2004
).11.
T.
Ando
, T.
Uchihashi
, and N.
Kodera
, “High-speed AFM and applications to biomolecular systems
,” Annu. Rev. Biophys.
42
(1
), 393
–414
(2013
).12.
B.
Drake
, C. B.
Prater
, A. L.
Weisenhorn
, S. A. C.
Gould
, T. R.
Albrecht
, C. F.
Quate
, D. S.
Cannell
, H. G.
Hansma
, and P. K.
Hansma
, “Imaging crystals, polymers, and processes in water with the atomic force microscope
,” Science
243
(4898
), 1586
–1589
(1989
).13.
Y. F.
Dufrêne
, D.
Martínez-Martín
, I.
Medalsy
, D.
Alsteens
, and D. J.
Müller
, “Multiparametric imaging of biological systems by force-distance curve–based AFM
,” Nat. Methods
10
(9
), 847
–854
(2013
).14.
E. T.
Herruzo
, A. P.
Perrino
, and R.
Garcia
, “Fast nanomechanical spectroscopy of soft matter
,” Nat. Commun.
5
(1
), 3126
(2014
).15.
F.
Huber
, H. P.
Lang
, N.
Backmann
, D.
Rimoldi
, and C.
Gerber
, “Direct detection of a BRAF mutation in total RNA from melanoma cells using cantilever arrays
,” Nat. Nanotechnol.
8
(2
), 125
–129
(2013
).16.
E.-L.
Florin
, V. T.
Moy
, and H. E.
Gaub
, “Adhesion forces between individual ligand-receptor pairs
,” Science
264
(5157
), 415
–417
(1994
).17.
R.
Zhang
, Y.
Zhang
, Z. C.
Dong
, S.
Jiang
, C.
Zhang
, L. G.
Chen
, L.
Zhang
, Y.
Liao
, J.
Aizpurua
, Y.
Luo
et al, “Chemical mapping of a single molecule by plasmon-enhanced Raman scattering
,” Nature
498
(7452
), 82
–86
(2013
).18.
I. J.
Chen
, M.
Aapro
, A.
Kipnis
, A.
Ilin
, P.
Liljeroth
, and A. S.
Foster
, “Precise atom manipulation through deep reinforcement learning
,” Nat. Commun.
13
(1
), 7499
(2022
).19.
P.
Leinen
, M.
Esders
, K. T.
Schütt
, C.
Wagner
, K.-R.
Müller
, and F. S.
Tautz
, “Autonomous robotic nanofabrication with reinforcement learning
,” Sci. Adv.
6
(36
), eabb6987
(2020
).20.
J. A.
Stroscio
and D. M.
Eigler
, “Atomic and molecular manipulation with the scanning tunneling microscope
,” Science
254
(5036
), 1319
–1326
(1991
).21.
N.
Shumiya
, M. S.
Hossain
, J.-X.
Yin
, Z.
Wang
, M.
Litskevich
, C.
Yoon
, Y.
Li
, Y.
Yang
, Y.-X.
Jiang
, G.
Cheng
et al, “Evidence of a room-temperature quantum spin Hall edge state in a higher-order topological insulator
,” Nat. Mater.
21
(10
), 1111
–1115
(2022
).22.
X.-B.
Li
, W.-K.
Huang
, Y.-Y.
Lv
, K.-W.
Zhang
, C.-L.
Yang
, B.-B.
Zhang
, Y. B.
Chen
, S.-H.
Yao
, J.
Zhou
, M.-H.
Lu
, et al, “Experimental observation of topological edge states at the surface step edge of the topological insulator ZrTe5
,” Phys. Rev. Lett.
116
(17
), 176803
(2016
).23.
X.
Dong
, M.
Wang
, D.
Yan
, X.
Peng
, J.
Li
, W.
Xiao
, Q.
Wang
, J.
Han
, J.
Ma
, Y.
Shi
, and Y.
Yao
, “Observation of topological edge states at the step edges on the surface of type-II weyl semimetal TaIrTe4
,” ACS Nano
13
(8
), 9571
–9577
(2019
).24.
B.
Jäck
, Y.
Xie
, B.
Andrei Bernevig
, and A.
Yazdani
, “Observation of backscattering induced by magnetism in a topological edge state
,” Proc. Natl. Acad. Sci. U. S. A.
117
(28
), 16214
–16218
(2020
).25.
W.
Chen
, M.
Gabay
, and P. J.
Hirschfeld
, “Doping dependence of gap inhomogeneities at Bi2Sr2CaCu2Oδsurfaces
,” New J. Phys.
14
(3
), 033004
(2012
).26.
P. R.
Dunstan
, T. G. G.
Maffeïs
, M. P.
Ackland
, G. T.
Owen
, and S. P.
Wilks
, “The correlation of electronic properties with nanoscale morphological variations measured by SPM on semiconductor devices
,” J. Phys.: Condens. Matter
15
(42
), S3095
(2003
).27.
H.
Qiu
, X.
Dong
, J. H.
Shim
, J.
Cho
, and J. M.
Mativetsky
, “Effective charge collection area during conductive and photoconductive atomic force microscopy
,” Appl. Phys. Lett.
112
(26
), 263102
(2018
).28.
H.
Si
, S.
Zhang
, S.
Ma
, Z.
Xiong
, A.
Kausar
, Q.
Liao
, Z.
Zhang
, A.
Sattar
, Z.
Kang
, and Y.
Zhang
, “Emerging conductive atomic force microscopy for metal halide perovskite materials and solar cells
,” Adv. Energy Mater.
10
(10
), 1903922
(2020
).29.
Y.
Liu
, K. P.
Kelley
, R. K.
Vasudevan
, H.
Funakubo
, M. A.
Ziatdinov
, and S. V.
Kalinin
, “Experimental discovery of structure–property relationships in ferroelectric materials via active learning
,” Nat. Mach. Intell.
4
(4
), 341
–350
(2022
).30.
T.
Peters
, W.
Zhu
, M.
Checa
, L.
Collins
, and S.
Trolier-McKinstry
, “Influence of doping and thickness on domain avalanches in lead zirconate titanate thin films
,” Appl. Phys. Lett.
122
(13
), 132906
(2023
).31.
P. I.
Frazier
, “Bayesian optimization
,” in Recent Advances in Optimization and Modeling of Contemporary Problems
(Informs
, 2018
), pp. 255
–278
.32.
A. G.
Wilson
, Z.
Hu
, R.
Salakhutdinov
, and E. P.
Xing
, “Deep kernel learning
,” in Proceedings of the 19th International Conference on Artificial Intelligence and Statistics
, edited by G.
Arthur
and C. R.
Christian
(PMLR: Proceedings of Machine Learning Research
, 2016
), pp. 370
–378
.33.
C. K. I.
Williams
and D.
Barber
, “Bayesian classification with Gaussian processes
,” IEEE Trans. Pattern Anal. Mach. Intell.
20
(12
), 1342
–1351
(1998
).34.
S.
Rana
, C.
Li
, S.
Gupta
, V.
Nguyen
, and S.
Venkatesh
, “High dimensional Bayesian optimization with elastic Gaussian process
,” in Proceedings of the 34th International Conference on Machine Learning
, edited by P.
Doina
and T.
Yee Whye
(PMLR: Proceedings of Machine Learning Research
, 2017
), pp. 2883
–2891
.35.
M.
Ziatdinov
, O.
Dyck
, A.
Maksov
, X.
Li
, X.
Sang
, K.
Xiao
, R. R.
Unocic
, R.
Vasudevan
, S.
Jesse
, and S. V.
Kalinin
, “Deep learning of atomically resolved scanning transmission electron microscopy images: chemical identification and tracking local transformations
,” ACS Nano
11
(12
), 12742
–12752
(2017
).36.
J.
Sotres
, H.
Boyd
, and J. F.
Gonzalez-Martinez
, “Enabling autonomous scanning probe microscopy imaging of single molecules with deep learning
,” Nanoscale
13
(20
), 9193
–9203
(2021
).37.
Y.
Liu
, J.
Yang
, B. J.
Lawrie
, K. P.
Kelley
, M.
Ziatdinov
, S. V.
Kalinin
, and M.
Ahmadi
, “Disentangling electronic transport and hysteresis at individual grain boundaries in hybrid perovskites via automated scanning probe microscopy
,” ACS Nano
17
(10
), 9647
–9657
(2023
).38.
Y.
Liu
, K. P.
Kelley
, H.
Funakubo
, S. V.
Kalinin
, and M.
Ziatdinov
, “Exploring physics of ferroelectric domain walls in real time: Deep learning enabled scanning probe microscopy
,” Adv. Sci.
9
(31
), 2203957
(2022
).39.
Y.
Liu
, K. P.
Kelley
, R. K.
Vasudevan
, W.
Zhu
, J.
Hayden
, J.-P.
Maria
, H.
Funakubo
, M. A.
Ziatdinov
, S.
Trolier-McKinstry
, and S. V.
Kalinin
, “Automated experiments of local non-linear behavior in ferroelectric materials
,” Small
18
(48
), 2204130
(2022
).40.
Y.
Liu
, R.
Proksch
, C. Y.
Wong
, M.
Ziatdinov
, and S. V.
Kalinin
, “Disentangling ferroelectric wall dynamics and identification of pinning mechanisms via deep learning
,” Adv. Mater.
33
(43
), 2103680
(2021
).41.
Y.
Liu
, R. K.
Vasudevan
, K. P.
Kelley
, H.
Funakubo
, M.
Ziatdinov
, and S. V.
Kalinin
, “Learning the right channel in multimodal imaging: Automated experiment in piezoresponse force microscopy
,” npj Comput. Mater.
9
(1
), 34
(2023
).42.
Y.
Liu
, J.
Yang
, R. K.
Vasudevan
, K. P.
Kelley
, M.
Ziatdinov
, S. V.
Kalinin
, and M.
Ahmadi
, “Exploring the relationship of microstructure and conductivity in metal halide perovskites via active learning-driven automated scanning probe microscopy
,” J. Phys. Chem. Lett.
14
(13
), 3352
–3359
(2023
).43.
A.
Krull
, P.
Hirsch
, C.
Rother
, A.
Schiffrin
, and C.
Krull
, “Artificial-intelligence-driven scanning probe microscopy
,” Commun. Phys.
3
(1
), 54
(2020
).44.
Z.
Zhu
, J.
Lu
, F.
Zheng
, C.
Chen
, Y.
Lv
, H.
Jiang
, Y.
Yan
, A.
Narita
, K.
Müllen
, X.-Y.
Wang
, and Q.
Sun
, “A deep-learning framework for the automated recognition of molecules in scanning-probe-microscopy images
,” Angew. Chem.
134
(49
), e202213503
(2022
).45.
S.
Kandel
, T.
Zhou
, A. V.
Babu
, Z.
Di
, X.
Li
, X.
Ma
, M.
Holt
, A.
Miceli
, C.
Phatak
, and M. J.
Cherukara
, “Demonstration of an AI-driven workflow for autonomous high-resolution scanning microscopy
,” Nat. Commun.
14
(1
), 5501
(2023
).46.
W. K.
Szeremeta
, R. L.
Harniman
, C. R.
Bermingham
, and M.
Antognozzi
, “Towards a fully automated scanning probe microscope for biomedical applications
,” Sensors
21
(9
), 3027
(2021
).47.
M.
Rashidi
and R. A.
Wolkow
, “Autonomous scanning probe microscopy in situ tip conditioning through machine learning
,” ACS Nano
12
(6
), 5185
–5189
(2018
).48.
K.
Xu
, W.
Sun
, Y.
Shao
, F.
Wei
, X.
Zhang
, W.
Wang
, and P.
Li
, “Recent development of PeakForce Tapping mode atomic force microscopy and its applications on nanoscience
,” Nanotechnol. Rev.
7
(6
), 605
–621
(2018
).49.
W.
Xiang
, Y.
Tian
, and X.
Liu
, “Dynamic analysis of tapping mode atomic force microscope (AFM) for critical dimension measurement
,” Precis. Eng.
64
, 269
–279
(2020
).50.
B.
Anczykowski
, B.
Gotsmann
, H.
Fuchs
, J. P.
Cleveland
, and V. B.
Elings
, “How to measure energy dissipation in dynamic mode atomic force microscopy
,” Appl. Surf. Sci.
140
(3–4
), 376
–382
(1999
).51.
N. M.
Al Hasan
, H.
Hou
, T.
Gao
, J.
Counsell
, S.
Sarker
, S.
Thienhaus
, E.
Walton
, P.
Decker
, A.
Mehta
, A.
Ludwig
, and I.
Takeuchi
, “Combinatorial exploration and mapping of phase transformation in a Ni–Ti–Co thin film library
,” ACS Comb. Sci.
22
(11
), 641
–648
(2020
).52.
M.
Benz
, M. R.
Molla
, A.
Böser
, A.
Rosenfeld
, and P. A.
Levkin
, “Marrying chemistry with biology by combining on-chip solution-based combinatorial synthesis and cellular screening
,” Nat. Commun.
10
(1
), 2879
(2019
).53.
A. J.
Gormley
and M. A.
Webb
, “Machine learning in combinatorial polymer chemistry
,” Nat. Rev. Mater.
6
(8
), 642
–644
(2021
).54.
A.
Ludwig
, “Discovery of new materials using combinatorial synthesis and high-throughput characterization of thin-film materials libraries combined with computational methods
,” npj Comput. Mater.
5
(1
), 70
(2019
).55.
R. C.
Pullar
, Y.
Zhang
, L.
Chen
, S.
Yang
, J. R. G.
Evans
, A. N.
Salak
, D. A.
Kiselev
, A. L.
Kholkin
, V. M.
Ferreira
, and N. M.
Alford
, “Dielectric measurements on a novel Ba1−xCaxTiO3 (BCT) bulk ceramic combinatorial library
,” J. Electroceram.
22
(1–3
), 245
–251
(2008
).56.
M. A.
Ziatdinov
, Y.
Liu
, A. N.
Morozovska
, E. A.
Eliseev
, X.
Zhang
, I.
Takeuchi
, and S. V.
Kalinin
, “Hypothesis learning in automated experiment: Application to combinatorial materials libraries
,” Adv. Mater.
34
(20
), e2201345
(2022
).57.
S. V.
Kalinin
, Y.
Liu
, A.
Biswas
, G.
Duscher
, U.
Pratiush
, K.
Roccapriore
, M.
Ziatdinov
, and R.
Vasudevan
, “Human-in-the-loop: The future of machine learning in automated electron microscopy
,” Microscopy Today
32
(1
), 35
–41
(2024
).58.
S. V.
Kalinin
, M.
Ziatdinov
, M.
Ahmadi
, A.
Ghosh
, K.
Roccapriore
, Y.
Liu
, and R. K.
Vasudevan
, “Designing workflows for materials characterization
,” Appl. Phys. Rev.
11
(1
), 011314
(2024
).© 2024 Author(s). Published under an exclusive license by AIP Publishing.
2024
Author(s)
You do not currently have access to this content.
