An active learning strategy using sampling based on uncertainties shows the promise of accelerating the development of new materials. We study the efficiencies of the active learning iteration loop with different uncertainty estimators to find the “best” material in four different experimental datasets. We use a bootstrap approach aggregating with support vector regression as the base learner to obtain uncertainties associated with model predictions. If the bootstrap replicate number is small, the variance estimated by the empirical standard error estimator is found to be close to the true variance, whereas the jackknife based estimators give an upward or downward biased estimation of variance. As increases, the bias of the jackknife based estimators decreases and the variance estimated finally converges to the true one. Therefore, the empirical standard error estimator needs the least number of iteration loops to find the best material in the datasets, especially when the bootstrap replicate number is small. Our work demonstrates that an appropriate Bootstrap replicate is conducive to minimizing calculation costs during the materials property optimization by active learning.
REFERENCES
1.
C.
Shen
, C.
Wang
, X.
Wei
, Y.
Li
, S.
van der Zwaag
, and W.
Xu
, “Physical metallurgy-guided machine learning and artificial intelligent design of ultrahigh-strength stainless steel
,” Acta Mater.
179
, 201
–214
(2019
). 2.
J.
Im
, S.
Lee
, T.-W.
Ko
, H. W.
Kim
, Y.
Hyon
, and H.
Chang
, “Identifying Pb-free perovskites for solar cells by machine learning
,” npj Comput. Mater.
5
, 37
(2019
). 3.
V.
Stanev
, C.
Oses
, A. G.
Kusne
, E.
Rodriguez
, J.
Paglione
, S.
Curtarolo
, and I.
Takeuchi
, “Machine learning modeling of superconducting critical temperature
,” npj Comput. Mater.
4
, 29
(2018
). 4.
A.
Mannodi-Kanakkithodi
, G.
Pilania
, T. D.
Huan
, T.
Lookman
, and R.
Ramprasad
, “Machine learning strategy for accelerated design of polymer dielectrics
,” Sci. Rep.
6
, 20952
(2016
). 5.
G.
Pilania
, C.
Wang
, X.
Jiang
, S.
Rajasekaran
, and R.
Ramprasad
, “Accelerating materials property predictions using machine learning
,” Sci. Rep.
3
, 2810
(2013
). 6.
R.
Yuan
, Y.
Tian
, D.
Xue
, D.
Xue
, Y.
Zhou
, X.
Ding
, J.
Sun
, and T.
Lookman
, “Accelerated search for BaTiO-based ceramics with large energy storage at low fields using machine learning and experimental design
,” Adv. Sci.
6
(21
), 1901395
(2019
). 7.
D.
Xue
, D.
Xue
, R.
Yuan
, Y.
Zhou
, P. V.
Balachandran
, X.
Ding
, J.
Sun
, and T.
Lookman
, “An informatics approach to transformation temperatures of NiTi-based shape memory alloys
,” Acta Mater.
125
, 532
–541
(2017
). 8.
T.
Lookman
, P. V.
Balachandran
, D.
Xue
, and R.
Yuan
, “Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design
,” npj Comput. Mater.
5
, 21
(2019
). 9.
P. V.
Balachandran
, D.
Xue
, J.
Theiler
, J.
Hogden
, and T.
Lookman
, “Adaptive strategies for materials design using uncertainties
,” Sci. Rep.
6
, 19660
(2016
). 10.
T.
Lookman
, P. V.
Balachandran
, D.
Xue
, J.
Hogden
, and J.
Theiler
, “Statistical inference and adaptive design for materials discovery
,” Curr. Opin. Solid State Mater. Sci.
21
(3
), 121
–128
(2017
). 11.
G.
Pilania
, A.
Mannodi-Kanakkithodi
, B. P.
Uberuaga
, R.
Ramprasad
, J. E.
Gubernatis
, and T.
Lookman
, “Machine learning bandgaps of double perovskites
,” Sci. Rep.
6
, 19375
(2016
). 12.
J.
Schmidt
, M. R. G.
Marques
, S.
Botti
, and M. A. L.
Marques
, “Recent advances and applications of machine learning in solid-state materials science
,” npj Comput. Mater.
5
, 83
(2019
). 13.
R.
Dehghannasiri
, D.
Xue
, P. V.
Balachandran
, M. R.
Yousefi
, L. A.
Dalton
, T.
Lookman
, and E. R.
Dougherty
, “Optimal experimental design for materials discovery
,” Comput. Mater. Sci.
129
, 311
–322
(2017
). 14.
D. A.
Cohn
, Z.
Ghahramani
, and M. I.
Jordan
, “Active learning with mixture models,” in Multiple Model Approaches to Modelling and Control, edited by R. Murray-Smith and T. A. Johansen (Taylor & Francis, 1997), pp. 167–183.15.
L.
Bassman
, P.
Rajak
, R. K.
Kalia
, A.
Nakano
, F.
Sha
, J.
Sun
, D. J.
Singh
, M.
Aykol
, P.
Huck
, K.
Persson
, and P.
Vashishta
, “Active learning for accelerated design of layered materials
,” npj Comput. Mater.
4
, 74
(2018
). 16.
J.
Theiler
and B. G.
Zimmer
, “Selecting the selector: Comparison of update rules for discrete global optimization
,” Stat. Anal. Data Mining: ASA Data Sci. J.
10
(4
), 211
–229
(2017
). 17.
D.
Xue
, P. V.
Balachandran
, J.
Hogden
, J.
Theiler
, D.
Xue
, and T.
Lookman
, “Accelerated search for materials with targeted properties by adaptive design
,” Nat. Commun.
7
, 11241
(2016
). 18.
R.
Yuan
, Z.
Liu
, P. V.
Balachandran
, D.
Xue
, Y.
Zhou
, X.
Ding
, J.
Sun
, D.
Xue
, and T.
Lookman
, “Accelerated discovery of large electrostrains in BaTiO-based piezoelectrics using active learning
,” Adv. Mater.
30
(7
), 1702884
(2018
). 19.
C.
Wen
, Y.
Zhang
, C.
Wang
, D.
Xue
, Y.
Bai
, S.
Antonov
, L.
Dai
, T.
Lookman
, and Y.
Su
, “Machine learning assisted design of high entropy alloys with desired property
,” Acta Mater.
170
, 109
–117
(2019
). 20.
D. R.
Jones
, M.
Schonlau
, and W. J.
Welch
, “Efficient global optimization of expensive black-box functions
,” J. Glob. Optim.
13
(4
), 455
–492
(1998
). 21.
K.
Terayama
, R.
Tamura
, Y.
Nose
, H.
Hiramatsu
, H.
Hosono
, Y.
Okuno
, and K.
Tsuda
, “Efficient construction method for phase diagrams using uncertainty sampling
,” Phys. Rev. Mater.
3
, 033802
(2019
). 22.
B.
Efron
, “Estimation and accuracy after model selection
,” J. Am. Stat. Assoc.
109
(507
), 991
–1007
(2014
). 23.
J.
Sexton
and P.
Laake
, “Standard errors for bagged and random forest estimators
,” Comput. Stat. Data Anal.
53
(3
), 801
–811
(2009
). 24.
B.
Efron
and C.
Stein
, “The jackknife estimate of variance
,” Ann. Stat.
9
(3
), 586
–596
(1981
). 25.
J.
Ling
, M.
Hutchinson
, E.
Antono
, S.
Paradiso
, and B.
Meredig
, “High-dimensional materials and process optimization using data-driven experimental design with well-calibrated uncertainty estimates
,” Integr. Mater. Manuf. Innov.
6
(3
), 207
–217
(2017
). 26.
U.
Beyaztas
and A.
Alin
, “Sufficient jackknife-after-bootstrap method for detection of influential observations in linear regression models
,” Stat. Pap.
55
(4
), 1001
–1018
(2014
). 27.
R. I.
Jennrich
, “Nonparametric estimation of standard errors in covariance analysis using the infinitesimal jackknife
,” Psychometrika
73
(4
), 579
–594
(2008
). 28.
C.
Brokampa
, M. B.
Rao
, “P.
Ryan
, and R.
Jandarov
, “A comparison of resampling and recursive partitioning methods in random forest for estimating the asymptotic variance using the infinitesimal jackknife
,” STAT
6
(1
), 360
–372
(2017
). 29.
S.
Wager
, T.
Hastie
, and B.
Efron
, “Confidence intervals for random forests: The jackknife and the infinitesimal jackknife
,” J. Mach. Learn Res.
15
(1
), 1625
–1651
(2014
).30.
H. X.
Yan
, H. T.
Zhang
, R.
Ubic
, M. J.
Reece
, J.
Liu
, Z.
Shen
, and Z.
Zhang
, “A lead-free high-Curie-point ferroelectric ceramic, CaBiNbO
,” Adv. Mater.
17
(10
), 1261
–1265
(2005
). 31.
R.
Yuan
, D.
Xue
, D.
Xue
, Y.
Zhou
, X.
Ding
, J.
Sun
, and T.
Lookman
, “The search for BaTiO-based piezoelectrics with large piezoelectric coefficient using machine learning
,” IEEE Trans. Ultrason. Ferr.
66
(2
), 394
–401
(2019
). 32.
L.
Breiman
, “Bagging predictors
,” Mach. Learn.
24
(2
), 123140
(1996
). 33.
M.
Awad
and R.
Khanna
, Support Vector Regression
(Apress
, Berkeley, CA
, 2015
), pp. 67
–80
.34.
L.
Xu
, L.
Wencong
, J.
Shengli
, L.
Yawei
, and C.
Nianyi
, “Support vector regression applied to materials optimization of SiAlON ceramics
,” Chemom. Intell. Lab. Syst.
82
(1
), 8
–14
(2006
). 35.
I. O.
Alade
, I.
Olumegbon
, and A.
Bagudu
, “Lattice constant prediction of A(2)XY(6) cubic crystals (, Cs, Rb, TI; ; , Cl, Br, I) using computational intelligence approach
,” J. Appl. Phys.
127
(1
), 015303
(2020
). © 2020 Author(s).
2020
Author(s)
You do not currently have access to this content.