Visualizing 3D molecular structures is crucial to understanding and predicting their chemical behavior. However, static 2D hand-drawn skeletal structures remain the preferred method of chemical communication. Here, we combine cutting-edge technologies in augmented reality (AR), machine learning, and computational chemistry to develop MolAR, an open-source mobile application for visualizing molecules in AR directly from their hand-drawn chemical structures. Users can also visualize any molecule or protein directly from its name or protein data bank ID and compute chemical properties in real time via quantum chemistry cloud computing. MolAR provides an easily accessible platform for the scientific community to visualize and interact with 3D molecular structures in an immersive and engaging way.
REFERENCES
1.
L.
Pray
, “Discovery of DNA structure and function: Watson and Crick
,” Nat. Educ.
1
, 100
(2008
), available at https://www.nature.com/scitable/topicpage/discovery-of-dna-structure-and-function-watson-397/.2.
E.
Francoeur
, “The forgotten tool: The design and use of molecular models
,” Soc. Stud. Sci.
27
, 7
–40
(1997
).3.
A. J.
Olson
, “Perspectives on structural molecular biology visualization: From past to present
,” J. Mol. Biol.
430
, 3997
–4012
(2018
).4.
R.
Skibba
, “Virtual reality comes of age
,” Nature
553
, 402
–404
(2018
).5.
F. C.
Rodríguez
, M.
Dal Peraro
, and L. A.
Abriata
, “Democratizing interactive, immersive experiences for science education with WebXR
,” Nat. Comput. Sci.
1
, 631
–632
(2021
).6.
S.
Singhal
, S.
Bagga
, P.
Goyal
, and V.
Saxena
, “Augmented chemistry: Interactive education system
,” Int. J. Comput. Appl.
49
, 1
(2012
).7.
H.-K.
Wu
, S. W.-Y.
Lee
, H.-Y.
Chang
, and J.-C.
Liang
, “Current status, opportunities and challenges of augmented reality in education
,” Comput. Educ.
62
, 41
–49
(2013
).8.
H. S.
Fernandes
, N. M. F. S. A.
Cerqueira
, and S. F.
Sousa
, “Developing and using BioSIMAR, an augmented reality program to visualize and learn about chemical structures in a virtual environment on any internet-connected device
,” J. Chem. Educ.
98
, 1789
–1794
(2021
).9.
J. R.
Schmid
, M. J.
Ernst
, and G.
Thiele
, “Structural chemistry 2.0: Combining augmented reality and 3D online models
,” J. Chem. Educ.
97
, 4515
–4519
(2020
).10.
K.
Eriksen
, B. E.
Nielsen
, and M.
Pittelkow
, “Visualizing 3D molecular structures using an augmented reality app
,” J. Chem. Educ.
97
, 1487
–1490
(2020
).11.
S.
Yang
, B.
Mei
, and X.
Yue
, “Mobile augmented reality assisted chemical education: Insights from elements 4D
,” J. Chem. Educ.
95
, 1060
–1062
(2018
).12.
R.-J.
Sung
, A. T.
Wilson
, S. M.
Lo
, L. M.
Crowl
, J.
Nardi
, K.
St. Clair
, and J. M.
Liu
, “BiochemAR: An augmented reality educational tool for teaching macromolecular structure and function
,” J. Chem. Educ.
97
, 147
–153
(2020
).13.
P.
Wolle
, M. P.
Müller
, and D.
Rauh
, “Augmented reality in scientific publications—Taking the visualization of 3D structures to the next level
,” ACS Chem. Biol.
13
, 496
–499
(2018
).14.
J. M.
Argüello
and R. E.
Dempski
, “Fast, simple, student generated augmented reality approach for protein visualization in the classroom and home study
,” J. Chem. Educ.
97
, 2327
–2331
(2020
).15.
J. K.
Aw
, K. C.
Boellaard
, T. K.
Tan
, J.
Yap
, Y. P.
Loh
, B.
Colasson
, É.
Blanc
, Y.
Lam
, and F. M.
Fung
, “Interacting with three-dimensional molecular structures using an augmented reality mobile app
,” J. Chem. Educ.
97
, 3877
–3881
(2020
).16.
B.
Sanii
, “Creating augmented reality USDZ files to visualize 3D objects on student phones in the classroom
,” J. Chem. Educ.
97
, 253
–257
(2020
).17.
F. C.
Rodríguez
, G.
Frattini
, L. F.
Krapp
, H.
Martinez-Hung
, D. M.
Moreno
, M.
Roldán
, J.
Salomón
, L.
Stemkoski
, S.
Traeger
, M.
Dal Peraro
, and L. A.
Abriata
, “MoleculARweb: A web site for chemistry and structural biology education through interactive augmented reality out of the box in commodity devices
,” J. Chem. Educ.
98
, 2243
–2255
(2021
).18.
J. D.
Hirst
, D. R.
Glowacki
, and M.
Baaden
, “Molecular simulations and visualization: Introduction and overview
,” Faraday Discuss.
169
, 9
–22
(2014
).19.
Z. A.
Jiménez
, “Teaching and learning chemistry via augmented and immersive virtual reality
,” in Technology Integration in Chemistry Education and Research (TICER)
(American Chemical Society
, 2019
), Vol. 1318, pp. 31
–52
.20.
S.
Cai
, X.
Wang
, and F.-K.
Chiang
, “A case study of augmented reality simulation system application in a chemistry course
,” Comput. Human Behav.
37
, 31
–40
(2014
).21.
M.
Chen
and B.
Liao
, “Augmented reality laboratory for high school electrochemistry course
,” in 2015 IEEE 15th International Conference on Advanced Learning Technologies, 6–9 July 2015
(IEEE
, 2015
), pp. 132
–136
.22.
H.
Hou
and Y.
Lin
, “The development and evaluation of an educational game integrated with augmented reality and virtual laboratory for chemistry experiment learning
,” in 2017 6th IIAI International Congress on Advanced Applied Informatics (IIAI-AAI), 9–13 July 2017
(IEEE
, 2017
), pp. 1005
–1006
.23.
D. D.
Iordache
, C.
Pribeanu
, and A.
Balog
, “Influence of specific AR capabilities on the learning effectiveness and efficiency
,” Stud. Inf. Control
21
, 233
–240
(2012
).24.
K. L.
Narasimha Swamy
, P. S.
Chavan
, and S.
Murthy
, “StereoChem: Augmented reality 3D molecular model visualization app for teaching and learning stereochemistry
,” in 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT), 9–13 July 2018
(IEEE
, 2018
), pp. 252
–256
.25.
P.
Maier
and G.
Klinker
, “Augmented chemical reactions: An augmented reality tool to support chemistry teaching
,” in 2013 2nd Experiment@ International Conference (exp.At’13), 18–20 September 2013
(IEEE
, 2013
), pp. 164
–165
.26.
A.
Nachairit
and N.
Srisawasdi
, “Using mobile augmented reality for chemistry learning of acid-base titration: Correlation between motivation and perception
,” in Proceedings of the 23rd International Conference on Computers in Education
(Asia-Pacific Society for Computers in Education
, Hangzhou
, 2015
), pp. 519
–528
.27.
K. N.
Plunkett
, “A simple and practical method for incorporating augmented reality into the classroom and laboratory
,” J. Chem. Educ.
96
, 2628
–2631
(2019
).28.
L. M. M. S. A.
Qassem
, H. A.
Hawai
, S.
AlShehhi
, M. J.
Zemerly
, and J. W. P.
Ng
, “AIR-EDUTECH: Augmented immersive reality (AIR) technology for high school chemistry education
,” in 2016 IEEE Global Engineering Education Conference (EDUCON), 10–13 April 2016
(IEEE
, 2016
), pp. 842
–847
.29.
C. N.
Peterson
, S. Z.
Tavana
, O. P.
Akinleye
, W. H.
Johnson
, and M. B.
Berkmen
, “An idea to explore: Use of augmented reality for teaching three-dimensional biomolecular structures
,” Biochem. Mol. Biol. Educ.
48
, 276
–282
(2020
).30.
M.
Uselton
, “MoleculAR geometry: Chemical visualizations in augmented reality
,” Honors College of Middle Tennessee State University, 2020
.31.
M.
Zheng
and M. P.
Waller
, “ChemPreview: An augmented reality-based molecular interface
,” J. Mol. Graphics Modell.
73
, 18
–23
(2017
).32.
B.
Zhu
, M.
Feng
, H.
Lowe
, J.
Kesselman
, L.
Harrison
, and R. E.
Dempski
, “Increasing enthusiasm and enhancing learning for biochemistry-laboratory safety with an augmented-reality program
,” J. Chem. Educ.
95
, 1747
–1754
(2018
).33.
M.
Coster
, “MoleculAR: An augmented reality app for organic chemistry
,” https://organicchemexplained.com/molecular-augmented-reality-app/; accessed 24 November 2021.34.
See https://play.google.com/store/apps/details?id=club.amase.mocubed for MO-cubed.
35.
CRISPR-3D. See https://innovativegenomics.org/crispr-3d
36.
See https://apps.apple.com/us/app/molar-augmented-reality/id1559504847 to download the MolAR app for iOS devices; accessed 24 November 2021.
37.
H.
Weir
, K.
Thompson
, A.
Woodward
, B.
Choi
, A.
Braun
, and T. J.
Martínez
, “ChemPix: Automated recognition of hand-drawn hydrocarbon structures using deep learning
,” Chem. Sci.
12
, 10622
–10633
(2021
).38.
K.
Cunningham
, “Convert printed and handwritten chemical diagrams to SMILES
,” https://mathpix.com/blog/handwritten-chem-diagrams; accessed 24 November 2021.39.
S.
Seritan
, K.
Thompson
, and T. J.
Martínez
, “TeraChem cloud: A high-performance computing service for scalable distributed GPU-accelerated electronic structure calculations
,” J. Chem. Inf. Model.
60
, 2126
–2137
(2020
).40.
National Cancer Institute
, Online SMILES Translator, https://cactus.nci.nih.gov/translate/index.html; accessed 16 August 2021.41.
U.
Raucci
, A.
Valentini
, E.
Pieri
, H.
Weir
, S.
Seritan
, and T. J.
Martínez
, “Voice-controlled quantum chemistry
,” Nat. Comput. Sci.
1
, 42
–45
(2021
).42.
Apple ARKit
, https://developer.apple.com/documentation/arkit; accessed 16 August 2021.43.
Pixar Animation Studios
, Introduction to USD, https://graphics.pixar.com/usd/docs/Introduction-to-USD.html; accessed 16 August 2021.44.
D.
Sehnal
, S.
Bittrich
, M.
Deshpande
, R.
Svobodová
, K.
Berka
, V.
Bazgier
, S.
Velankar
, S. K.
Burley
, J.
Koča
, and A. S.
Rose
, “Mol* viewer: Modern web app for 3D visualization and analysis of large biomolecular structures
,” Nucleic Acids Res.
49
, W431
–W437
(2021
).45.
Amazon Web Services
. Detecting labels, https://docs.aws.amazon.com/rekognition/latest/dg/labels.html; accessed 16 August 2021.46.
Google,
Detect Labels, https://cloud.google.com/vision/docs/labels; accessed 16 August 2021.© 2022 Author(s). Published under an exclusive license by AIP Publishing.
2022
Author(s)
You do not currently have access to this content.
