We describe a project to create a rhythm video game in which players can use real instruments to play with music streamed by the game, capturing sound through the microphone available in mobile devices or computers, and using machine learning to analyze and detect chords played by the real instrument. The game also uses visual, auditory, and tactile elements to guide and support instrumental study.
REFERENCES
1.
A. C.
North
, D. J.
Hargreaves
, and S. A.
O’Neill
, “The importance of music to adolescents
,” British J. of Educational Psychology
70
, 255
–272
(2000
).2.
M.
Sharp
, R.
Ak
, and T.
Hedberg
Jr, “A survey of the advancing use and development of machine learning in smart manufacturing
,” Manu-facturing Systems
48
, 170
–179
(2018
).3.
K.
Arulkumaran
, M. P.
Deisenroth
, M.
Brundage
, and A. A.
Bharath
, “Deep reinforcement learning: A brief survey
,” IEEE Signal Processing Magazine
34
, 26
–38
(2017
).4.
T. S.
Hoang
and M.
Cohen
, “Multimodal Metronome — Rhythm game for musical instruments,” in SHS Web of Conf.: ETLTC Int. Conf. on ICT Integration in Technical Education
, Vol. 139
, edited by Debopriyo
Roy
(Aizu-Wakamatsu
, Japan
, 2022
) p. 03024
.5.
6.
7.
mbaske
, “Chord detection with unity ml-agents (audio sensor)
,” https://github.com/mbaske/ml-chord-detection (2020
).8.
J.
Jaime
, I.
Barbancho
, C.
Urdiales
, L. J.
Tardón
, and A. M.
Barbancho
, “A new multiformat rhythm game for music tutoring
,” Multimedia Tools and Applications
75
, 4349
–4362
(2016
).9.
I.
Granic
, A.
Lobel
, and R. C.
Engels
, “The benefits of playing video games
,” American Psychologist
69
, 66
(2014
).10.
S.-I.
Chiu
, J.-Z.
Lee
, and D.-H.
Huang
, “Video game addiction in children and teenagers in taiwan
,” CyberPsychology & Behavior
7
, 571
–581
(2004
).
This content is only available via PDF.
© 2023 AIP Publishing LLC.
2023
AIP Publishing LLC
You do not currently have access to this content.
