Shaping musical vibratos using multi-modal pedagogical interactions

The music vibrato is termed a “pulsation in pitch, intensity, and timbre” because of its effectiveness in artistic rendering. However, this sonic trick is largely still a challenge in music pedagogy across music conservatories. In classroom practice, music teachers use demonstration, body gestures, and metaphors to convey their artistic intentions and the modern computer tools are seldom employed. In our proposed framework, we use musical vibrato visualization and sonification tools as a multi-modal computer interface for pedagogical purpose. Specifically, we compare master performance audio with student performance audio using signal analysis tools. Then, we obtain various similarity measures based on these signal analysis results. Based on these similarity measures we implement multi-modal interactions for music students to shape their music learning process. The visualization interface is based on audio features including dynamics, pitch and timbre. The sonifications interface is based on recorded audio and synthesized audio. To enhance the music relevance of our proposed framework, both visualization and sonification tools are targeted to serve a musical communicating to convey musical concepts in an intuitive manner. The proposed framework is evaluation using subjective ratings from music students and objective assessment of measurable training goals.