Neuro-Bach and Neuro-Mozart as tools for teaching music production
The study of music theory and the history of global musical culture within the framework of teaching students in the supplementary education program «Music Production» for the disciplines of «Composition» and «Arrangement» necessitates the adaptation of theoretical material to current methods of consuming educational content.
In the past 5-7 years, there has been a growing interest among educators in higher and supplementary education in the application of edutainment strategies in the teaching of fine arts and humanities disciplines. This trend corresponds with the distinctive consumption patterns of educational content by students from Generation Z and Generation Alpha. In this context, a popular format for presenting theoretical content has become educational videos, which, since 2023, with the introduction of generative neural networks into the creative industries, can be expanded to consider the application of AI avatars in teaching.
The projects «Neuro-Bach» and «Neuro-Mozart» describe an experiment we conducted in the fall of 2024, involving students from the «Music Production» supplementary education program at the School of Design of the National Research University Higher School of Economics and first-year students from the Faculty of Law at the Russian State Academy of Intellectual Property.
Avatars of historical figures retell music theory
The idea of researching the possibilities of AI avatars created from historical figures who significantly contributed to the development of music theory and musical composition arose from observing students' interest in the capabilities of generative neural networks for creating deepfakes.
To illustrate theoretical material on the history of technological innovations in musical composition, we brought to life, using the generative neural network HEDRA, the portraits of Pythagoras, Euripides, A. Kircher, W.A. Mozart, and R. Zaripov. For each historical figure, we generated speech fragments through the GPT Mini chatbot, based on the context of their conceptual views available in open sources.
The resulting video scenes, featuring educational content, were included as video illustrations in the course «Arrangement» for students of the advanced training program «Music Production» at the School of Design of the National Research University Higher School of Economics, as well as in the video course «Phygital in Music» for Tatiana Khrapko’s educational project «Phygiathlon.»
Evgeniya Evpak (AI avatar). Video lecture «Why did human being start composing music at the early stage of cultural civilizations». (in Russ.). Educational project «Phygiathlon». VK Video. November, 15, 2024.
Digitized Mozart can provide a review on any musical project.
In the introductory video for the course «Phygital in Music», we made an attempt to utilize an animated avatar of W. A. Mozart to obtain a review of a musical project from a classical composer, brought to life through prompt engineering methods. The «revival» employed the GPT-mini chatbot and the generative neural network HEDRA.
The outcome of the «revival» of the generative Mozart (time codes 02:26 — 04:22 in the video) met our expectations: the review provided by the neural network for the musical composition, based on a contextually conditioned prompt, closely resembled the thought style of the Austrian composer.
Evgeniya Evpak (AI Avatar). Video Lecture «Phygital in Music. The Digital Avatar of the Artist.» (In Russ.). Educational Project «PHYGIATHLON.» VK Video. October 29, 2024.
Experiment with RSAIP Students: Expectations from Educational AI Videos
On November 12, 2024, students from the Faculty of Law at the Russian State Academy of Intellectual Property, aged between 17 and 23, participated in a survey aimed at assessing the level of trust in educational video content featuring animated historical figures and virtual avatars of instructors in theoretical disciplines.
The purpose of the survey was to identify the criteria that influence students' trust in virtual avatars of instructors and their willingness to engage with educational content created using AI.
The music literacy levels of respondents were represented by 5 groups: (1) 8,4% — graduated Bachelor, Master, Ph.D. in Music; (2) 8,4% — graduated vocational training program in Music; (3) 53,8% — studied Music in music school; (4) 10,4% — studied Music with private lessons; (5) 19% — without any background in Music.
During the survey, 48 respondents (46 students and 2 lecturers) — 60,4% females and 39,6% males — provided detailed answers reflecting on example educational videos for the course «Phygital in Music». The questions addressed issues such as students' readiness to receive a theoretical lecture on the history of music from the perspective of an animated historical figure, their openness to attending virtual multimedia phygital exhibitions in museums, their overall attitude towards the format of video lectures as educational content post-pandemic, predictions regarding students' reactions to a video lecture delivered by a virtual avatar of a selected university instructor, and the preferred duration of videos featuring virtual avatars, among other topics.
Survey outcomes
It was revealed, that 31,3% respondents (15 people) previously had relevant experience in watching educational videos with virtual avatars.
Attitudes of the respondents to efficiency of video lectures as educational content format elucidated 2 trends: (a) 56,3% respondents (27 people) agree that video lectures are efficient because a student can always press «stop» and «replay» button, (b) 37,5% respondents (18 people) think that offline dialogue format is more efficient for students, and 6,3% respondents (3 people) agree to obtain knowledge only from traditional offline lectures and seminars.
Readiness of students to watch educational content with AI avatars showed 4 trends: (a) 68,8% respondents (33 people) would be interested to consume the educational course with AI avatars, (b) for 16,7% (8 people) readiness to consume AI-generated educational content depends on the expertise of AI-simulated lecturer, (c) 12,5% (6 people) would prefer reading textbooks and listening to human lecturers, (d) and 2,1% (1 person) told that «the uncanny valley effect will distract him from the lecture».
An ideal duration of AI educational video, according to the opinions of respondents, makes: - No longer than 15 minutes — 18,8% (9 people); - No longer than 10-12 minutes — 27,1% (13 people); - Ready to watch AI educational video of any duration — 14,6% (7 people); - No longer than 5 minutes — 18,8% (9 people); - No longer than 3 minutes — 8,3% (4 people).
What if lectures on music theory from now on shall be conducted by a virtual avatar of the instructor?
Under these conditions, students agree to listen to video lectures delivered by a virtual avatar of the music theory instructor.
Conclusions: What should the ideal educational video on music theory featuring AI avatars entail?
Musical Educational Video in the Manner of Public Request: Neuro-Bach Teaches How to Compose Fugues in Bach Style.
Inspired by the wishes of the students at the Russian State Academy of Intellectual Property, we have endeavored to adapt a new instructional video on music theory to the criteria of effective educational material. Utilizing the SUNO neural network, we generated a musical prototype in which Johann Sebastian Bach narrates, in his own voice, how to compose fugues in Bach style.
The instructional video carries a humorous tone due to several factual inaccuracies inherent to neural networks, particularly regarding the compositional and tonal structure of the musical material, which in strict style fugues should conclude in a major key and not be accompanied by a vocal part.
We believe this aspect will be beneficial for oral analysis in face-to-face lectures on harmony and polyphony within the framework of the composition and arrangement course, serving as a valuable didactic resource.
To be continued…
