{"id":12158,"date":"2022-04-20T10:24:27","date_gmt":"2022-04-20T07:24:27","guid":{"rendered":"http:\/\/journals.khnu.km.ua\/vestnik\/?p=12158"},"modified":"2022-04-20T10:24:27","modified_gmt":"2022-04-20T07:24:27","slug":"tehnologiya-audiogeneracziyi-systemy-syntezu-ta-analizu-muzychnyh-kompozyczij","status":"publish","type":"post","link":"https:\/\/journals.khnu.km.ua\/vestnik\/?p=12158","title":{"rendered":"\u0422\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0456\u044f \u0430\u0443\u0434\u0456\u043e\u0433\u0435\u043d\u0435\u0440\u0430\u0446\u0456\u0457 \u0441\u0438\u0441\u0442\u0435\u043c\u0438 \u0441\u0438\u043d\u0442\u0435\u0437\u0443 \u0442\u0430 \u0430\u043d\u0430\u043b\u0456\u0437\u0443 \u043c\u0443\u0437\u0438\u0447\u043d\u0438\u0445 \u043a\u043e\u043c\u043f\u043e\u0437\u0438\u0446\u0456\u0439"},"content":{"rendered":"

<\/p>\n

\u0422\u0415\u0425\u041d\u041e\u041b\u041e\u0413\u0406\u042f \u0410\u0423\u0414\u0406\u041e\u0413\u0415\u041d\u0415\u0420\u0410\u0426\u0406\u0407 \u0421\u0418\u0421\u0422\u0415\u041c\u0418 \u0421\u0418\u041d\u0422\u0415\u0417\u0423 \u0422\u0410 \u0410\u041d\u0410\u041b\u0406\u0417\u0423 \u041c\u0423\u0417\u0418\u0427\u041d\u0418\u0425 \u041a\u041e\u041c\u041f\u041e\u0417\u0418\u0426\u0406\u0419<\/p>\n

AUDIO GENERATION TECHNOLOGY OF A SYSTEM OF SYNTHESIS AND ANALYSIS OF MUSIC COMPOSITIONS<\/p>\n

\u0421\u0442\u043e\u0440\u0456\u043d\u043a\u0438: 64-67. \u041d\u043e\u043c\u0435\u0440: \u2116<\/strong>1<\/strong>, 202<\/strong>2<\/strong> (<\/strong>305<\/strong>)\u00a0 \"\"<\/a> <\/strong>
\n\u00a0<\/strong>\u0410\u0432\u0442\u043e\u0440\u0438:<\/strong>
\n\u0412\u041e\u0419\u0422\u041a\u041e \u0412. \u0412.
\nhttps:\/\/orcid.org\/0000-0002-3329-7256<\/a>
\ne-mail: defakfki@i.ua<\/a>
\n\u0411\u0415\u0412\u0417 \u0421. \u0412.
\nhttps:\/\/orcid.org\/0000-0001-5561-9036<\/a>
\ne-mail: bevz@vntu.edu.ua<\/a>
\n\u0411\u0423\u0420\u0411\u0415\u041b\u041e \u0421. \u041c.
\nhttps:\/\/orcid.org\/0000-0002-8554-2292<\/a>
\ne-mail: burbelo@vntu.edu.ua<\/a>
\n\u0421\u0422\u0410\u0412\u0418\u0426\u042c\u041a\u0418\u0419 \u041f. \u0412.
\nhttps:\/\/orcid.org\/0000-0002-9139-6076<\/a>
\ne-mail: morfly3000@gmail.com<\/a>
\n\u0412\u0456\u043d\u043d\u0438\u0446\u044c\u043a\u0438\u0439 \u043d\u0430\u0446\u0456\u043e\u043d\u0430\u043b\u044c\u043d\u0438\u0439 \u0442\u0435\u0445\u043d\u0456\u0447\u043d\u0438\u0439 \u0443\u043d\u0456\u0432\u0435\u0440\u0441\u0438\u0441\u0442\u0435\u0442
\nViktoriia VOITKO, Svitlana BEVZ, Sergii BURBELO, Pavlo STAVYTSKYI
\nVinnytsia National Technical University
\nDOI:<\/strong> https:\/\/www.doi.org\/10.31891\/2307-5732-2022-305-1-64-67<\/a><\/p>\n

\u0410\u043d\u043e\u0442\u0430\u0446\u0456\u044f \u043c\u043e\u0432\u043e\u044e \u043e\u0440\u0438\u0433\u0456\u043d\u0430\u043b\u0443<\/strong><\/p>\n

\u0420\u043e\u0437\u0433\u043b\u044f\u043d\u0443\u0442\u043e \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0456\u044e \u0442\u0430 \u0441\u043f\u043e\u0441\u043e\u0431\u0438 \u0430\u0443\u0434\u0456\u043e\u0433\u0435\u043d\u0435\u0440\u0430\u0446\u0456\u0457 \u0432 \u0441\u0438\u0441\u0442\u0435\u043c\u0456 \u0441\u0438\u043d\u0442\u0435\u0437\u0443 \u0442\u0430 \u0430\u043d\u0430\u043b\u0456\u0437\u0443 \u043c\u0443\u0437\u0438\u0447\u043d\u0438\u0445 \u043a\u043e\u043c\u043f\u043e\u0437\u0438\u0446\u0456\u0439. \u0421\u0435\u0440\u0435\u0434 \u043d\u0438\u0445 \u0454 \u0444\u0443\u043d\u043a\u0446\u0456\u043e\u043d\u0430\u043b \u043d\u0430\u0433\u0440\u0430\u0432\u0430\u043d\u043d\u044f \u043c\u0443\u0437\u0438\u0447\u043d\u0438\u0445 \u043a\u043e\u043c\u043f\u043e\u0437\u0438\u0446\u0456\u0439 \u0437\u0430 \u0434\u043e\u043f\u043e\u043c\u043e\u0433\u043e\u044e \u0433\u043e\u043b\u043e\u0441\u0443. \u041a\u0440\u0456\u043c \u0442\u043e\u0433\u043e, \u0432\u0445\u0456\u0434\u043d\u0456 \u0434\u0430\u043d\u0456 \u043c\u043e\u0436\u0443\u0442\u044c \u0431\u0443\u0442\u0438 \u0432\u0438\u043a\u043e\u0440\u0438\u0441\u0442\u0430\u043d\u0456 \u044f\u043a \u0448\u0430\u0431\u043b\u043e\u043d, \u043d\u0430 \u043e\u0441\u043d\u043e\u0432\u0456 \u044f\u043a\u043e\u0433\u043e \u0431\u0443\u0434\u0435 \u0437\u0433\u0435\u043d\u0435\u0440\u043e\u0432\u0430\u043d\u043e \u043f\u043e\u0434\u0456\u0431\u043d\u0443 \u043c\u0443\u0437\u0438\u0447\u043d\u0443 \u043a\u043e\u043c\u043f\u043e\u0437\u0438\u0446\u0456\u044e. \u0420\u0435\u0434\u0430\u0433\u0443\u0432\u0430\u043d\u043d\u044f \u0432\u0456\u0434\u0431\u0443\u0432\u0430\u0454\u0442\u044c\u0441\u044f \u0437\u0430 \u0434\u043e\u043f\u043e\u043c\u043e\u0433\u043e\u044e \u0432\u0438\u043a\u043e\u0440\u0438\u0441\u0442\u0430\u043d\u043d\u044f \u0434\u0432\u043e\u0432\u0438\u043c\u0456\u0440\u043d\u043e\u0457 \u043f\u043b\u043e\u0449\u0438\u043d\u0438, \u0449\u043e \u0432\u0438\u0437\u043d\u0430\u0447\u0430\u0454 \u0437\u0430\u043b\u0435\u0436\u043d\u0456\u0441\u0442\u044c \u0432\u0438\u0441\u043e\u0442\u0438 \u043d\u043e\u0442\u0438 \u0432\u0456\u0434 \u0447\u0430\u0441\u0443 \u0456 \u043c\u0430\u0454 \u0432\u0438\u0433\u043b\u044f\u0434 \u0441\u0435\u043a\u0432\u0435\u043d\u0441\u0435\u0440\u0430. \u0414\u043b\u044f \u0440\u0435\u0430\u043b\u0456\u0437\u0430\u0446\u0456\u0457 \u0430\u0443\u0434\u0456\u043e\u0441\u0438\u043d\u0442\u0435\u0437\u0443 \u043d\u0430 \u043e\u0441\u043d\u043e\u0432\u0456 \u0448\u0430\u0431\u043b\u043e\u043d\u0443 \u0432\u0438\u043a\u043e\u0440\u0438\u0441\u0442\u043e\u0432\u0443\u0454\u0442\u044c\u0441\u044f \u0430\u0440\u0445\u0456\u0442\u0435\u043a\u0442\u0443\u0440\u0430 \u0433\u043b\u0438\u0431\u0438\u043d\u043d\u043e\u0433\u043e \u043d\u0430\u0432\u0447\u0430\u043d\u043d\u044f \u0437 \u0432\u0430\u0440\u0456\u0430\u0446\u0456\u0439\u043d\u0438\u043c \u0430\u0443\u0434\u0456\u043e\u043a\u043e\u0434\u0443\u0432\u0430\u043b\u044c\u043d\u0438\u043a\u043e\u043c, \u0449\u043e \u0434\u043e\u0437\u0432\u043e\u043b\u044f\u0454 \u0442\u0440\u0435\u043d\u0443\u0432\u0430\u0442\u0438 \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u0443 \u043c\u0435\u0440\u0435\u0436\u0443 \u0439 \u0432\u0456\u0434\u0442\u0432\u043e\u0440\u044e\u0432\u0430\u0442\u0438 \u0434\u0430\u043d\u0456, \u0441\u0445\u043e\u0436\u0456 \u0434\u043e \u0432\u0445\u0456\u0434\u043d\u0438\u0445. \u041f\u0456\u0434 \u0447\u0430\u0441 \u0432\u0438\u043a\u043e\u0440\u0438\u0441\u0442\u0430\u043d\u043d\u044f \u0442\u0430\u043a\u043e\u0433\u043e \u043f\u0456\u0434\u0445\u043e\u0434\u0443 \u0432\u0430\u0436\u043b\u0438\u0432\u0438\u043c \u043a\u0440\u043e\u043a\u043e\u043c \u0454 \u043f\u0435\u0440\u0435\u0442\u0432\u043e\u0440\u0435\u043d\u043d\u044f \u0432\u0445\u0456\u0434\u043d\u0438\u0445 \u0434\u0430\u043d\u0438\u0445, \u044f\u043a\u0456 \u043f\u043e\u0434\u0430\u043d\u0456 \u0443 \u0432\u0438\u0433\u043b\u044f\u0434\u0456 \u043d\u0435\u043e\u0431\u0440\u043e\u0431\u043b\u0435\u043d\u043e\u0433\u043e \u0430\u0443\u0434\u0456\u043e\u0444\u043e\u0440\u043c\u0430\u0442\u0443, \u0432 \u0441\u043f\u0435\u043a\u0442\u0440\u043e\u0433\u0440\u0430\u043c\u0443, \u0449\u043e \u0454 \u0437\u043e\u0431\u0440\u0430\u0436\u0435\u043d\u043d\u044f\u043c, \u043d\u0430 \u043e\u0441\u043d\u043e\u0432\u0456 \u044f\u043a\u043e\u0433\u043e \u0432\u0456\u0434\u0431\u0443\u0432\u0430\u0454\u0442\u044c\u0441\u044f \u043f\u0440\u043e\u0446\u0435\u0441 \u0442\u0440\u0435\u043d\u0443\u0432\u0430\u043d\u043d\u044f \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u043e\u0457 \u043c\u0435\u0440\u0435\u0436\u0456 \u0442\u0430 \u0430\u0443\u0434\u0456\u043e\u0433\u0435\u043d\u0435\u0440\u0430\u0446\u0456\u044f. \u0412 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0456 \u043e\u043f\u0435\u0440\u0430\u0446\u0456\u0439 \u0441\u0442\u0432\u043e\u0440\u0435\u043d\u043d\u044f \u043c\u0443\u0437\u0438\u0447\u043d\u0438\u0445 \u043a\u043e\u043c\u043f\u043e\u0437\u0438\u0446\u0456\u0439 \u043a\u043e\u0440\u0438\u0441\u0442\u0443\u0432\u0430\u0447 \u0434\u043e\u0434\u0430\u0442\u043a\u043e\u0432\u043e \u043c\u043e\u0436\u0435 \u0432\u0438\u043a\u043e\u0440\u0438\u0441\u0442\u043e\u0432\u0443\u0432\u0430\u0442\u0438 \u0440\u0435\u0436\u0438\u043c \u0440\u0443\u0447\u043d\u043e\u0433\u043e \u0440\u0435\u0434\u0430\u0433\u0443\u0432\u0430\u043d\u043d\u044f \u0434\u043b\u044f \u043e\u0442\u0440\u0438\u043c\u0430\u043d\u043d\u044f \u0431\u0430\u0436\u0430\u043d\u043e\u0457 \u0432\u0438\u0445\u0456\u0434\u043d\u043e\u0457 \u043c\u0443\u0437\u0438\u0447\u043d\u043e\u0457 \u043a\u043e\u043c\u043f\u043e\u0437\u0438\u0446\u0456\u0457.
\n\u041a\u043b\u044e\u0447\u043e\u0432\u0456 \u0441\u043b\u043e\u0432\u0430:<\/strong> \u0430\u0443\u0434\u0456\u043e\u0441\u0438\u043d\u0442\u0435\u0437, \u0441\u043f\u0435\u043a\u0442\u0440\u043e\u0433\u0440\u0430\u043c\u0430, \u0432\u0430\u0440\u0456\u0430\u0446\u0456\u0439\u043d\u0438\u0439 \u0430\u0443\u0434\u0456\u043e\u043a\u043e\u0434\u0443\u0432\u0430\u043b\u044c\u043d\u0438\u043a.<\/p>\n

\u0420\u043e\u0437\u0448\u0438\u0440\u0435\u043d\u0430 \u0430\u043d\u043e\u0442\u0430\u0446\u0456\u044f \u0430\u043d\u0433\u043b\u0456\u0439\u0441\u044c\u043a\u043e\u044e \u00a0\u043c\u043e\u0432\u043e\u044e<\/strong><\/p>\n

System of audio synthesis and analysis of music compositions is considered. It consists of two primary parts, the audio analysis component, and the music synthesis component. An audio generation component implements various ways of creating audio sequences. One of them is aimed to record melodies played with voice and transform them into sequences played with selected musical instruments. In addition, an audio input created with a human voice can be utilized as a seed, that is used to generate similar music sequences using artificial intelligence. Finally, a manual approach for music generation and editing is available. After automatic mechanisms for composition generation are used, the results of their work are presented on a two-dimensional plane which represents the dependence of music note pitches on time. It is possible to manually adjust the result of audio generation or create new music sequences with this approach. A creation process could be used iteratively to create multiple parallel music sequences that are to be played as a single audio composition. To implement a seed-based audio synthesis, a deep learning architecture based on a variational autoencoder is used to train a neural network that can reproduce input-like data. When using such an approach an additional important step must be considered. All the input data must be converted from a raw audio format to spectrograms which are represented as grayscale images. Moreover, the result of a sound generation is also represented in a spectrogram and therefore, must be converted back to an output audio format that can be played using speakers. This is required as using spectrograms helps to discard redundant data that raw audio format contains and thus significantly reduces resources consumption and increases overall synthesis speed.
\nKeywords:<\/strong> audio synthesis, spectrogram, variational audio encoder.<\/p>\n

\u041b\u0456\u0442\u0435\u0440\u0430\u0442\u0443\u0440\u0430<\/strong><\/p>\n

\u00a0<\/strong>Van Den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A.W. and Kavukcuoglu, K., 2016. WaveNet: A generative model for raw audio. SSW, 125, p.2.<\/span><\/p>\n

    \n
  1. Dhariwal, P., Jun, H., Payne, C., Kim, J.W., Radford, A. and Sutskever, I., 2020. Jukebox: A generative model for music. arXiv preprint arXiv:2005.00341.<\/li>\n
  2. Nistal, Javier, Stefan Lattner, and Gael Richard. “DrumGAN: Synthesis of drum sounds with timbral feature conditioning using Generative Adversarial Networks.” arXiv preprint arXiv:2008.12073 (2020).<\/li>\n
  3. Viktoriia V. Voitko, Svitlana V. Bevz, Sergii M. Burbelo, Pavlo V. Stavytskyi, Bogdan Pinaiev, Zbigniew Omiotek, Doszhon Baitussupov, and Aigul Bazarbayeva “Automated system of audio components analysis and synthesis”, Proc. SPIE 11045, Optical Fibers and Their Applications 2018, 110450V (15 March 2019); https:\/\/doi.org\/10.1117\/12.2522313<\/li>\n
  4. Viktoriia V. Voitko, Svitlana V. Bevz, Sergii M. Burbelo, Pavlo V. Stavytskyi, Oleksandr M. Khoshaba, Natalia O. Rysynets, Olena Yu. Teplova, Andrzej Smolarz, Saule Smailova, Assel Mussabekova, and Bakhyt Yeraliyeva “Analysis of the development approaches of the system of audio synthesis and recognition with the option of using photonic processors”, Proc. SPIE 12040, Photonics Applications in Astronomy, Communications, Industry, and High Energy Physics Experiments 2021, 120400N (3 November 2021); https:\/\/doi.org\/10.1117\/12.2611464<\/li>\n
  5. Foster, David. Generative deep learning: teaching machines to paint, write, compose, and play. Sebastopol, CA: O’Reilly Media, 2019. Print.<\/li>\n<\/ol>\n

     <\/p>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[62],"tags":[],"_links":{"self":[{"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/posts\/12158"}],"collection":[{"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=12158"}],"version-history":[{"count":1,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/posts\/12158\/revisions"}],"predecessor-version":[{"id":12160,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/posts\/12158\/revisions\/12160"}],"wp:attachment":[{"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=12158"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=12158"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=12158"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}