Publications-Theses
Article View/Open
Publication Export
-
題名 基於 GRU 生成音樂
The Application of GRU to Generate Music作者 賴晨心
Lai, Chen-Hsin貢獻者 蔡瑞煌<br>韓志翔
Tsaih, Rua-Huan<br>Han, Tzu-Shian
賴晨心
Lai, Chen-Hsin關鍵詞 人工智慧
音樂生成
Neural networks
GRU
Music generation日期 2021 上傳時間 2-Sep-2021 15:59:35 (UTC+8) 摘要 儘管從直覺上講，透過適當安排好的輸入/輸出變數表示和 GRU 模型 （Cho 等人，2014a ）的超參數能夠學習巴赫和斯卡拉蒂鋼琴音樂來生成音樂，但文獻上仍沒有這樣的學術和實際試驗。本研究接受此挑戰，並設計相關實驗來了解是否能利用 GRU 模型學習巴赫和斯卡拉蒂鋼琴音樂後生成音樂。本研究採用基於節拍的事件數據表示（Huang 和 Yang，2020），並使用不同的輸入/輸出變數表示進行實驗。而在推理階段，我們將 GRU 模型的每個輸出序列連接起來並轉換回 MIDI 文件，從而生成音樂。實驗結果顯示了應用 GRU 生成音樂的正面性。
Although, intuitively, a proper arrangement of input/output presentation and hyperparameters of the state-of-the-art GRU model (Cho et al., 2014a) is capable of learning Bach and Scarlatti piano music and generating the music, there are no such academic and practical trials. This study addresses the challenge of generating music from learning Bach and Scarlatti piano music through the GRU model. This study employs the beat-based event data representation (Huang and Yang, 2020) and conducts an experiment with different input/output representations. As to the inferencing stage, each output sequence of the GRU model is concatenated and transformed back into a MIDI file, so that the music is generated. The experiment shows a positive result regarding the application of GRU to generate music.參考文獻 Böck, Sebastian, Florian Krebs, and Gerhard Widmer. (2016). Joint beat and downbeat tracking with recurrent neural networks. Procedure of the 17th Int. Society for Music Information Retrieval Conference (ISMIR), 2016, pp. 255–261. https://archives.ismir.net/ismir2016/paper/000186.pdf.Briot, Jean-Pierre, Gaëtan Hadjeres, and François-David Pachet. (2019). Deep Learning Techniques for Music Generation – A Survey. arXiv:1709.01620 [cs.SD]. https://arxiv.org/abs/1709.01620.Cho, Kyunghyun, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. (2014a). Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. arXiv:1406.1078 [cs.CL]. https://arxiv.org/abs/1406.1078.Cho, Kyunghyun, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. (2014b). On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. arXiv:1409.1259 [cs.CL]. https://arxiv.org/abs/1409.1259.Chollet, François. (2015). Keras. https://github.com/ fchollet/keras.Chung, Junyoung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. (2014). Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv:1412.3555 [cs.NE]. https://arxiv.org/abs/1412.3555.Conklin, Darrell. (2003). Music generation from statistical models. Journal of New Music Research, vol. 45, 2016, issue 2, pp. 160 - 183. https://www.tandfonline.com/doi/abs/10.1080/09298215.2016.1173708?journalCode=nnmr20.Cooper, Grosvenor W, Grosvenor Cooper, and Leonard B Meyer. (1963). The Rhythmic Structure of Music. University of Chicago Press. .Cornelisa, Olmo, Micheline Lesaffreb, Dirk Moelantsb, and Marc Leman. (2010). Access to Ethnic Music: Advances and Perspectives in Content-based Music Information Retrieval. Signal Processing, vol. 90, no. 4, pp. 1008–1031. https://www.sciencedirect.com/science/article/abs/pii/S0165168409002874?casa_token=si0RB_X80SQAAAAA:poo0HRgdCMLFcX9LrLen9k1zeT6kmDIdHOd_D13iFicSfKbMTUZnQ23ZrJoIM48CQjXaHfJOtMVx.Dozat, Timothy. (2016). Incorporating Nesterov Momentum into Adam. ICLR Workshop, (1):2013–2016. https://openreview.net/forum?id=OM0jvwB8jIp57ZJjtNEZ.Hadjeres, Gaëtan and François Pachet. (2016). DeepBach: a Steerable Model for Bach chorales generation. arXiv preprint arXiv:1612.01010 (2016). https://arxiv.org/abs/1612.01010.Hassan, Hasminda, Zunairah Haji Murat, Valerie Ross, and Norlida Buniyamin. (2012). A preliminary study on the effects of music on human brainwaves. International Conference on Control, Automation and Information Sciences (ICCAIS), Saigon, Vietnam, 2012, pp. 176-180. https://ieeexplore.ieee.org/document/6466581/citations#citations.Hawthorne, Curtis, Anna Huang, Daphne Ippolito, and Douglas Eck. (2018). Transformer-NADE for Piano Performances. NIPS 2nd Workshop on Machine Learning for Creativity and Design. https://www.semanticscholar.org/paper/Transformer-NADE-for-Piano-Performances-Hawthorne/96dd86198c5be28fe91b67f8d49996033fa58bf6.Hochreiter, Sepp and Jürgen Schmidhuber. (1997). Long Short-Term Memory. Neural Computation Vol. 9, Issue 8, November 15, 1997, pp.1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735.Hsiao, Wen-Yi. (2019). miditoolkit. https://libraries.io/pypi/miditoolkit.Huang, Yu-Siang and Yi-Hsuan Yang. (2020). Pop Music Transformer- Beat-based Modeling and Generation of Expressive Pop Piano Compositions. arXiv:2002.00212 [cs.SD]. https://arxiv.org/abs/2002.00212.Kingma, Diederik P and Jimmy Ba. (2017). ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION. arXiv:1412.6980 [cs.LG]. https://arxiv.org/abs/1412.6980.Kingma, Diederik P and Jimmy Lei Ba. (2015). Adam: a Method for Stochastic Optimization. International Conference on Learning Representations, pages 1–13. https://arxiv.org/abs/1412.6980.Kong, Qiuqiang, Bochen Li, Jitong Chen, and Yuxuan Wang. (2020). GiantMIDI-Piano: A large-scale MIDI dataset for classical piano music. arXiv:2010.07061 [cs.IR]. https://arxiv.org/pdf/2010.07061.pdf.Kong, Qiuqiang, Yin Cao, Turab Iqbal, Yong Xu, Wenwu Wang, and Mark D. Plumbley. (2019). Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems. arXiv:1904.03476 [cs.SD]. https://arxiv.org/abs/1904.03476.Kong, Qiuqiang, Bochen Li, Xuchen Song, Yuan Wan, and Yuxuan Wang. (2020). High-resolution Piano Transcription with Pedals by Regressing Onsets and Offsets Times. arXiv:2010.01815 [cs.SD]. https://arxiv.org/abs/2010.01815.Laitz, Steven. (2008). The Complete Musician (2 ed.). New York: Oxford University Press, Inc. pp. 96.MIDI Manufacturers Association (MMA). (14/04/2017). MIDI Specifications. https://www.midi.org/specifications.Mikolov, Tomas, Armand Joulin, Sumit Chopra, Michael Mathieu, and Marc`Aurelio Ranzato. (2015). Learning Longer Memory in Recurrent Neural Networks. arXiv:1412.7753 [cs.NE]. https://arxiv.org/abs/1412.7753.Nesterov, Yurii. (1983). A method for unconstrained convex minimization problem with the rate of convergence o(1/k2). Doklady ANSSSR (translated as Soviet.Math.Docl.), 269:543–547.. https://www.semanticscholar.org/paper/A-method-for-unconstrained-convex-minimization-with-Nesterov/ed910d96802212c9e45d956adaa27d915f5d7469.Olah, Christopher. (August 27, 2015). Understanding LSTM Networks. https://colah.github.io/posts/2015-08-Understanding-LSTMs/.Oore, Sageev, Ian Simon, Sander Dieleman, Douglas Eck, and Karen Simonyan. (2018). This Time with Feeling: Learning Expressive Musical Performance. arXiv preprint arXiv:1808.03715. https://arxiv.org/abs/1808.03715.Qian, Ning. (1999). On the momentum term in gradient descent learning algorithms. Neural networks : the official journal of the International Neural Network Society, 12(1):145–151. https://www.sciencedirect.com/science/article/abs/pii/S0893608098001166.Turley, L.W and Ronald E Milliman. (2000). Atmospheric effects on shopping behaviour: a review of the experimental evidence. Journal of Business Research, Vol. 49, No. 2, pp. 193-211. https://www.sciencedirect.com/science/article/abs/pii/S0148296399000107.Tze, Peter, and Ming Chou. (2010). Attention drainage effect: How background music effects concentration in Taiwanese college students. Journal of the Scholarship of Teaching and Learning, Vol. 10, No. 1, January 2010, pp. 36 – 46. https://eric.ed.gov/?id=EJ882124. 描述 碩士
國立政治大學
資訊管理學系
108356034資料來源 http://thesis.lib.nccu.edu.tw/record/#G0108356034 資料類型 thesis dc.contributor.advisor 蔡瑞煌<br>韓志翔 zh_TW dc.contributor.advisor Tsaih, Rua-Huan<br>Han, Tzu-Shian en_US dc.contributor.author (Authors) 賴晨心 zh_TW dc.contributor.author (Authors) Lai, Chen-Hsin en_US dc.creator (作者) 賴晨心 zh_TW dc.creator (作者) Lai, Chen-Hsin en_US dc.date (日期) 2021 en_US dc.date.accessioned 2-Sep-2021 15:59:35 (UTC+8) - dc.date.available 2-Sep-2021 15:59:35 (UTC+8) - dc.date.issued (上傳時間) 2-Sep-2021 15:59:35 (UTC+8) - dc.identifier (Other Identifiers) G0108356034 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/136853 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊管理學系 zh_TW dc.description (描述) 108356034 zh_TW dc.description.abstract (摘要) 儘管從直覺上講，透過適當安排好的輸入/輸出變數表示和 GRU 模型 （Cho 等人，2014a ）的超參數能夠學習巴赫和斯卡拉蒂鋼琴音樂來生成音樂，但文獻上仍沒有這樣的學術和實際試驗。本研究接受此挑戰，並設計相關實驗來了解是否能利用 GRU 模型學習巴赫和斯卡拉蒂鋼琴音樂後生成音樂。本研究採用基於節拍的事件數據表示（Huang 和 Yang，2020），並使用不同的輸入/輸出變數表示進行實驗。而在推理階段，我們將 GRU 模型的每個輸出序列連接起來並轉換回 MIDI 文件，從而生成音樂。實驗結果顯示了應用 GRU 生成音樂的正面性。 zh_TW dc.description.abstract (摘要) Although, intuitively, a proper arrangement of input/output presentation and hyperparameters of the state-of-the-art GRU model (Cho et al., 2014a) is capable of learning Bach and Scarlatti piano music and generating the music, there are no such academic and practical trials. This study addresses the challenge of generating music from learning Bach and Scarlatti piano music through the GRU model. This study employs the beat-based event data representation (Huang and Yang, 2020) and conducts an experiment with different input/output representations. As to the inferencing stage, each output sequence of the GRU model is concatenated and transformed back into a MIDI file, so that the music is generated. The experiment shows a positive result regarding the application of GRU to generate music. en_US dc.description.tableofcontents 1.INTRODUCTION 12.LITERATURE REVIEW 52.1. MUSIC 52.2. MUSIC DATA REPRESENTATION 62.3. MIDI-LIKE EVENT-BASED DATA REPRESENTATION 92.3.1. Performance Event 102.3.2. NoteTuple Event 102.3.3. Beat-Based Event 112.4. RNN, LSTM, GRU 122.4.1. RNN 122.4.2. LSTM 132.4.3. GRU 142.5. OPTIMIZER 152.5.1. Momentum 152.5.2. NAG 152.5.3. Adam 162.5.4. NAdam 163.EXPERIMENT DESIGN 173.1. FRAMEWORK 173.2. VARIABLE DESCRIPTIONS 203.3. DATA 263.4. MODEL IMPLEMENTATION 283.5. MODEL TRAINING AND TESTING 314.EVALUATION 374.1. SUBJECTIVE EVALUATION 374.2. OBJECTIVE EVALUATION 424.2.1. Comparison Event 424.2.2. Comparison Event 434.2.3. Comparison Event 434.2.4. Comparison Event 445.DISCUSSION & CONCLUSION 455.1. CONCLUSION 455.2. LIMITATION AND FUTURE WORK 45REFERENCE 47APPENDIX 50 zh_TW dc.format.extent 10672699 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0108356034 en_US dc.subject (關鍵詞) 人工智慧 zh_TW dc.subject (關鍵詞) 音樂生成 zh_TW dc.subject (關鍵詞) Neural networks en_US dc.subject (關鍵詞) GRU en_US dc.subject (關鍵詞) Music generation en_US dc.title (題名) 基於 GRU 生成音樂 zh_TW dc.title (題名) The Application of GRU to Generate Music en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) Böck, Sebastian, Florian Krebs, and Gerhard Widmer. (2016). Joint beat and downbeat tracking with recurrent neural networks. Procedure of the 17th Int. Society for Music Information Retrieval Conference (ISMIR), 2016, pp. 255–261. https://archives.ismir.net/ismir2016/paper/000186.pdf.Briot, Jean-Pierre, Gaëtan Hadjeres, and François-David Pachet. (2019). Deep Learning Techniques for Music Generation – A Survey. arXiv:1709.01620 [cs.SD]. https://arxiv.org/abs/1709.01620.Cho, Kyunghyun, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. (2014a). Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. arXiv:1406.1078 [cs.CL]. https://arxiv.org/abs/1406.1078.Cho, Kyunghyun, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. (2014b). On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. arXiv:1409.1259 [cs.CL]. https://arxiv.org/abs/1409.1259.Chollet, François. (2015). Keras. https://github.com/ fchollet/keras.Chung, Junyoung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. (2014). Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv:1412.3555 [cs.NE]. https://arxiv.org/abs/1412.3555.Conklin, Darrell. (2003). Music generation from statistical models. Journal of New Music Research, vol. 45, 2016, issue 2, pp. 160 - 183. https://www.tandfonline.com/doi/abs/10.1080/09298215.2016.1173708?journalCode=nnmr20.Cooper, Grosvenor W, Grosvenor Cooper, and Leonard B Meyer. (1963). The Rhythmic Structure of Music. University of Chicago Press. .Cornelisa, Olmo, Micheline Lesaffreb, Dirk Moelantsb, and Marc Leman. (2010). Access to Ethnic Music: Advances and Perspectives in Content-based Music Information Retrieval. Signal Processing, vol. 90, no. 4, pp. 1008–1031. https://www.sciencedirect.com/science/article/abs/pii/S0165168409002874?casa_token=si0RB_X80SQAAAAA:poo0HRgdCMLFcX9LrLen9k1zeT6kmDIdHOd_D13iFicSfKbMTUZnQ23ZrJoIM48CQjXaHfJOtMVx.Dozat, Timothy. (2016). Incorporating Nesterov Momentum into Adam. ICLR Workshop, (1):2013–2016. https://openreview.net/forum?id=OM0jvwB8jIp57ZJjtNEZ.Hadjeres, Gaëtan and François Pachet. (2016). DeepBach: a Steerable Model for Bach chorales generation. arXiv preprint arXiv:1612.01010 (2016). https://arxiv.org/abs/1612.01010.Hassan, Hasminda, Zunairah Haji Murat, Valerie Ross, and Norlida Buniyamin. (2012). A preliminary study on the effects of music on human brainwaves. International Conference on Control, Automation and Information Sciences (ICCAIS), Saigon, Vietnam, 2012, pp. 176-180. https://ieeexplore.ieee.org/document/6466581/citations#citations.Hawthorne, Curtis, Anna Huang, Daphne Ippolito, and Douglas Eck. (2018). Transformer-NADE for Piano Performances. NIPS 2nd Workshop on Machine Learning for Creativity and Design. https://www.semanticscholar.org/paper/Transformer-NADE-for-Piano-Performances-Hawthorne/96dd86198c5be28fe91b67f8d49996033fa58bf6.Hochreiter, Sepp and Jürgen Schmidhuber. (1997). Long Short-Term Memory. Neural Computation Vol. 9, Issue 8, November 15, 1997, pp.1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735.Hsiao, Wen-Yi. (2019). miditoolkit. https://libraries.io/pypi/miditoolkit.Huang, Yu-Siang and Yi-Hsuan Yang. (2020). Pop Music Transformer- Beat-based Modeling and Generation of Expressive Pop Piano Compositions. arXiv:2002.00212 [cs.SD]. https://arxiv.org/abs/2002.00212.Kingma, Diederik P and Jimmy Ba. (2017). ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION. arXiv:1412.6980 [cs.LG]. https://arxiv.org/abs/1412.6980.Kingma, Diederik P and Jimmy Lei Ba. (2015). Adam: a Method for Stochastic Optimization. International Conference on Learning Representations, pages 1–13. https://arxiv.org/abs/1412.6980.Kong, Qiuqiang, Bochen Li, Jitong Chen, and Yuxuan Wang. (2020). GiantMIDI-Piano: A large-scale MIDI dataset for classical piano music. arXiv:2010.07061 [cs.IR]. https://arxiv.org/pdf/2010.07061.pdf.Kong, Qiuqiang, Yin Cao, Turab Iqbal, Yong Xu, Wenwu Wang, and Mark D. Plumbley. (2019). Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems. arXiv:1904.03476 [cs.SD]. https://arxiv.org/abs/1904.03476.Kong, Qiuqiang, Bochen Li, Xuchen Song, Yuan Wan, and Yuxuan Wang. (2020). High-resolution Piano Transcription with Pedals by Regressing Onsets and Offsets Times. arXiv:2010.01815 [cs.SD]. https://arxiv.org/abs/2010.01815.Laitz, Steven. (2008). The Complete Musician (2 ed.). New York: Oxford University Press, Inc. pp. 96.MIDI Manufacturers Association (MMA). (14/04/2017). MIDI Specifications. https://www.midi.org/specifications.Mikolov, Tomas, Armand Joulin, Sumit Chopra, Michael Mathieu, and Marc`Aurelio Ranzato. (2015). Learning Longer Memory in Recurrent Neural Networks. arXiv:1412.7753 [cs.NE]. https://arxiv.org/abs/1412.7753.Nesterov, Yurii. (1983). A method for unconstrained convex minimization problem with the rate of convergence o(1/k2). Doklady ANSSSR (translated as Soviet.Math.Docl.), 269:543–547.. https://www.semanticscholar.org/paper/A-method-for-unconstrained-convex-minimization-with-Nesterov/ed910d96802212c9e45d956adaa27d915f5d7469.Olah, Christopher. (August 27, 2015). Understanding LSTM Networks. https://colah.github.io/posts/2015-08-Understanding-LSTMs/.Oore, Sageev, Ian Simon, Sander Dieleman, Douglas Eck, and Karen Simonyan. (2018). This Time with Feeling: Learning Expressive Musical Performance. arXiv preprint arXiv:1808.03715. https://arxiv.org/abs/1808.03715.Qian, Ning. (1999). On the momentum term in gradient descent learning algorithms. Neural networks : the official journal of the International Neural Network Society, 12(1):145–151. https://www.sciencedirect.com/science/article/abs/pii/S0893608098001166.Turley, L.W and Ronald E Milliman. (2000). Atmospheric effects on shopping behaviour: a review of the experimental evidence. Journal of Business Research, Vol. 49, No. 2, pp. 193-211. https://www.sciencedirect.com/science/article/abs/pii/S0148296399000107.Tze, Peter, and Ming Chou. (2010). Attention drainage effect: How background music effects concentration in Taiwanese college students. Journal of the Scholarship of Teaching and Learning, Vol. 10, No. 1, January 2010, pp. 36 – 46. https://eric.ed.gov/?id=EJ882124. zh_TW dc.identifier.doi (DOI) 10.6814/NCCU202101416 en_US
