學術產出-Theses
Article View/Open
Publication Export
-
題名 以感知損失神經網路平滑化樂高平板磚之影像樂高風格化技術
2D Lego Flat Tiles Generation with Perceptual Loss Neural Network作者 賴雅鈴
Lai, Ya-Ling貢獻者 紀明德
Chi, Ming-Te
賴雅鈴
Lai, Ya-Ling關鍵詞 樂高
編碼器-解碼器網路架構
拼貼
LEGO
Encoder-decoder network architecture
Collage日期 2023 上傳時間 9-Mar-2023 18:36:50 (UTC+8) 摘要 樂高公司持續推出新的系列及不同種類的磚，這樣的多樣性也使得樂高深受大人小孩的喜愛，對於一些新推出的樂高系列，相對較少有研究進行探討，但也會有與其他領域，像是拼貼、排列、像素化問題等或與樂高研究議題相關的地方。像素化藝術方面的研究一直以來都非常受歡迎，但這樣的風格就是每個區域都為方形，對於一些圖形較圓滑的地方無法很好地表現出來。本研究的樂高豆豆系列則是將畫素以實體元件表現出來，而除了方形的磚，也有一些帶有較圓滑邊緣的磚。但當我們要以人工的方式去拼一個形狀時，在磚形狀及顏色的選擇上，就會花費非常多的時間，如果要拼的東西越大，所花費的時間也就更久，會浪費許多勞力和時間。為了解決這些問題，本研究首先嘗試將編碼器-解碼器架構的網路與二維樂高平板磚建構相結合。以樂高平板磚中的方形磚及帶圓滑邊緣的磚轉換為圖片作為輸入，並透過給定損失函數，將現有的照片馬賽克神經網路研究延伸至樂高平板磚的組合問題。同時，我們也針對輸入圖形及生成的圖片，做一系列的比較及分析，證明此系統的有效性。
LEGO continues to release new series and different types of bricks, which has made it popular with both adults and children. However, there has been relatively little research on some of the newer Lego series, but there are also related studies in other areas such as puzzles, arrangements, pixelation, and other Lego research topics. Pixel art research has always been very popular, but this style is characterized by square regions, which makes it difficult to represent smoother shapes. The Lego Dots series studied in this research is just like expressing pixels as physical components, and in addition to square bricks, there are also bricks with smoother edges. However, when we try to manually assemble a shape with bricks, it takes a lot of time to choose the shape and color of the bricks, and the larger the thing we want to assemble, the longer it takes, wasting a lot of labor and time.In order to solve these problems, this research first attempts to combine the network of encoder-decoder architecture with the construction of two-dimensional Lego flat bricks. Taking square bricks and bricks with rounded edges in Lego flat bricks as input, and through a given loss function, the existing photomosaic neural network research is extended from the collage problem to the combination problem of Lego flat bricks. At the same time, we also made a series of comparisons and analyzes on the input images and the generated images to show the effectiveness of this system.參考文獻 [ 1 ] Di Blasi, G., Gallo, G., & Petralia, M. (2005, September). Puzzle image mosaic. In Proc.IASTED/VIIP (pp. 33-37).[ 2 ] Zou, C., Cao, J., Ranaweera, W., Alhashim, I., Tan, P., Sheffer, A., & Zhang, H. (2016). Legible compact calligrams. ACM Transactions on Graphics (TOG), 35(4), 1-12.[ 3 ] Kwan, K. C., Sinn, L. T., Han, C., Wong, T. T., & Fu, C. W. (2016). Pyramid of arclength descriptor for generating collage of shapes. ACM Trans. Graph., 35(6), 229-1.[ 4 ] Chen, M., Xu, F., & Lu, L. (2019). Manufacturable pattern collage along a boundary. Computational Visual Media, 5, 293-302.[ 5 ] Akiyama, O. (2017). ASCII art synthesis with convolutional networks. In Proc. NIPS Workshop Mach. Learn. Creativity Design (pp. 1-7).[ 6 ] Tesfaldet, M., Saftarli, N., Brubaker, M. A., & Derpanis, K. G. (2018). Convolutional Photomosaic Generation via Multi-Scale Perceptual Losses. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops (pp. 0-0).[ 7 ] Kim, J., & Pellacini, F. (2002). Jigsaw image mosaics. ACM Transactions on Graphics, 21(3), 657-664.[ 8 ] Xu, P., Ding, J., Zhang, H., & Huang, H. (2019). Discernible image mosaic with edge-aware adaptive tiles. Computational Visual Media, 5, 45-58.[ 9 ] Gerstner, T., DeCarlo, D., Alexa, M., Finkelstein, A., Gingold, Y., & Nealen, A. (2012, June). Pixelated image abstraction. In Proceedings of the Symposium on Non-Photorealistic Animation and Rendering (pp. 29-36).[ 10 ] Inglis, T., & Kaplan, C. S. (2012). Pixelating vector line art. SIGGRAPH Posters, 108.[ 11 ] Shang, Y., & Wong, H. C. (2021). Automatic portrait image pixelization. Computers & Graphics, 95, 47-59.[ 12 ] Huang, M. R., & Lee, R. R. (2015). Pixel Art Color Palette Synthesis. In Information Science and Applications (pp. 327-334). Springer Berlin Heidelberg.[ 13 ] Orchard, J., & Kaplan, C. S. (2008, June). Cut-out image mosaics. In Proceedings of the 6th international symposium on Non-photorealistic animation and rendering (pp. 79-87).[ 14 ] Shen, I. C., & Chen, B. Y. (2021). Clipgen: A deep generative model for clipart vectorization and synthesis. IEEE Transactions on Visualization and Computer Graphics, 28(12), 4211-4224.[ 15 ] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.[ 16 ] Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600-612.[ 17 ] Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018). The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 586-595).[ 18 ] Johnson, J., Alahi, A., & Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14 (pp. 694-711). Springer International Publishing.[ 19 ] Sacht, L. (2022). Structure-aware bottle cap art. Computers & Graphics, 107, 277-288.[ 20 ] Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Süsstrunk, S. (2010). Slic superpixels (No. REP_WORK).[ 21 ] Han, C., Wen, Q., He, S., Zhu, Q., Tan, Y., Han, G., & Wong, T. T. (2018). Deep unsupervised pixelization. ACM Transactions on Graphics (TOG), 37(6), 1-11.[ 22 ] Doyle, L., Anderson, F., Choy, E., & Mould, D. (2019). Automated pebble mosaic stylization of images. Computational Visual Media, 5, 33-44.[ 23 ] Hsiang-Yu Wang, Ming-Te Chi, Mapping 2D Lego Construction into Tiling Problem with Graph Neural Network (2022). 描述 碩士
國立政治大學
資訊科學系
109753103資料來源 http://thesis.lib.nccu.edu.tw/record/#G0109753103 資料類型 thesis dc.contributor.advisor 紀明德 zh_TW dc.contributor.advisor Chi, Ming-Te en_US dc.contributor.author (Authors) 賴雅鈴 zh_TW dc.contributor.author (Authors) Lai, Ya-Ling en_US dc.creator (作者) 賴雅鈴 zh_TW dc.creator (作者) Lai, Ya-Ling en_US dc.date (日期) 2023 en_US dc.date.accessioned 9-Mar-2023 18:36:50 (UTC+8) - dc.date.available 9-Mar-2023 18:36:50 (UTC+8) - dc.date.issued (上傳時間) 9-Mar-2023 18:36:50 (UTC+8) - dc.identifier (Other Identifiers) G0109753103 en_US dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/143833 - dc.description (描述) 碩士 zh_TW dc.description (描述) 國立政治大學 zh_TW dc.description (描述) 資訊科學系 zh_TW dc.description (描述) 109753103 zh_TW dc.description.abstract (摘要) 樂高公司持續推出新的系列及不同種類的磚，這樣的多樣性也使得樂高深受大人小孩的喜愛，對於一些新推出的樂高系列，相對較少有研究進行探討，但也會有與其他領域，像是拼貼、排列、像素化問題等或與樂高研究議題相關的地方。像素化藝術方面的研究一直以來都非常受歡迎，但這樣的風格就是每個區域都為方形，對於一些圖形較圓滑的地方無法很好地表現出來。本研究的樂高豆豆系列則是將畫素以實體元件表現出來，而除了方形的磚，也有一些帶有較圓滑邊緣的磚。但當我們要以人工的方式去拼一個形狀時，在磚形狀及顏色的選擇上，就會花費非常多的時間，如果要拼的東西越大，所花費的時間也就更久，會浪費許多勞力和時間。為了解決這些問題，本研究首先嘗試將編碼器-解碼器架構的網路與二維樂高平板磚建構相結合。以樂高平板磚中的方形磚及帶圓滑邊緣的磚轉換為圖片作為輸入，並透過給定損失函數，將現有的照片馬賽克神經網路研究延伸至樂高平板磚的組合問題。同時，我們也針對輸入圖形及生成的圖片，做一系列的比較及分析，證明此系統的有效性。 zh_TW dc.description.abstract (摘要) LEGO continues to release new series and different types of bricks, which has made it popular with both adults and children. However, there has been relatively little research on some of the newer Lego series, but there are also related studies in other areas such as puzzles, arrangements, pixelation, and other Lego research topics. Pixel art research has always been very popular, but this style is characterized by square regions, which makes it difficult to represent smoother shapes. The Lego Dots series studied in this research is just like expressing pixels as physical components, and in addition to square bricks, there are also bricks with smoother edges. However, when we try to manually assemble a shape with bricks, it takes a lot of time to choose the shape and color of the bricks, and the larger the thing we want to assemble, the longer it takes, wasting a lot of labor and time.In order to solve these problems, this research first attempts to combine the network of encoder-decoder architecture with the construction of two-dimensional Lego flat bricks. Taking square bricks and bricks with rounded edges in Lego flat bricks as input, and through a given loss function, the existing photomosaic neural network research is extended from the collage problem to the combination problem of Lego flat bricks. At the same time, we also made a series of comparisons and analyzes on the input images and the generated images to show the effectiveness of this system. en_US dc.description.tableofcontents 第一章 緒論 11.1 研究動機 11.2 問題描述 31.3 論文貢獻 31.4 論文章節架構 4第二章 相關研究 52.1 圖片馬賽克問題 52.2 拼貼問題 62.3 像素化藝術 8第三章 研究方法 103.1 系統流程 103.2 圖片資料集 113.3 資料預處理 123.4 編碼器與解碼器架構 143.5 選擇圖塊 153.6 感知損失 153.7 顏色處理 173.8 量化分析 18第四章 結果與限制 254.1 研究細節 254.2 實驗數據比較 254.2.1 與Baseline方法比較 254.2.2 輸入圖塊集差異 264.2.3 組裝結果大小 284.2.4 選用尺度不同 294.2.5 損失函數使用不同 314.3 案例分析 324.4 結果 334.4 環境 404.5 限制 40第五章 結論與未來展望 425.1 結論 425.2 未來展望 43參考文獻 44附錄 46 zh_TW dc.format.extent 6155083 bytes - dc.format.mimetype application/pdf - dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0109753103 en_US dc.subject (關鍵詞) 樂高 zh_TW dc.subject (關鍵詞) 編碼器-解碼器網路架構 zh_TW dc.subject (關鍵詞) 拼貼 zh_TW dc.subject (關鍵詞) LEGO en_US dc.subject (關鍵詞) Encoder-decoder network architecture en_US dc.subject (關鍵詞) Collage en_US dc.title (題名) 以感知損失神經網路平滑化樂高平板磚之影像樂高風格化技術 zh_TW dc.title (題名) 2D Lego Flat Tiles Generation with Perceptual Loss Neural Network en_US dc.type (資料類型) thesis en_US dc.relation.reference (參考文獻) [ 1 ] Di Blasi, G., Gallo, G., & Petralia, M. (2005, September). Puzzle image mosaic. In Proc.IASTED/VIIP (pp. 33-37).[ 2 ] Zou, C., Cao, J., Ranaweera, W., Alhashim, I., Tan, P., Sheffer, A., & Zhang, H. (2016). Legible compact calligrams. ACM Transactions on Graphics (TOG), 35(4), 1-12.[ 3 ] Kwan, K. C., Sinn, L. T., Han, C., Wong, T. T., & Fu, C. W. (2016). Pyramid of arclength descriptor for generating collage of shapes. ACM Trans. Graph., 35(6), 229-1.[ 4 ] Chen, M., Xu, F., & Lu, L. (2019). Manufacturable pattern collage along a boundary. Computational Visual Media, 5, 293-302.[ 5 ] Akiyama, O. (2017). ASCII art synthesis with convolutional networks. In Proc. NIPS Workshop Mach. Learn. Creativity Design (pp. 1-7).[ 6 ] Tesfaldet, M., Saftarli, N., Brubaker, M. A., & Derpanis, K. G. (2018). Convolutional Photomosaic Generation via Multi-Scale Perceptual Losses. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops (pp. 0-0).[ 7 ] Kim, J., & Pellacini, F. (2002). Jigsaw image mosaics. ACM Transactions on Graphics, 21(3), 657-664.[ 8 ] Xu, P., Ding, J., Zhang, H., & Huang, H. (2019). Discernible image mosaic with edge-aware adaptive tiles. Computational Visual Media, 5, 45-58.[ 9 ] Gerstner, T., DeCarlo, D., Alexa, M., Finkelstein, A., Gingold, Y., & Nealen, A. (2012, June). Pixelated image abstraction. In Proceedings of the Symposium on Non-Photorealistic Animation and Rendering (pp. 29-36).[ 10 ] Inglis, T., & Kaplan, C. S. (2012). Pixelating vector line art. SIGGRAPH Posters, 108.[ 11 ] Shang, Y., & Wong, H. C. (2021). Automatic portrait image pixelization. Computers & Graphics, 95, 47-59.[ 12 ] Huang, M. R., & Lee, R. R. (2015). Pixel Art Color Palette Synthesis. In Information Science and Applications (pp. 327-334). Springer Berlin Heidelberg.[ 13 ] Orchard, J., & Kaplan, C. S. (2008, June). Cut-out image mosaics. In Proceedings of the 6th international symposium on Non-photorealistic animation and rendering (pp. 79-87).[ 14 ] Shen, I. C., & Chen, B. Y. (2021). Clipgen: A deep generative model for clipart vectorization and synthesis. IEEE Transactions on Visualization and Computer Graphics, 28(12), 4211-4224.[ 15 ] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.[ 16 ] Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600-612.[ 17 ] Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018). The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 586-595).[ 18 ] Johnson, J., Alahi, A., & Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14 (pp. 694-711). Springer International Publishing.[ 19 ] Sacht, L. (2022). Structure-aware bottle cap art. Computers & Graphics, 107, 277-288.[ 20 ] Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Süsstrunk, S. (2010). Slic superpixels (No. REP_WORK).[ 21 ] Han, C., Wen, Q., He, S., Zhu, Q., Tan, Y., Han, G., & Wong, T. T. (2018). Deep unsupervised pixelization. ACM Transactions on Graphics (TOG), 37(6), 1-11.[ 22 ] Doyle, L., Anderson, F., Choy, E., & Mould, D. (2019). Automated pebble mosaic stylization of images. Computational Visual Media, 5, 33-44.[ 23 ] Hsiang-Yu Wang, Ming-Te Chi, Mapping 2D Lego Construction into Tiling Problem with Graph Neural Network (2022). zh_TW
