1. NCCU Academic Hub
  2. Publications
  3. College of Commerce
  4. Theses
  5. Item

Publications-Theses

Article View/Open

Publication Export

Google ScholarTM

NCCU Library

Citation Infomation

Related Publications in TAIR

題名 推理類神經網路及其應用
The Reasoning Neural Network and It`s Applications
作者 徐志鈞
Chun, Hsu Chih
貢獻者 蔡瑞煌
Tsaih
徐志鈞
Hsu Chih Chun
關鍵詞 推理類神經網路
軟性學習程序
線性分割條件
不相關節點
推理機能
The Reasoning Neural Network
the softening learning algorithm
linearly separating condition
日期 1994
上傳時間 29-Apr-2016 16:30:45 (UTC+8)
摘要 大部的類神經網路均為解決特定問題而設計，並非真正去模擬人腦的功能
Most of artification Neural Networks are designed to resolve
參考文獻 Bibliography\r\n中文部份\r\n[葉怡成93]葉怡成，類神經網路模式應用與實作，儒林圖書公司，民國八十三年.\r\n英文部份\r\n\r\n[Arai89] Arai M., \"Mapping abilities of three-Layer Neural Networks, \", In\r\nProceedings of the International Conference on Neural Networks\r\n(Washington, DC:IEEE), pp I:419-424, 1989.\r\n[Arai93] Arai M., \"Bounds on the Number of Hidden Nodes in Binary-Valued\r\nThree-Layer Neural Networks,\", In IEEE Transactions on Neural\r\nNetworks vol 6, pp 855-860, 1993.\r\n[Chau89] Y. Chauvin, \"A back-propagation algorithm with optimal use of hidden\r\nnode,\" In Advance in Neural Information Processing (2), D.S . Toutetzky,\r\nEd. (Denver 1988), 1989,pp. 519-526.\r\n[Cybe89] G. Cybenko, \"Approximation by superpositions of a sigmoidal function,\"\r\nMathematics of Control, Signals, and Systems, 2(4): pp. 303-314, 1989.\r\n[Cast93] Castellano G., Fanelli A. M., Pelillo M. ,\"An Empirical Comparsion of\r\nNode Pruning Methods for Layerd Feed-forward Neural Networks\", In\r\nProceedings of 1993 International Joint Conference on Neural Networks,\r\npp 321-326, 1993.\r\n[Cun90] Y. Ie Cun, 1.s. Denker, and S.A. Solla, \"Optimal brain damage.\" In D.\r\nTourezky, editor, Advances in Neural Information ProceSSing System 2, pp\r\n598-605. Morgan Kaufmann, 1990.\r\n[Free92] Freeman 1. A. , Skapura D. M.,\"Neural Networks Algorithm, Applications,\r\nand Programming Techniques\", Addison-Wesley, Reading MA, 1992.\r\n[Hertz91] J. Hertz, A. Krogh, R. Palmer, \"Introduction to the theory of neural\r\ncomputation\", Addison-Wesley, Reading MA, 1991.\r\n[Hagi93] Hagiwara M.,\"Removal of hidden nodes and Weights for Back\r\nPropagation Networks\", In Proceedings of 1993 International Joint\r\nConference on Neural Networks, pp 35 1-353,1993.\r\n[Hajn87] A Hajnal, W. Maass, P.Pudlak, M. Szegedy, and G. Turan, \"Threshold\r\ncircuits of bounded depth.\" In Proceedings of the J 987 IEEE Symposium\r\non the Funciations of Science, pp. 99-110,1987.\r\n[Hint87] Geoffrey E. Hinton and Terrence J. Sejnomsk.i. \"Neural network\r\narchitectures for AI\" Tutolial No. MJ?2, AAA187, Seattle, W A, July 1987.\r\n[Huan91] Huang, S. c., & Huang, Y. F. \"Bounds on the number of hidden neurons in\r\nmultilayer perceptrons,\", In IEEE Transactions on Neural Networks Vol 2,\r\npp 47-55, 1991.\r\n[Hush93 J Don R. Hush and Bill G. Horne, \"Progess in Supervised Neural Networks:\r\nWhat`s New Since Lippmann?\", In IEEE SIGNAL ProceSSing magazine, pp\r\n8-39, January 1993.\r\n[Ishi90] M. Ishikawa, \"A structural learning algorithm with forgetting of link\r\nweights,\" Tech. Rep. TR-90-7, Electroteclmical Lab., Tsukuba-City, Japan,\r\n1990.\r\n[Jac088] Adaptation.\" Neural Networks, Vol 1, pp 295-307.\r\n[Ji90] C. Ji, R.R. Snapp, and D. Psaltis, \"Gereralizing smoothness constraints\r\nfrom discrete samples,\" Neural Commputation, Vol. 2, no. 2, pp. 188-197,\r\n1990.\r\n[Kam90] E . D. Karnin,\"A simple procedure for pruning back-propagation trained\r\nneural network,\" IEEE Trans. Neural Networks, Vol. 2, no. 2, pp. 239-242,\r\n1990.\r\n[Krus88] Kruschke, J, \"Creating local and distributed Bottlenecks in Hidden Layers\r\nof Back pop gat ion Networks,\" in Touretzky, D., Hiton, G., and Sejnowski,\r\nT (Eds): Proceeding oj the Connectionist Models Summer School 1988,\r\nMorgan Kaufman Pub!., San Mateo, California, 1988.\r\n[Makh89] 1. Makhoul, A. EI-Jaroudi, and R. Schwartz, \" Fonnation of disconnected\r\ndecision regions with a single hidden layer.\" In Proceedings oj the\r\nInternational Joint Conference on Neural Networks. Vol 1, pp 455-460,\r\n1989.\r\n[Merz88] Merzenich, M.M., Recanzone, G., Jenkins, W.M., Allard, TT, and Nudo,\r\nR)., \"Cortical Representional Plasticity,\" In Rakie, P. and Singer, W.\r\n(Eds) : Neurobiology of Neocortex. John Wiley and Sons Limited. S.\r\nBernhard. Dahlem Konferenzen, 1988.\r\n[McIn89] McInerny, J.M., K.G. Hainer, S. Biafore, and R. Hecht-Nielsen (1989).\r\n\"Back Propagation Error Surfaces Can Have Local Minima.\", In\r\nInternational Joint Conference on Neural Networks (Washington 1989),\r\nvol. II, 627. New York:IEEE.\r\n[Mins69] Minsky, M.L. and S.A. Papert, \"Perceptron,\" Cambridge: MIT Press.\r\nPartially reprinted in Anderson and Rosenberg, 1988.\r\n[Morg91J D.P. Morgan and c.L. Scofield, Neural Networks and Speech Processing.\r\nKluwer Academic Publishers, 1991.\r\n[Moze89] M. C. Mozer and P. Smolensky, \"Skeletonization: A technique for trimming\r\nthe fat from a network via relevance assessment,\", In Advances in Neural\r\nInformation Processing (1), D.S. Touretzky, Ed. (Denver 1988), 1989, pp.\r\n107-115.\r\n[Mura91] K.Murase, YMatsunaga and Y.Nakade: \"A Back-propagation Algorithm\r\nwith Automatically Determines the Number of Association Units\", Proc. of\r\nInternational Joint Conference on Neural Networks(IJCNN-SINGAPORE),\r\nI, p.783-788, 1991.\r\n[pe1i93J M. Pelillo and A. M. Fanelli, \"A method of pruning layered feed-forward\r\nneural networks,\" in Proc IWANN`93, Sitges, Barcelona, June 1993\r\n(Berlin: Springer-Verlag).\r\n[Reed93] R. Reed, \"Pruning A1goritluns-A Survey\", in IEEE Transactions On\r\nNeural Network, Vo14, NO.5, September 1993, pp. 740-747.\r\n[Rume86] Rumelhart, D.E., James McClelland, \"Parallel Distributed Processing, \", Vol\r\n1 and 2, MIT Press, Cambridge, MA, 1986.\r\n[Siet88] Sietsma., 1. and R.lF. Dow (1988), \"Neural Net Pruning-Why and How.\"\r\nIn IEEE International Conference on Neural Networks (San Diego 1988),\r\nVol. 1,325-333. New York:IEEE.\r\n[Tsai93] Tsaih, R., \"The Softening Learning Procedure.\" In Mathematical and\r\nComputer Modeling, Vol. 18, No.8, 61-64.\r\n[Tsai93] Tsaih, R., \"The Softening Learning Procedure for The Layered Feedforward\r\nNetworks With Multiple Output nodes\". Proceeding of\r\nIntemational Joint Conference on Neural Networks, Nagoya, I, 593-596.\r\n[Wata93] Watanabe E. and Shimizu H., \"Algorithm for Pruning Hidden Nodes in\r\nMulti Layerd Neural Network for binary StimulusClassification Problem\",\r\nIn Proceedings of 1993 Intemational Joint Conference on Neural\r\nNetworks, pp 327-330, 1993 .\r\n[Weig91] A. S. Weigend, D. E. Rumelhart, and B. A. Huberman, \"Generalization by\r\nweight-elimination with application to forcasting,\" in Advances in Neural\r\nInformation Processing (3), R. Lippmann, J. Moody, and D. Touretzky,\r\nEds, 1991, pp. 875-882.\r\n[Werb90] P. J. Werbos, \"Backpropagation Through Time: What It Does and How to\r\nDo It\", In Proceedings of the IEEE, Vol 78, No.1 0, October 1990, pp .\r\n1550-1560.\r\n[Werb74] P. Werbos. IIBeyond Regression :New Tools for Prediction and the Analysis\r\nin the Behavioral Sciences.\" Ph.D. thesis, Harvard, Cambridge, MA,\r\nAugust 1974.
描述 碩士
國立政治大學
資訊管理學系
81356007
資料來源 http://thesis.lib.nccu.edu.tw/record/#B2002003876
資料類型 thesis
dc.contributor.advisor 蔡瑞煌zh_TW
dc.contributor.advisor Tsaihen_US
dc.contributor.author (Authors) 徐志鈞zh_TW
dc.contributor.author (Authors) Hsu Chih Chunen_US
dc.creator (作者) 徐志鈞zh_TW
dc.creator (作者) Chun, Hsu Chihen_US
dc.date (日期) 1994en_US
dc.date.accessioned 29-Apr-2016 16:30:45 (UTC+8)-
dc.date.available 29-Apr-2016 16:30:45 (UTC+8)-
dc.date.issued (上傳時間) 29-Apr-2016 16:30:45 (UTC+8)-
dc.identifier (Other Identifiers) B2002003876en_US
dc.identifier.uri (URI) https://ah.lib.nccu.edu.tw/item?item_id=100568-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 資訊管理學系zh_TW
dc.description (描述) 81356007zh_TW
dc.description.abstract (摘要) 大部的類神經網路均為解決特定問題而設計，並非真正去模擬人腦的功能zh_TW
dc.description.abstract (摘要) Most of artification Neural Networks are designed to resolveen_US
dc.description.tableofcontents Contents\r\n1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1\r\n\r\n2 Literature Review\r\n 2.1. The Back Propagation Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3\r\n 2.1.1 Network Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4\r\n 2.1.2 The Back Propagation Learning algorithm . . . . . . . . . . . . . . . . . . . . . . . . . .4\r\n 2.1.3 The Update Rule in The Back Propagation Learning algorithm . . . . . . . . . 7\r\n 2.1.4 Local Minimum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8\r\n 2.2 Node Pruning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10\r\n 2.2.1 Sensitivity Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10\r\n 2.2.2 Penalty Term and Weight Decay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13\r\n 2.2.3 Other pruning algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15\r\n 2.3 Softening Learning Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18\r\n 2.3.1 Network Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18\r\n 2.3.2 Two Classes Categorization Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19\r\n 2.3.3 The Learning Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20\r\n 2.3.4 Cramming Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21\r\n 2.3.5 Reasoning Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22\r\n\r\n3 The Reasoning Neural Network\r\n 3.1 The RNN Network Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25\r\n 3.2 Prime Parts of the RNN Learning Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 25\r\n 3.3 Explanation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27\r\n 3.4 One Edition of the RNN Learning Procedure . . . . . . . . . . . . . . . . . . . . . .. . . . . 29\r\n\r\n4 Experiments\r\n 4.1 Parity problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33\r\n 4.2 Output-hidden problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35\r\n 4.3 Encoder Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37\r\n 4.4 Chinese character recognition problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40\r\n\r\n5 Discussions and Future works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45\r\n\r\nBibliography 48zh_TW
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#B2002003876en_US
dc.subject (關鍵詞) 推理類神經網路zh_TW
dc.subject (關鍵詞) 軟性學習程序zh_TW
dc.subject (關鍵詞) 線性分割條件zh_TW
dc.subject (關鍵詞) 不相關節點zh_TW
dc.subject (關鍵詞) 推理機能zh_TW
dc.subject (關鍵詞) The Reasoning Neural Networken_US
dc.subject (關鍵詞) the softening learning algorithmen_US
dc.subject (關鍵詞) linearly separating conditionen_US
dc.title (題名) 推理類神經網路及其應用zh_TW
dc.title (題名) The Reasoning Neural Network and It`s Applicationsen_US
dc.type (資料類型) thesisen_US
dc.relation.reference (參考文獻) Bibliography\r\n中文部份\r\n[葉怡成93]葉怡成，類神經網路模式應用與實作，儒林圖書公司，民國八十三年.\r\n英文部份\r\n\r\n[Arai89] Arai M., \"Mapping abilities of three-Layer Neural Networks, \", In\r\nProceedings of the International Conference on Neural Networks\r\n(Washington, DC:IEEE), pp I:419-424, 1989.\r\n[Arai93] Arai M., \"Bounds on the Number of Hidden Nodes in Binary-Valued\r\nThree-Layer Neural Networks,\", In IEEE Transactions on Neural\r\nNetworks vol 6, pp 855-860, 1993.\r\n[Chau89] Y. Chauvin, \"A back-propagation algorithm with optimal use of hidden\r\nnode,\" In Advance in Neural Information Processing (2), D.S . Toutetzky,\r\nEd. (Denver 1988), 1989,pp. 519-526.\r\n[Cybe89] G. Cybenko, \"Approximation by superpositions of a sigmoidal function,\"\r\nMathematics of Control, Signals, and Systems, 2(4): pp. 303-314, 1989.\r\n[Cast93] Castellano G., Fanelli A. M., Pelillo M. ,\"An Empirical Comparsion of\r\nNode Pruning Methods for Layerd Feed-forward Neural Networks\", In\r\nProceedings of 1993 International Joint Conference on Neural Networks,\r\npp 321-326, 1993.\r\n[Cun90] Y. Ie Cun, 1.s. Denker, and S.A. Solla, \"Optimal brain damage.\" In D.\r\nTourezky, editor, Advances in Neural Information ProceSSing System 2, pp\r\n598-605. Morgan Kaufmann, 1990.\r\n[Free92] Freeman 1. A. , Skapura D. M.,\"Neural Networks Algorithm, Applications,\r\nand Programming Techniques\", Addison-Wesley, Reading MA, 1992.\r\n[Hertz91] J. Hertz, A. Krogh, R. Palmer, \"Introduction to the theory of neural\r\ncomputation\", Addison-Wesley, Reading MA, 1991.\r\n[Hagi93] Hagiwara M.,\"Removal of hidden nodes and Weights for Back\r\nPropagation Networks\", In Proceedings of 1993 International Joint\r\nConference on Neural Networks, pp 35 1-353,1993.\r\n[Hajn87] A Hajnal, W. Maass, P.Pudlak, M. Szegedy, and G. Turan, \"Threshold\r\ncircuits of bounded depth.\" In Proceedings of the J 987 IEEE Symposium\r\non the Funciations of Science, pp. 99-110,1987.\r\n[Hint87] Geoffrey E. Hinton and Terrence J. Sejnomsk.i. \"Neural network\r\narchitectures for AI\" Tutolial No. MJ?2, AAA187, Seattle, W A, July 1987.\r\n[Huan91] Huang, S. c., & Huang, Y. F. \"Bounds on the number of hidden neurons in\r\nmultilayer perceptrons,\", In IEEE Transactions on Neural Networks Vol 2,\r\npp 47-55, 1991.\r\n[Hush93 J Don R. Hush and Bill G. Horne, \"Progess in Supervised Neural Networks:\r\nWhat`s New Since Lippmann?\", In IEEE SIGNAL ProceSSing magazine, pp\r\n8-39, January 1993.\r\n[Ishi90] M. Ishikawa, \"A structural learning algorithm with forgetting of link\r\nweights,\" Tech. Rep. TR-90-7, Electroteclmical Lab., Tsukuba-City, Japan,\r\n1990.\r\n[Jac088] Adaptation.\" Neural Networks, Vol 1, pp 295-307.\r\n[Ji90] C. Ji, R.R. Snapp, and D. Psaltis, \"Gereralizing smoothness constraints\r\nfrom discrete samples,\" Neural Commputation, Vol. 2, no. 2, pp. 188-197,\r\n1990.\r\n[Kam90] E . D. Karnin,\"A simple procedure for pruning back-propagation trained\r\nneural network,\" IEEE Trans. Neural Networks, Vol. 2, no. 2, pp. 239-242,\r\n1990.\r\n[Krus88] Kruschke, J, \"Creating local and distributed Bottlenecks in Hidden Layers\r\nof Back pop gat ion Networks,\" in Touretzky, D., Hiton, G., and Sejnowski,\r\nT (Eds): Proceeding oj the Connectionist Models Summer School 1988,\r\nMorgan Kaufman Pub!., San Mateo, California, 1988.\r\n[Makh89] 1. Makhoul, A. EI-Jaroudi, and R. Schwartz, \" Fonnation of disconnected\r\ndecision regions with a single hidden layer.\" In Proceedings oj the\r\nInternational Joint Conference on Neural Networks. Vol 1, pp 455-460,\r\n1989.\r\n[Merz88] Merzenich, M.M., Recanzone, G., Jenkins, W.M., Allard, TT, and Nudo,\r\nR)., \"Cortical Representional Plasticity,\" In Rakie, P. and Singer, W.\r\n(Eds) : Neurobiology of Neocortex. John Wiley and Sons Limited. S.\r\nBernhard. Dahlem Konferenzen, 1988.\r\n[McIn89] McInerny, J.M., K.G. Hainer, S. Biafore, and R. Hecht-Nielsen (1989).\r\n\"Back Propagation Error Surfaces Can Have Local Minima.\", In\r\nInternational Joint Conference on Neural Networks (Washington 1989),\r\nvol. II, 627. New York:IEEE.\r\n[Mins69] Minsky, M.L. and S.A. Papert, \"Perceptron,\" Cambridge: MIT Press.\r\nPartially reprinted in Anderson and Rosenberg, 1988.\r\n[Morg91J D.P. Morgan and c.L. Scofield, Neural Networks and Speech Processing.\r\nKluwer Academic Publishers, 1991.\r\n[Moze89] M. C. Mozer and P. Smolensky, \"Skeletonization: A technique for trimming\r\nthe fat from a network via relevance assessment,\", In Advances in Neural\r\nInformation Processing (1), D.S. Touretzky, Ed. (Denver 1988), 1989, pp.\r\n107-115.\r\n[Mura91] K.Murase, YMatsunaga and Y.Nakade: \"A Back-propagation Algorithm\r\nwith Automatically Determines the Number of Association Units\", Proc. of\r\nInternational Joint Conference on Neural Networks(IJCNN-SINGAPORE),\r\nI, p.783-788, 1991.\r\n[pe1i93J M. Pelillo and A. M. Fanelli, \"A method of pruning layered feed-forward\r\nneural networks,\" in Proc IWANN`93, Sitges, Barcelona, June 1993\r\n(Berlin: Springer-Verlag).\r\n[Reed93] R. Reed, \"Pruning A1goritluns-A Survey\", in IEEE Transactions On\r\nNeural Network, Vo14, NO.5, September 1993, pp. 740-747.\r\n[Rume86] Rumelhart, D.E., James McClelland, \"Parallel Distributed Processing, \", Vol\r\n1 and 2, MIT Press, Cambridge, MA, 1986.\r\n[Siet88] Sietsma., 1. and R.lF. Dow (1988), \"Neural Net Pruning-Why and How.\"\r\nIn IEEE International Conference on Neural Networks (San Diego 1988),\r\nVol. 1,325-333. New York:IEEE.\r\n[Tsai93] Tsaih, R., \"The Softening Learning Procedure.\" In Mathematical and\r\nComputer Modeling, Vol. 18, No.8, 61-64.\r\n[Tsai93] Tsaih, R., \"The Softening Learning Procedure for The Layered Feedforward\r\nNetworks With Multiple Output nodes\". Proceeding of\r\nIntemational Joint Conference on Neural Networks, Nagoya, I, 593-596.\r\n[Wata93] Watanabe E. and Shimizu H., \"Algorithm for Pruning Hidden Nodes in\r\nMulti Layerd Neural Network for binary StimulusClassification Problem\",\r\nIn Proceedings of 1993 Intemational Joint Conference on Neural\r\nNetworks, pp 327-330, 1993 .\r\n[Weig91] A. S. Weigend, D. E. Rumelhart, and B. A. Huberman, \"Generalization by\r\nweight-elimination with application to forcasting,\" in Advances in Neural\r\nInformation Processing (3), R. Lippmann, J. Moody, and D. Touretzky,\r\nEds, 1991, pp. 875-882.\r\n[Werb90] P. J. Werbos, \"Backpropagation Through Time: What It Does and How to\r\nDo It\", In Proceedings of the IEEE, Vol 78, No.1 0, October 1990, pp .\r\n1550-1560.\r\n[Werb74] P. Werbos. IIBeyond Regression :New Tools for Prediction and the Analysis\r\nin the Behavioral Sciences.\" Ph.D. thesis, Harvard, Cambridge, MA,\r\nAugust 1974.zh_TW