學術產出-Theses

Article View/Open

Publication Export

Google ScholarTM

政大圖書館

Citation Infomation

  • No doi shows Citation Infomation
題名 應用資料採礦技術於資料庫加值中的插補方法比較
Imputation of value-added database in data mining
作者 黃雅芳
貢獻者 鄭宇庭<br>謝邦昌
<br>
黃雅芳
關鍵詞 資料採礦
資料庫加值
稀少資料
遺漏值
插補
data mining
value-added database
rare data
missing data
imputation
日期 2003
上傳時間 2009-09-14
摘要 資料在企業資訊來源中扮演了極為重要的角色,特別是在現今知識與技術的世代裡。如果對於一個有意義且具有代表性資料庫中的遺漏值能夠正確的處理,那麼對於企業資訊而言,是一個大有可為的突破。
      然而,有時我們或許會遇到一些不是那麼完善的資料庫,當資料庫中的資料有遺漏值時,從這樣資料庫中所獲得的結果,或許會是一些有偏差或容易令人誤解的結果。因此,本研究的目的在於插補遺漏值為資料庫加值,進而根據遺漏值類型建立插補模型。
      如果遺漏值為連續型,用迴歸模型和倒傳遞類神經模型來進行插補;如果遺漏值為類別型,採用邏輯斯迴歸、倒傳遞類神經和決策樹進行插補分析。經由模擬的結果顯示,對於連續型的遺漏值,迴歸模型提供了最佳的插補估計;而類別型的遺漏值,C5.0決策樹是最佳的選擇。此外,對於資料庫中的稀少資料,當連續型的遺漏值,倒傳遞類神經模型提供了最佳的插補估計;而類別型的遺漏值,亦是C5.0決策樹是最佳的選擇。
Data plays a vital role as source of information to the organization especially in the era of information and technology. A meaningful, qualitative and representative database if properly handled could mean a promising breakthrough to the organizations.
      However, from time to time, we may encounter a not so perfect database, that is we have the situation where the data in the database is missing. With the incomplete database, the results obtained from such database may provide biased or misleading solutions. Therefore, the purpose of this research is to place its emphasis on imputing missing data of the value-added database then builds the model in accordance to the type of data.
      If the missing data type is continuous, regression model and BPNN neural network is applied. If the missing data type is categorical, logistic regression, BPNN neural network and decision tree is chosen for the application. Our result has shown that for the continuous missing data, the regression model proved to deliver the best estimate. For the categorical missing data, C5.0 decision tree model is the chosen one. Besides, as regards the rare data missing in the database, our result has shown that for the continuous missing data, the BPNN neural network proved to deliver the best estimate.
     For the categorical missing data, C5.0 decision tree model is the chosen one.
參考文獻 1. Alan Agresti(1996)An Introduction to Categorical Data Analysis. Wiley interscience.
2. Alvin C. Rencher(2002)Methods of Multivariate Analysis, 2nd ed. Wiley interscience.
3. Donald B. Rubin Multiple imputation for nonresponse in surveys. Wiley series in Probability and Statistics.
4. Joop J. Hox (1999) A review of current software for handing missing data. Kwantitatieve Methoden, 62, 123-138
5. Judi Scheffer (2002) Dealing with missing data. Res. Lett. Inf. Math. Sci. pp153-160
6. M. P. Craven (1997) A faster learning neural network classifier using selective backpropagation. Proceedings of the fourth IEEE international Conference on electronics, circuits and systems, Cairo, Egypt, Volume 1, pp 254-258
7. Margaret H. Dunham(2002)Data Mining---Introductory and Advanced Topics. Prentice Hall.
8. Robert E. Fay (1996) Alternative paradigms for the analysis of imputed survey data. Journal of the American statistical association, Vol. 91, No. 434, 490-498
9. Steven Roman(2002)Access Database Design & Programming. O"Reilly
10. Hyunyoon Yun, Danshim Ha, Buhyun Hwang and Keun Ho Ryu (2003) Mining association rules on significant rare data using relative support. The journal of systems and software 67. pp181-191
11. John O. Rawlings, Sastry G. Pantula and David A. Dickey(1998) Applied Regression Analysis---A Research Tool, 2nd ed. Springer.
12. Roderick J.A. Little and Donald B. Rubin(2002) Statistical Analysis with Missing Data, 2nd ed. Wiley interscience.
13. William G. Madow, Ingram Olkin and Donald B. Rubin (1983) Incomplete data in sample surveys:Theory and Bibliographies. Academic Press.
描述 碩士
國立政治大學
統計研究所
91354018
92
資料來源 http://thesis.lib.nccu.edu.tw/record/#G0091354018
資料類型 thesis
dc.contributor.advisor 鄭宇庭<br>謝邦昌zh_TW
dc.contributor.advisor <br>en_US
dc.contributor.author (Authors) 黃雅芳zh_TW
dc.creator (作者) 黃雅芳zh_TW
dc.date (日期) 2003en_US
dc.date.accessioned 2009-09-14-
dc.date.available 2009-09-14-
dc.date.issued (上傳時間) 2009-09-14-
dc.identifier (Other Identifiers) G0091354018en_US
dc.identifier.uri (URI) https://nccur.lib.nccu.edu.tw/handle/140.119/30887-
dc.description (描述) 碩士zh_TW
dc.description (描述) 國立政治大學zh_TW
dc.description (描述) 統計研究所zh_TW
dc.description (描述) 91354018zh_TW
dc.description (描述) 92zh_TW
dc.description.abstract (摘要) 資料在企業資訊來源中扮演了極為重要的角色,特別是在現今知識與技術的世代裡。如果對於一個有意義且具有代表性資料庫中的遺漏值能夠正確的處理,那麼對於企業資訊而言,是一個大有可為的突破。
      然而,有時我們或許會遇到一些不是那麼完善的資料庫,當資料庫中的資料有遺漏值時,從這樣資料庫中所獲得的結果,或許會是一些有偏差或容易令人誤解的結果。因此,本研究的目的在於插補遺漏值為資料庫加值,進而根據遺漏值類型建立插補模型。
      如果遺漏值為連續型,用迴歸模型和倒傳遞類神經模型來進行插補;如果遺漏值為類別型,採用邏輯斯迴歸、倒傳遞類神經和決策樹進行插補分析。經由模擬的結果顯示,對於連續型的遺漏值,迴歸模型提供了最佳的插補估計;而類別型的遺漏值,C5.0決策樹是最佳的選擇。此外,對於資料庫中的稀少資料,當連續型的遺漏值,倒傳遞類神經模型提供了最佳的插補估計;而類別型的遺漏值,亦是C5.0決策樹是最佳的選擇。
zh_TW
dc.description.abstract (摘要) Data plays a vital role as source of information to the organization especially in the era of information and technology. A meaningful, qualitative and representative database if properly handled could mean a promising breakthrough to the organizations.
      However, from time to time, we may encounter a not so perfect database, that is we have the situation where the data in the database is missing. With the incomplete database, the results obtained from such database may provide biased or misleading solutions. Therefore, the purpose of this research is to place its emphasis on imputing missing data of the value-added database then builds the model in accordance to the type of data.
      If the missing data type is continuous, regression model and BPNN neural network is applied. If the missing data type is categorical, logistic regression, BPNN neural network and decision tree is chosen for the application. Our result has shown that for the continuous missing data, the regression model proved to deliver the best estimate. For the categorical missing data, C5.0 decision tree model is the chosen one. Besides, as regards the rare data missing in the database, our result has shown that for the continuous missing data, the BPNN neural network proved to deliver the best estimate.
     For the categorical missing data, C5.0 decision tree model is the chosen one.
en_US
dc.description.tableofcontents LIST OF TABLES……………………………………………………………iii
     LIST OF FIGURES……………………………………………………………v
     1 Introduction………………………………………………………………1
      1.1 Research Background………………………………………………1
      1.2 Research Motive and Purpose……………………………………3
      1.3 Research Procedure…………………………………………………4
      1.4 Research Outlay……………………………………………………5
     2 Literature Review………………………………………………………6
      2.1 Database and Data Warehouse……………………………………6
      2.2 Relational Database………………………………………………7
      2.3 Data Mining…………………………………………………………9
      2.4 Introduction to Missing Data…………………………………12
      2.5 introduction to Rare data………………………………………13
      2.6 Imputation Methods………………………………………………14
      2.7 The Predictive Model for the imputation……………………21
      2.7.1 Regression Model…………………………………………21
      2.7.2 Logistic Regression Model………………………………22
      2.7.3 Artificial Neural Network………………………………23
      2.7.4 Decision Tree………………………………………………27
     3 Research Methodology…………………………………………………30
      3.1 Research Concept…………………………………………………30
      3.2 Research Frame……………………………………………………31
     4 Evaluating Performance………………………………………………34
      4.1 Data understanding………………………………………………34
      4.2 Data preparation…………………………………………………35
      4.3 Modeling……………………………………………………………38
      4.4 Evaluation…………………………………………………………54
     5 Conclusions and Suggestions…………………………………………76
      5.1 Conclusions…………………………………………………………76
      5.2 Suggestions…………………………………………………………78
     References……………………………………………………………………82
zh_TW
dc.language.iso en_US-
dc.source.uri (資料來源) http://thesis.lib.nccu.edu.tw/record/#G0091354018en_US
dc.subject (關鍵詞) 資料採礦zh_TW
dc.subject (關鍵詞) 資料庫加值zh_TW
dc.subject (關鍵詞) 稀少資料zh_TW
dc.subject (關鍵詞) 遺漏值zh_TW
dc.subject (關鍵詞) 插補zh_TW
dc.subject (關鍵詞) data miningen_US
dc.subject (關鍵詞) value-added databaseen_US
dc.subject (關鍵詞) rare dataen_US
dc.subject (關鍵詞) missing dataen_US
dc.subject (關鍵詞) imputationen_US
dc.title (題名) 應用資料採礦技術於資料庫加值中的插補方法比較zh_TW
dc.title (題名) Imputation of value-added database in data miningen_US
dc.type (資料類型) thesisen
dc.relation.reference (參考文獻) 1. Alan Agresti(1996)An Introduction to Categorical Data Analysis. Wiley interscience.zh_TW
dc.relation.reference (參考文獻) 2. Alvin C. Rencher(2002)Methods of Multivariate Analysis, 2nd ed. Wiley interscience.zh_TW
dc.relation.reference (參考文獻) 3. Donald B. Rubin Multiple imputation for nonresponse in surveys. Wiley series in Probability and Statistics.zh_TW
dc.relation.reference (參考文獻) 4. Joop J. Hox (1999) A review of current software for handing missing data. Kwantitatieve Methoden, 62, 123-138zh_TW
dc.relation.reference (參考文獻) 5. Judi Scheffer (2002) Dealing with missing data. Res. Lett. Inf. Math. Sci. pp153-160zh_TW
dc.relation.reference (參考文獻) 6. M. P. Craven (1997) A faster learning neural network classifier using selective backpropagation. Proceedings of the fourth IEEE international Conference on electronics, circuits and systems, Cairo, Egypt, Volume 1, pp 254-258zh_TW
dc.relation.reference (參考文獻) 7. Margaret H. Dunham(2002)Data Mining---Introductory and Advanced Topics. Prentice Hall.zh_TW
dc.relation.reference (參考文獻) 8. Robert E. Fay (1996) Alternative paradigms for the analysis of imputed survey data. Journal of the American statistical association, Vol. 91, No. 434, 490-498zh_TW
dc.relation.reference (參考文獻) 9. Steven Roman(2002)Access Database Design & Programming. O"Reillyzh_TW
dc.relation.reference (參考文獻) 10. Hyunyoon Yun, Danshim Ha, Buhyun Hwang and Keun Ho Ryu (2003) Mining association rules on significant rare data using relative support. The journal of systems and software 67. pp181-191zh_TW
dc.relation.reference (參考文獻) 11. John O. Rawlings, Sastry G. Pantula and David A. Dickey(1998) Applied Regression Analysis---A Research Tool, 2nd ed. Springer.zh_TW
dc.relation.reference (參考文獻) 12. Roderick J.A. Little and Donald B. Rubin(2002) Statistical Analysis with Missing Data, 2nd ed. Wiley interscience.zh_TW
dc.relation.reference (參考文獻) 13. William G. Madow, Ingram Olkin and Donald B. Rubin (1983) Incomplete data in sample surveys:Theory and Bibliographies. Academic Press.zh_TW