學術產出-Periodical Articles
Article View/Open
Publication Export
-
題名 A generic construct based workload model for web search 作者 諶家蘭
Seng, Jia-Lang ; Ko, I-Feng ; Lin, Binshan貢獻者 會計系 關鍵詞 Web search; Information retrieval; Generic construct; Benchmark method; Workload model; Performance measurement; Evaluation; Comparison 日期 2009.09 上傳時間 11-Nov-2013 17:04:28 (UTC+8) 摘要 Benchmarks are vital tools in the performance measurement, evaluation, and comparison of computer hardware and software systems. Standard benchmarks such as the TREC, TPC, SPEC, SAP, Oracle, Microsoft, IBM, Wisconsin, AS3AP, OO1, OO7, XOO7 benchmarks have been used to assess the system performance. These benchmarks are domain-specific and domain-dependent in that they model typical applications and tie to a problem domain. Test results from these benchmarks are estimates of possible system performance for certain pre-determined problem types. When the user domain differs from the standard problem domain or when the application workload is divergent from the standard workload, they do not provide an accurate way to measure the system performance of the user problem domain. System performance of the actual problem domain in terms of data and transactions may vary significantly from the standard benchmarks. In this research, we address the issue of generalization and precision of benchmark workload model for web search technology. The current performance measurement and evaluation method suffers from the rough estimate of system performance which varies widely when the problem domain changes. The performance results provided by the vendors cannot be reproduced nor reused in the real users’ environment. Hence, in this research, we tackle the issue of domain boundness and workload boundness which represents the root of the problem of imprecise, ir-representative, and ir-reproducible performance results. We address the issue by presenting a domain-independent and workload-independent workload model benchmark method which is developed from the perspective of the user requirements and generic constructs. We present a user-driven workload model to develop a benchmark in a process of workload requirements representation, transformation, and generation via the common carrier of generic constructs. We aim to create a more generalized and precise evaluation method which derives test suites from the actual user domain and application setting. The workload model benchmark method comprises three main components. They are a high-level workload specification scheme, a translator of the scheme, and a set of generators to generate the test database and the test suite. They are based on the generic constructs. The specification scheme is used to formalize the workload requirements. The translator is used to transform the specification. The generator is used to produce the test database and the test workload. We determine the generic constructs via the analysis of search methods. The generic constructs form a page model, a query model, and a control model in the workload model development. The page model describes the web page structure. The query model defines the logics to query the web. The control model defines the control variables to set up the experiments. 關聯 Information Processing & Management, 45(5) , 529-554 資料類型 article DOI http://dx.doi.org/10.1016/j.ipm.2009.04.004 dc.contributor 會計系 en_US dc.creator (作者) 諶家蘭 zh_TW dc.creator (作者) Seng, Jia-Lang ; Ko, I-Feng ; Lin, Binshan en_US dc.date (日期) 2009.09 en_US dc.date.accessioned 11-Nov-2013 17:04:28 (UTC+8) - dc.date.available 11-Nov-2013 17:04:28 (UTC+8) - dc.date.issued (上傳時間) 11-Nov-2013 17:04:28 (UTC+8) - dc.identifier.uri (URI) http://nccur.lib.nccu.edu.tw/handle/140.119/61590 - dc.description.abstract (摘要) Benchmarks are vital tools in the performance measurement, evaluation, and comparison of computer hardware and software systems. Standard benchmarks such as the TREC, TPC, SPEC, SAP, Oracle, Microsoft, IBM, Wisconsin, AS3AP, OO1, OO7, XOO7 benchmarks have been used to assess the system performance. These benchmarks are domain-specific and domain-dependent in that they model typical applications and tie to a problem domain. Test results from these benchmarks are estimates of possible system performance for certain pre-determined problem types. When the user domain differs from the standard problem domain or when the application workload is divergent from the standard workload, they do not provide an accurate way to measure the system performance of the user problem domain. System performance of the actual problem domain in terms of data and transactions may vary significantly from the standard benchmarks. In this research, we address the issue of generalization and precision of benchmark workload model for web search technology. The current performance measurement and evaluation method suffers from the rough estimate of system performance which varies widely when the problem domain changes. The performance results provided by the vendors cannot be reproduced nor reused in the real users’ environment. Hence, in this research, we tackle the issue of domain boundness and workload boundness which represents the root of the problem of imprecise, ir-representative, and ir-reproducible performance results. We address the issue by presenting a domain-independent and workload-independent workload model benchmark method which is developed from the perspective of the user requirements and generic constructs. We present a user-driven workload model to develop a benchmark in a process of workload requirements representation, transformation, and generation via the common carrier of generic constructs. We aim to create a more generalized and precise evaluation method which derives test suites from the actual user domain and application setting. The workload model benchmark method comprises three main components. They are a high-level workload specification scheme, a translator of the scheme, and a set of generators to generate the test database and the test suite. They are based on the generic constructs. The specification scheme is used to formalize the workload requirements. The translator is used to transform the specification. The generator is used to produce the test database and the test workload. We determine the generic constructs via the analysis of search methods. The generic constructs form a page model, a query model, and a control model in the workload model development. The page model describes the web page structure. The query model defines the logics to query the web. The control model defines the control variables to set up the experiments. en_US dc.format.extent 2145700 bytes - dc.format.mimetype application/pdf - dc.language.iso en_US - dc.relation (關聯) Information Processing & Management, 45(5) , 529-554 en_US dc.subject (關鍵詞) Web search; Information retrieval; Generic construct; Benchmark method; Workload model; Performance measurement; Evaluation; Comparison en_US dc.title (題名) A generic construct based workload model for web search en_US dc.type (資料類型) article en dc.identifier.doi (DOI) 10.1016/j.ipm.2009.04.004 en_US dc.doi.uri (DOI) http://dx.doi.org/10.1016/j.ipm.2009.04.004 en_US
