TREC現況及其對資訊檢索研究之影響

Abstract

測試集為一在規範化環境中測量資訊檢索效益的機制，研究者常藉著它來改進資訊檢索系統。然而，過去所使用的測試集規模通常很小，無法有效模擬真實的資訊檢索環境。自1992年起，一個稱為TREC ( Text Retrieval Conference)的評估會議每年在美國舉行。它是一個大型的實驗室測試機制，可以比較不同檢索系統的效能，並能提供一個論壇讓參與者相互切磋討論。今日，TREC在資訊檢索領域中已成為倍受矚目的標竿測試環境。本文試圖對TREC做一個概覽性的介紹，包括它所使用的測試集、測量準則、測試項目及過去的發展概況。另外，也透過一些相關文獻的分析，對TREC在資訊檢索領域中的價值及所扮演的角色作一簡要的評述。

Test Collection is a mechanism which is used to explore relative benefits of different retrieval system in a normalized environment, and researchers traditionally use it to improve their retrieval systems. However, for many cases in the past, the scale of Test Collection was quite small, thus it can not simulate the real state of information retrieval task. Since 1992, a series of annual benchmarking evaluation exercises, called TREC (Text REtrieval Conference), have launched in the USA. TREC experiments were designed to allow large-scale laboratory testing, compare the effectiveness and performance of different information retrieval techniques, and provide a forum for research groups to discuss their work together. Today, TREC has become the major experimental effort in the IR field. In this paper, we present an overview of TREC, the test collection, the test measures, the main tasks and specific `tracks`, and the development during the past six years. Besides, we also briefly review the performance and the evaluation of IR.