TrebleCLEF Logo
Evaluation, Best Practices and Collaboration for Multilingual Information Access
Summer School
Best Practices
Latest News
Cross Language Evaluation Form: CLEF 2009 Call for Participation
Registration for CLEF 2009 is now...
- Link

INFILE: INFILE : Information, Filtering, Evaluation Evaluation campaign for information...
- Link

Resources & Links

This page contains pointers to helpful resources and references. Please help us extend and maintain this list by sending your feedback.

Evaluation Forums

CLEF

Cross-Language Evaluation Forum

TREC

Text Retrieval Conference

NTCIR

NII-NACSIS Test Collection for IR Systems

INEX

INitiative for the Evaluation of XML Retrieval

FIRE Forum for Information Retrieval Evaluation

Research Programmes

ELSNET

European Network of Excellence in Human Languages Technologies

TIDES

Translingual Information Detection, Extraction and Summarization (DARPA)

Resources

ELRA/ELDA

Evaluations and Language Resources Distribution Association

LDC

Linguistic Data Consortium

ODS

United Nations Official Documents Online - consists of 800,000 searchable parallel documents in Arabic, Chinese, English, French, Russian and Spanish, primarily in PDF and / or MS Word format. Resource Shelf summarizes the main features and other UN resources.

Prise

Free retrieval system from NIST, complete with (simplistic) German and French stemmers

SDA data
German/French/Italian 1988-90
Please consult the README.txt file.

Training collection used for the TREC6-8 CLIR tracks (password-protected. Please consult the CLEF administration.)

Web of Online-Dictionaries

online dictionaries for 100s of languages

Altavista Babelfish

(Systran-powered) online machine translation from Altavista

Google

On-line translation tools

FreeTranslation

On-line translation tools

InterTran

On-line translation tools

Reverso Online

On-line translation tools

Selected References on Evaluation
(for CLEF papers, see our Website under Publications)

Harman, D. K. (1993): Overview of the First Text REtrieval Conference (TREC-1). In The First Text REtrieval Conference (TREC-1), NIST Special Publication 500-207.

Harman, D. K. (1995): The TREC Conferences. In Sparck Jones, K. and Willett, P. (Eds.): Readings in Information Retrieval, Morgan Kaufmann Publishers.

Kando, N., Kuriyama, K., Nozue, T., Eguchi, K., Kato, H., Hidaka, S., and Adachi, J. (1999): The NTCIR Workshop: The First Evaluation Workshop on Japanese Text Retrieval and Cross-Language Information Retrieval. In Proceedings of the 4th International Workshop on Information Retrieval with Asian Languages.

Schäuble, P., and Sheridan, P. (1998): Cross-Language Information Retrieval (CLIR) Track Overview. In Proceedings of the Sixth Text Retrieval Conference (TREC6). NIST Special Publication 500-240.

Sparck Jones K., and Van Rijsbergen C. (1975): Report on the Need for and Provision of an "Ideal" Information Retrieval Test Collection. In British Library Research and Development Report 5266, Computer Laboratory, University of Cambridge.

Voorhees, E. (1998): Variations in Relevance Judgments and the Measurement of Retrieval Effectiveness. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.

Additions and comments welcome!