NTCIR-5 CLIR Task
last update: Oct./11/2005
[NTCIR Home] [NTCIR Workshop 5 Meeting Home]
What's New Oct./11/2005 No. of topics for each document sets Total numbers of topics for evaluation are as follows.
Reminder: deadline of submitting working notes (1) For groups not using English docs: Oct./15/2005 (2) For groups using English docs: Nov./01/2005 Please visit to http://research.nii.ac.jp/ntcir-ws5/ Oct./02/2005 Release of trec_eval results for search runs using English document sets (E and CJKE) Evaluation results for search runs using English document sets (E and CJKE) becomes available. Please download it. CAUTION: to participants using English document sets. We remove three topics (No.026) from our evaluation by trec_eval due to the fact that those topics have almost no relevant document. Therefore, in the case of using English document sets, the total number of is only 49. Sep./19/2005 Ave/Med/MAX/Mis statistics Statistics on average, median, max and min of mean average precision (MAP) by search type (eg., C-C-T) become available. Please download csv files (condensed into a file CSV.zip) from your web page for downloading evaluation results (Please access eval.html by the same ID and password as before. see below). Sep./13/2005 Release of evaluation results Delivery of evaluation results begins (except those for runs using English document sets). Please download it from our server. Its URL is http://rcir.nii.ac.jp/ntcir/access/XXX/YYY/ntcir5/clir/eval.html Note: (1) XXX and YYY have been already assigned to each participant respectively (same as those in document and topic release). (2) ID and password for access are also the same as those for download of document and topic. (3) Please access to "eval.html" file, not "index.html"file! CAUTION: to participants using Japanese document sets. We remove three topics (No.021, 023 and 039) from our evaluation by trec_eval due to the fact that those topics have almost no relevant document. Therefore, in the case of using Japanese document sets, the total number of is only 47. (JAPANESE ONLY) Sep.13/2005 Information for writing your working note Please visit to http://research.nii.ac.jp/ntcir-ws5/ Old Announcements Sep./08/2005 *Results of evaluating runs on CJK documents will be delivered to participants on Sep./15. *Deadline of submitting working notes is Oct./15 (Except the groups submitting runs of English documents) Aug/30/2005 Delay of delivery of evaluation results We are so sorry but delivery of evaluation results is prolonged for some reasons. The date will be announced later. The deadline of submitting working notes is also changed. NOTE: Deadline of submitting search runs using English Document is August/15/2005 23:59 (Japanese time) *Additional submission of runs using CJK docs is also accepted by the same time. July/07/2005 Release of English document collection for evaluation If you have a plan to submit rersults searching English document collection in any sub-task, Please let us know. We will send information for downloading Xinhua dataset to you. You can obtain more infomation about English document sets for evaluation from the list of document sets. *The number of documents in Daily Yomiuri 2000 was revised at 19:00 on July 07 (Japanese Time). May/18/2005 Submission Guideline Release The guideline for submitting search results is now available. Please read it carefully. May/01/2005 Topic Release The participants can download topic files from the NII server (URL and password have already been assigned at the time of document release). Please see 5.2 Topics in CFP about the format, and read cafully 6.Types of Runs in CFP before executing your search runs. CAUTION: Please use 2000-01 years' doc files for evaluation. The 1998-99 years' files are just for training. DO NOT include any records of 1998-99 files into your search results that you will submit to us. Jan./20/2005 Error Notice - CIRB040
Aug./20/2004 Release of Call for Participation |
Call for Participation Note: Deadline of application is Sep./30/2004
Schedule
2004/09/30 | Deadline of Registrations for Participation |
2004/11/20 | Release of Data (Document sets) |
2005/05/01 | Distribution of Search Topics |
2005/06/01 | Submission of Search Results |
2005/09/13* | Delivery of Evaluation Results |
2005/10/15** | Deadline for Working Notes |
2005/12/06-09 | NTCIR Workshop 5 (Conference) |
Note:
* Only results of search runs using CJK document sets.
** Except groups submitting runs on English document sets.
If you have any questions, please contact to
@nii.ac.jp
All right reserved: National Institute of Informatics (NII) Japan