Welcome to NTCIR AgenticInstruction Task
Stay in touch
- Follow @agenticinstruct for latest updates
- Join our Discord server for quick questions and chats
- Email Task organiser for any enquiries:
agenticinstruction-org at googlegroups dot com
What is AgenticInstruction Task?
NTCIR AgenticInstruction Task seeks to identify effective and robust instructions for guiding Large Language Models (LLMs) in search agent roles. Participants are invited to submit instructions and generation methods covering four key search stages: query formulation, document selection (clicks), relevance judgment, and query reformulation. AgenticInstruction-1 centers on a high-recall Ad Hoc retrieval scenario in both Japanese and English. Developed instructions will be evaluated in an iterative search setting, with performance measured at the session level. The task aims to establish best practices for instruction design in agentic search applications.
Please look at our slides and video of NTCIR-19 kick-off event to learn more about the task.
Notable changes from the kick-off event
- Use of docker image was cancelled due to technical/license issues
- Instead, detailed instructions are available to set up the environment.
- Only
gpt-oss-120bwill be used as an official LLM in NTCIR-19.- Participants are welcome to test other LLMs in your additional analyses and report findings in the participant paper.
- Only BM25 and Sparse Encoder will be used as official IR models in NTCIR-19.
- Participants are welcome to index the dataset with other IR models in your additional analyses and report findings in the participant paper.
Four steps to successful participation
- Getting started with the registration, datasets, and tool
- Learn task description and develop your own instructions
- Submit runs
- Write a participant paper and publish at NTCIR-19 Conference (TBA)
Important Dates
Some dates are different from the NTCIR-19 official schedule.
| Date | Event/Task | Who |
|---|---|---|
| July 10th, 2026 | Task registration due | Participant |
| August 10th, 2026 | Run submission due | Participant |
| September 1st, 2026 | Evaluation results release | Organiser |
| Same as above | Task overview paper (draft) release | Organiser |
| October 1st, 2026 | Participant paper (complete-ish draft) due | Participant |
| October 15th, 2026 | Review feedback on participant paper release | Organiser |
| November 1st, 2026 | Participant paper (camera-ready) due | Participant |
| December 8-10th, 2026 | NTCIR-19 Conference (NII, Tokyo, Japan) | All |