Call for Participation

We introduce a new shared task for challenge, aiming to benchmark semi-supervised and reinforced task-oriented dialog systems, built for automated customer-service for mobile operators. The task consists of two tracks:

Information extraction from dialog transcripts (Track 1)

Task-oriented dialog systems (Track 2)

An important feature for this shared task is that we release around 100K dialogs (in Chinese), which come from real-world dialog transcripts between real users and customer-service staffs from China Mobile, with privacy information anonymized. We call this dataset as MobileCS (mobile customer-service) dialog dataset, which differs from existing TOD datasets in both size and nature significantly.

To the best of our knowledge, MobileCS is not only the largest publicly available multi-domain TOD dataset, but also consists of real-life human-to-human data (namely collected in real-world scenarios). For comparison, the widely used MultiWOZ dataset consists of 10K dialogs and is in fact simulated data (namely collected in a Wizard-of-Oz simulated game).

A schema is provided, based on which 10K dialogs are labeled by crowdsourcing. The remaining 90K dialogs are unlabeled. The teams are required to use this mix of labeled and unlabeled data to train information extraction models (Track 1), which could provide a knowledge base for Track 2, and train TOD systems (Track 2), which could work as customer-service bots. We put aside 1,000 dialogs as evaluation data. The teams can choose to participate in Track 1 or Track 2, or both.

For each track, three teams with top performances will be recognized with prizes RMB20,000, RMB15,000 and RBM10,000, respectively. The prizes will be awarded at the Workshop.

Challenge Description:
v2.1 (Sep. 10, 2022, mainly adding Combined Score for Track 2)
v2.0 (July 1, 2022)

Code repository for baseline systems and evaluation scripts:
https://github.com/SereTOD/SereTOD2022

Challenge Registration:
Please fill the form (English form, Chinese form) and send to seretod2022 (at) gmail (dot) com to complete the registration. Registration deadline: July 1, 2022.

You are welcome to scan the following QR code to join the WeChat group (in Chinese), where we will distribute messages relevant to the Challenge.