TariQA 2027 · Moroccan Traffic Law QA Shared Task

01

Overview of the Shared Task

نظرة عامة على المهمة المشتركة

TariQA (طريق, tariq: road) is a shared task on answering questions about Moroccan traffic law, organized as part of the AbjadNLP 2027 Workshop on natural language processing for languages that use the Arabic script. Given a question about driving rules, road signs, right of way, penalties, licence points, vehicle documents or the Moroccan Highway Code (Code de la Route), a system must choose the single correct answer among three candidates.

Questions are written the way Moroccans actually ask them: in Modern Standard Arabic and Moroccan Darija (often with French loanwords such as embrayage or compteur). The task combines dialectal understanding, precise country-specific legal knowledge, and robustness to distractors that are deliberately close to the correct answer.

The task is open to everyone: academic groups, industry teams and individual researchers. Every team is invited to submit a system description paper to the AbjadNLP 2027 proceedings.

At a glance

Task: three-option multiple-choice question answering (output A, B or C).
Domain: Moroccan Highway Code and driving-school theory.
Languages: Modern Standard Arabic and Moroccan Darija, in one shared test set.
Metrics: accuracy (primary), F1 score (secondary), per-variety breakdown.
Submission: predictions on Codabench, papers on OpenReview.

02

Motivation

الدوافع

Road safety as a public concern

Access to accurate traffic-law information is relevant to road safety and driver education. Driving-test candidates and licensed drivers need to understand road rules, penalties, and administrative procedures. The proposed TariQA shared task aims to encourage research on question-answering systems that make this information accessible in Modern Standard Arabic and Moroccan Darija.

A domain requiring factual precision

Traffic-law questions depend on precise legal requirements, numerical thresholds, and exceptions. Fluent but incorrect answers can misrepresent these regulations. TariQA therefore proposes a shared evaluation setting for assessing whether language models select legally correct answers and handle distinctions that affect the application of traffic rules.

Limited resources and linguistic variation in Darija

Moroccan Darija remains underrepresented in NLP resources and benchmarks. Its spelling variation, dialect-specific expressions, and code-switching create challenges for connecting everyday questions with formal legal terminology. By including Modern Standard Arabic and Darija, the proposed shared task aims to investigate how these linguistic differences affect performance in a specialized domain.

Relevance to practical applications

Traffic-law question answering could support driving-test preparation tools, educational assistants, and public information services. TariQA aims to bring researchers together around a common dataset and evaluation protocol, enabling systematic comparison of approaches and analysis of their limitations. The resulting findings could inform the development of these applications and future research on legal question answering for Arabic varieties.

03

Data Collection and Creation

جمع البيانات وإنشاؤها

The TariQA dataset contains multiple-choice questions about Moroccan traffic law. Each question comes with three candidate answers (A, B, C) and exactly one gold label. Questions are drawn from two complementary sources:

Driving knowledge. Questions of the kind used in driving-school theory tests: road signs, right of way, overtaking, lights, speed, safe behaviour, vehicle documents and insurance.
The legal code. Questions grounded in specific articles of the Moroccan Highway Code and its implementing texts: classes of violations, fines, point deductions, prison terms, deadlines and administrative procedures. Article citations will be reported with the data release.

Distractors are plausible and lexically close to the correct option (same structure, different number, duration or authority).

TBA

MSA and Darija multiple-choice questions

3

answer options per question

MSADarija

language varieties

TBA

questions citing an article of the Moroccan Highway Code

-- %

baseline accuracy

Examples

Moroccan Darija

واش لازم توقف فعلامة إعطاء الأسبقية؟

Aنعم، الوقوف إجباري واخا تكون الطريق خاوية من اليمين واليسار.
Bلا، خاصك توقف غير إلا كانت العربات جاية من جهة اليمين بوحدها.
Cلا، إلا كانت الطريق خاوية. التوقف مطلوب غير مع STOP.

Legal code · MSA

تعتبر مخالفة من الدرجة الثانية إذا تم تجاوز السرعة القصوى المسموح بها بمدى يتراوح بين:

Aعشرة (10) إلى أقل من عشرين (20) كلم في الساعة
Bعشرين (20) إلى أقل من ثلاثين (30) كلم في الساعة
Cثلاثين (30) إلى أقل من أربعين (40) كلم في الساعة

Format and splits

Data is distributed as UTF-8 CSV files with one question per row. The answer column is withheld in the test set.

Split	Size	Labels	Release
Train	TBA (about 70%)	public	with the training data
Development	TBA (about 10%)	public	with the training data
Test	TBA (about 20%)	withheld	evaluation phase

Near-duplicate questions are grouped before splitting so that no test question has a paraphrase in the training data. Licence: TBA. A 15-question sample file shows the exact format.

Take the test جرّب بنفسك

Nine sample questions from the dataset. Pick an answer and watch the light.

MSA

04

Task Description

وصف المهمة

Input and output

Input question A B C

A question and its three candidate answers, in MSA or Darija.

Output answer ∈ {A, B, C}

Exactly one letter per question. Empty or invalid predictions count as wrong.

Rules in brief

Any permitted architecture or prompting strategy is allowed.
The test set must not be used for training, tuning or model selection, and must not be annotated by hand.
All teams enter the same single track.
Every resource used (models, corpora, tools, APIs) is reported in the system description paper.
Up to three official test runs per team. The last valid run is the official one.

Evaluation

Metric	Role	Notes
Accuracy	Primary, used for ranking	Share of test questions whose predicted letter equals the gold letter.
F1 score	Secondary, tie-breaker	Computed over the three labels A, B, C.

The official scorer will be released in the GitHub repository together with the training data, so that development scores are directly comparable with the leaderboard.

Baselines

System	Accuracy
Random choice	33.3%
Majority class (always B)	34.3%
Zero-shot open-weight LLM (organizers' baseline)	TBA
Fine-tuned encoder (organizers' baseline)	TBA

Baseline code and development-set scores will be published with the data release.

Submission format

Upload a zip archive to Codabench containing a single file named predictions.csv with one row per test question, in any order:

id,answer
TQA-00001,A
TQA-00002,C
TQA-00003,B

A sample submission and a validation script are available in the repository. Full instructions, including run limits and phases, are in the participation guidelines.

05

Tentative Timeline

الجدول الزمني المؤقت

All deadlines are 23:59 anywhere on Earth (UTC-12). Other dates are tentative; changes will be announced on the mailing list and Slack.

1

Date TBA
Registration opens
Register your team to receive the data-release announcement and Slack invitation.
2

Date TBA
Training and development data
Labelled train and dev splits, baselines and the official scorer. The Codabench development phase opens.
3

Date TBA
Test data release
Unlabelled test set. The evaluation phase opens on Codabench.
4

Date TBA
Test submissions close
Up to three runs per team. Only runs made inside the evaluation window count as official.
5

Date TBA
Results announced
Official leaderboard with per-variety breakdown.
6

Date TBA
System description papers due
Up to 4 pages plus unlimited references and appendices, through OpenReview.
7

Date TBA
Notification of acceptance
Reviews by fellow participants, checked by an organizer.
8

Date TBA
Camera-ready papers due
Final versions for the AbjadNLP 2027 proceedings.
9

9–14 May 2027
AbjadNLP 2027 Workshop
Presentation of the task overview and the participating systems. Macau.

06

Organizers' Details

اللجنة المنظمة

Loading organizers…

07

Participation Guidelines

شروط المشاركة

1

Register

Fill in the registration form with your team name, members and affiliations. You will receive the data-release announcement and the Slack invitation.

2

Build and submit

Download the data, develop your system, and upload predictions to Codabench: unlimited runs on the development set, up to three official runs on the test set.

3

Write it up

Describe your system in a 4-page paper through OpenReview, review two other submissions, and present your work at AbjadNLP 2027.

Full participation guidelines Paper submission guidelines Register now

08

Logistics and Support

الدعم والتواصل

Website and data

This site is hosted on GitHub Pages. Data, baselines, the scorer and the submission validator live in the official repository.

GitHub repository

Submission system

Predictions are evaluated on Codabench, which hosts the development and evaluation phases and the public leaderboard.

Codabench competition

Communication

Questions, announcements and support go through the Slack workspace, the mailing list and GitHub issues.

Slack invite Mailing list

Stay updated

Watch the GitHub repository for releases and join the Slack community for day-to-day discussion. For anything private, write to the organizers at .

Anti-harassment policy

TariQA and the AbjadNLP Workshop adopt the ACL Anti-Harassment Policy. Participants are encouraged to contact any organizer with concerns or questions, and to help us keep the task welcoming to everyone.

Announcements and updates

23 Oct 2026 Shared task website launched. Registration is open and the Slack workspace will be announced shortly.
Coming Training and development data, baselines and the official scorer, on 1 December 2026 (tentative).