An LLM-Based Agent for Legal Compliance Assessment: Architecture, Memory, and Evaluation in a Canadian and Quebec Context

Key points

• Checking whether a text complies with a body of law is slow when done by hand and fragile when done with keyword rules.1
• The agent relies on a large language model that scores a text on a fixed rubric, grounds its judgment in a reference legal corpus, and, when the score falls at or below a critical threshold, runs a corrective chain made of self-healing rewrites, a compliance label, an explainability report, and escalation to a human reviewer.1
• We describe what the language model does at each step, how the memory of the agent is organized, and how the same implementation can be pointed at different reference corpora.1
• We evaluate the agent in a Canadian and Quebec context, on a corpus of thirteen legal instruments that brings together federal and provincial texts, international treaties, and municipal documents of the City of Trois-Rivières.1

Sources (1)

• [1]An LLM-Based Agent for Legal Compliance Assessment: Architecture, Memory, and Evaluation in a Canadian and Quebec Context
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 10:24 PM

  This paper presents ComplianceAgent, the legal compliance sub-agent of COMPASS, a multi-agent framework that evaluates texts along four axes: digital sovereignty, carbon impact, compliance, and ethics.

  Checking whether a text complies with a body of law is slow when done by hand and fragile when done with keyword rules.

Extractive summary: sentences quoted from the sources.