Legal Guidance in the Wild: How Users Seek Legal Help in Real-World LLM Conversations

Key points

• To study this behavior, we first need to distinguish requests for legal guidance from other legally themed discussions and unrelated conversations.1
• We contribute a multi-axis taxonomy of legal queries capturing whether legal guidance is sought, how a request is formulated, and which legal domains it concerns; apply it to 2,122 naturally occurring WildChat-1M conversations; and create a human-reviewed gold evaluation set of 200 conversations.1
• Rights-and-options advice is the most common primary request type among legal-content conversations, and 25.3% of guidance-positive conversations span multiple legal topics.1
• Our framework supports large-scale studies of legal guidance in conversational AI, and the public release of the annotated data facilitates reproducible research and future model development.1

Sources (2)

• [1]Legal Guidance in the Wild: How Users Seek Legal Help in Real-World LLM Conversations
  Hugging Face Daily Papers · Oct 18, 12:00 AM

  People ask conversational AI models about their legal rights, obligations, procedures, and personal situations that may have legal consequences.

  To study this behavior, we first need to distinguish requests for legal guidance from other legally themed discussions and unrelated conversations.

• [2]Legal Guidance in the Wild: How Users Seek Legal Help in Real-World LLM Conversations
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 10:31 AM · same content

Extractive summary: sentences quoted from the sources.