Mmastodon TechnologySoftware first seen 5 h ago, last 5 h ago, peak #4

Fact-checking LLM legal citations against Japan's official statute registry

Original: Checking every Japanese statute article an LLM cites against the official e-Gov registry, in Japanese and in English. #

A project is checking whether large language models invent Japanese law articles by verifying every cited statute against e-Gov, Japan's official government law database, in both Japanese and English. The effort taps into growing concern over AI hallucinations in legal contexts, where a fabricated statute or article number could have serious consequences. The bilingual approach also probes whether models are more reliable in English or Japanese when citing Japanese law.

Why now: Concern is growing over AI hallucinations, especially in high-stakes domains like law, and Japanese statutes are an under-tested case.

e-GovJapanese lawlarge language models

Open on mastodon →

Evidence

• Checking every Japanese statute article an LLM cites against the official e-Gov registry, in Japanese and in English. # llm # nlp # japanese # opensource # software # coding # development # engineering # inclusive # community Do LLMs Invent Japanese Law Articles? A Bilingual… · hackaday@www.urbanmind.net · 3

API: https://socialmediatrends-api.osmike.com/v1/trends/1250869