Generative artificial intelligence tools have quickly become part of everyday legal practice. Lawyers are using these tools to draft briefs, summarize cases, and streamline research. But as recent cases make clear, those efficiencies come with a serious risk: sometimes, AI simply makes things up.
Courts across the country are now confronting filings with fabricated case law, inaccurate quotations, and non-existent citations. In one widely reported instance, a law firm submitted a brief with “pervasive inaccurate, misleading, and fabricated…representations of legal authority.” In another, a lawyer was sanctioned after submitting a filing containing hallucinated material and failing to verify its accuracy. And in yet another case, a lawyer included multiple made-up cases in a filing and later admitted he had not confirmed whether the authorities actually existed. The consequences can be severe. In a recent civil lawsuit, a federal judge sanctioned all four lawyers involved after determining that filings on both sides contained AI-generated hallucinated citations, canceled their trial, imposed monetary sanctions, and barred two attorneys from appearing before the U.S. District Court for the Northern District of Mississippi for two years.
These are not isolated incidents. They reflect a broader problem: AI-generated content can appear plausible, even persuasive, while being entirely wrong. And when that content is filed in court, the consequences are real.
Courts Are Losing Patience
Judges are increasingly treating AI hallucinations not as a novelty or a one-off mistake, but a recurring problem that “shows no sign of abating.” The message from courts is straightforward: lawyers are responsible for what they file, regardless of how it was generated. Whether a brief is drafted by a junior associate or an AI tool, the obligation remains the same: lawyers must ensure that the law cited is real and accurately represented.
This is not a new principle. Under Rule 3.3 of the Model Rules of Professional Conduct, lawyers have always owed a duty of candor to the court, which includes the obligation not to present “a false statement of fact or law to a tribunal.” The frequency of these errors, and the accompanying judicial response has prompted courts to directly address the use of AI and its implications. New York courts are now part of that broader regulatory response.
New York Courts Respond
In response, courts are beginning to formalize rules governing the use of AI in litigation. In the First Judicial Department (New York County Supreme Court, Civil Branch), a growing number of judges have adopted rules addressing AI use in court submissions.
While these rules vary in approach, they reflect a consistent goal of preventing the submission of inaccurate or fabricated legal authority.
Broadly, the rules fall into three categories.
Many of these rules go further, requiring attorneys to identify the AI program used and specify which portions of the submission were generated by it. Related rules emphasize that attorneys must independently verify that their filings contain no fabricated or fictitious cases, statutes, or other material. These rules reinforce a basic principle – one that predates AI: lawyers are responsible for ensuring the accuracy of their work.
Statewide Response
The trend toward regulating AI in litigation has now expanded beyond individual judges. Effective June 1, 2026, the New York State Unified Court System adopted Part 161, a statewide model rule governing the use of AI in court filings. While the rule does not prohibit AI use or require its disclosure, it reinforces attorneys’ existing obligations to review and verify AI-assisted work and ensuring that filings do not contain fabricated or fictitious authority.
What This Means for Practitioners
For practitioners, the takeaway is straightforward: AI can assist in drafting and research, but it cannot replace verification.
As a best practice, attorneys should (1) review the assigned judge’s individual rules to determine practices concerning use of AI; (2) independently verify every cited authority and quotation to ensure accuracy; and (3) consider maintaining a record of that verification if AI was used in drafting or research.
Lawyers remain responsible for every citation, quotation, and legal argument they submit to the court because when a filing contains hallucinated law, “Citation Not Found” is not a technical error – it’s a professional one.
Further contributions to this post by Samuel Bresnick.
