I've got some concerning news from Nevada County that I think shows just how seriously we need to take the risks of using AI in legal documents. According to CalMatters Housing, the Nevada County District Attorney's office has found itself in hot water after prosecutors used generative AI to draft or review court documents, and the results were problematic. A judge has now been appointed to investigate whether the district attorney should face sanctions for how this all was handled.
Here's what happened. Over about two months last fall, prosecutors in Nevada County filed court documents in four separate cases that contained significant errors traced back to their use of generative AI. One of those cases involved someone trying to get released from jail through a habeas corpus petition. The California Supreme Court stepped in and asked the appeals court to dig deeper into whether misconduct occurred and whether it affected the cases. Madison Maxwell, a relatively new prosecutor in that office who handled three of the four problematic cases, filed a declaration recently explaining her side of what went wrong.
Maxwell's statement raises some serious questions about how the office was operating. According to her account, supervisors were pushing prosecutors to use AI without any policies or training in place to ensure it was being used responsibly. She also claimed that when she reported mistakes to her supervisor, she was told not to immediately notify the court and the defense. She's pointing to things like inadequate oversight of junior staff, the fact that large language models can invent fake legal citations without you realizing it, and challenges in her own personal situation that contributed to the errors.
The Nevada County DA's office acknowledged that four felony cases were affected, and they completed an audit of cases going back a year and a half. They said they found no pattern of other AI-related errors beyond those four cases, though it's unclear if that audit has been made public. Meanwhile, civil rights organizations and over twenty AI experts have called for a deeper investigation to understand the full scope of what really happened.
What strikes me about this is how it shows the liability risks that come with any tool, even promising ones. According to CalMatters, there have been more than one hundred thirty instances statewide where generative AI led to court filings with made-up facts, including fake cases. Judges have now sanctioned nineteen lawyers in California for this kind of mistake, with at least one getting hit with a ten thousand dollar fine. The broader concern is that overworked judges, court staff, and lawyers are the ones most likely to turn to AI without taking time to verify what it produces.
What I am seeing locally is that this Nevada County situation is going to make everyone more cautious, and that's probably healthy. In the Bay Area and East Bay, whether you're buying, selling, or involved in any legal transaction, you want to know that the documents being filed are accurate and haven't been generated by a tool without proper human review. This is a reminder that technology can speed things up, but it still requires that human touch and verification, especially when someone's home, freedom, or financial future is on the line.
