Look, I came across a piece from CalMatters Housing that really caught my attention because it deals with something that affects how government agencies operate here in California, and ultimately impacts all of us. According to their reporting, the state announced it wasn't using any high-risk AI systems to make decisions about people's lives, but then discovered that state agencies have actually been using at least six automated systems to make significant decisions. We're talking about systems that determine whether someone gets cash assistance for food, qualifies for housing, or can access medical care. That's pretty serious stuff.
The root of the problem goes back to a law passed three years ago called Assembly Bill 302. This law was supposed to require the state's Department of Technology to keep a comprehensive inventory of all high-risk automated decision systems being used by state agencies and publish yearly reports. The idea sounds good on paper, right? But here's where it falls apart. When researchers made a public records request about the department's findings, they got back essentially nothing, just a spreadsheet with "no" written down for every agency.
What CalMatters uncovered is that the whole system depends on agencies basically policing themselves. There's no real verification process, no consequences if agencies don't report honestly, and worst of all, there's no clear agreement on what even counts as a "high-risk system" that needs to be reported. The law says these are systems that make important decisions about housing, education, employment, health care and other critical areas, but apparently some systems that have already caused real harm, like one that reduced In-Home Supportive Services for disabled Californians, weren't even reported as high-risk.
The researchers point out that this problem isn't unique to California. Other cities and states have tried similar transparency-focused approaches, like New York's oversight law for police surveillance technology, but these laws often fail because the agencies being regulated can find loopholes and use vague language to avoid real scrutiny. The deeper issue is that focusing only on transparency actually ends up normalizing the use of these automated systems in government rather than truly controlling them.
What I am seeing locally is that as technology becomes more embedded in how government makes decisions that affect housing access, benefits, and opportunity, we need stronger protections than just asking agencies to report what they're doing. The CalMatters piece makes a solid point that transparency alone isn't enough. For folks buying and selling property here in the Bay Area, and especially for those navigating government assistance programs that can impact their ability to qualify for housing, this matters because it means the systems making decisions about their lives might not be as accountable as they should be.
