‘Godmother of AI’ Says California’s Well-Intentioned AI Bill Will Harm the US Ecosystem
Today, AI is more advanced than ever. But with great power comes great responsibility. Policymakers, along with those in civil society and industry, are working toward governance that minimizes potential harm and shapes a safe, human-centered society powered by AI. I applaud some of these efforts but caution against others; the California Advanced Artificial Intelligence Models Safety and Security Innovation Act, or SB-1047, falls into the latter category. This well-meaning piece of legislation will have significant unintended consequences, not just for California but for the entire country.
AI policy must encourage innovation, set appropriate constraints, and mitigate the impacts of those constraints. Policy that fails to do so will at best miss the mark and at worst lead to disastrous, if not unintended, consequences.
If passed into law, SB-1047 would harm our burgeoning AI ecosystem, especially the parts of it that are already disadvantaged compared to today’s tech giants: the public sector, academia, and “small tech.” SB-1047 would unnecessarily penalize developers, stifle our open source community, and hinder academic AI research, all while failing to address the very real problems it was created to solve.
First, SB-1047 would unfairly penalize developers and stifle innovation. In cases of misuse of AI models, SB-1047 would hold the responsible party accountable. And the original developers of that model. Every AI developer—especially novice programmers and entrepreneurs—cannot predict every possible use of their model. SB-1047 would force developers to back down and act defensively—exactly what we’re trying to avoid.
Second, SB-1047 would stifle open source development. SB-1047 requires all models above a certain threshold to include a “kill switch,” a mechanism by which the program can be turned off at any time. If developers are concerned that the programs they download and build will be deleted, they will be much more hesitant to write code and collaborate. This kill switch would devastate the open source community—the source of countless innovations, not just in AI, but across everything from GPS to MRI to the internet itself.
Third, SB-1047 would cripple public and academic AI research. Open source development is important in the private sector, but necessary to academia, where progress cannot be made without collaboration and access to model data. Take computer science students who study open-source weighted AI models. How can we train the next generation of AI leaders if our institutions do not have access to the right models and data? A switch would further reduce the efforts of these students and researchers, who are already at a data and computational disadvantage compared to Big Tech. SB-1047 would sound the death knell for academic AI when we should double public sector investment in AI.
Most alarmingly, the bill does not address the potential harms of AI advances, including bias and deepfakes. Instead, SB-1047 sets an arbitrary threshold that regulates models that use a certain amount of computing power or cost $100 million to train. Rather than providing protection, the measure simply restricts innovation in fields, including academia. Today, academic AI models fall below this threshold, but if we rebalanced private and public investment in AI, academia would fall under SB-1047. Our AI ecosystem would be worse off for it.
We must take the opposite approach. In many conversations with President Biden over the past year, I have expressed the need for “out of the box thinking” to advance our nation’s AI education, research, and development. However, SB-1047 is overly restrictive and arbitrary, not only chilling California’s AI ecosystem but also having troubling downstream impacts on AI across the country.
I am not against AI regulation. Laws are essential to the safe and effective advancement of AI. But AI policy must empower open source development, provide consistent and reasonable rules, and build consumer trust. SB-1047 falls short of those standards. I offer a plea for cooperation to Senator Scott Wiener, the bill’s author: Let’s work together to craft AI legislation that will truly build the human-centered, technology-enabled society of tomorrow. Indeed, the future of AI depends on it. The Golden State—a pioneer and home to the most robust AI ecosystem in our country—is the beating heart of the AI movement; as California grows, so does the rest of the country.
More must read comments published by Luck:
The opinions expressed in Fortune.com commentaries are solely those of the author and do not necessarily reflect the views and beliefs of Luck.
