AI and the Law: Like Peanut Butter and Jelly, but the Jelly is Alive
Fractionally Legal v 58 Looks at Some of the New Developments in AI and the Law
Recently, I switched from OpenAI’s ChatGPT to Anthropic’s Claude as my chatbot-of-choice. I made the switch when OpenAI seemed to capitulate to the “Department of War’s” demands that they be able to use OpenAI’s artificial intelligence products for, potentially, autonomous targeting and to spy on Americans. Anthropic, on the other hand, took the principled approach of refusing, and is risking the wrath of Trump and his sycophants.
We all know how dangerous and creepy “domestic surveillance” (a.k.a. “Spying on Americans”) is, but the idea of an AI that can select human targets and activate human-killing machines without human intervention is downright terrifying. My understanding is that we’re not anywhere near that yet, but Anthropic is correct not to sell this technology to the U.S. government (or hopefully any government) or any corporation with the capacity to build those autonomous human-killing machines’ hardware.
I’m proud of my choice and support Anthropic. It has been said so many times but it’s always worth saying again: the apparatus of authoritarianism is not one big well-oiled machine — it’s millions of small acts of compliance and complicity, sometimes for profit, other times just to be “left alone.” Also, the stakes are pretty high here. If the Federal Government, or any government or company, actually wants to create an autonomous human-killing machine, none of us should be on the sidelines even if we’re absolutely sure those machines are only going to autonomously kill humans we don’t like. It never works out that way.
I probably never would have tried Claude but for OpenAI’s obsequiousness . For my needs, which are mainly reading long documents, statutes, cases, and drafting clauses based on existing templates, with a smattering of demand letters and some light “lawyer math” on cap tables and the like, ChatGPT and Claude are about the same, although based on one or two bot-to-bot competitions, Claude actually did a little better.
I’ve said this a few times before, but AI and the law are like peanut butter and jelly: they just go well together. That’s because the legal product is words that form clauses that form contracts, and words that form arguments. The weird problem, ironically, is that Claude is much wordier than ChatGPT. That is probably by design: Claude is mimicking what it thinks I want, which is long wordy narratives based on the templates (”training data”) I’ve provided. It was also trained on tons of legal documents which, in my experience, are often much wordier than they need to be. That’s a little annoying because I’m trying to use these chatbots to provide faster, cheaper legal services — getting lengthy responses, which I then need to digest using the AI products, sort of defeats the purpose. I actually hear this critique from a few actual legal practitioners: AI saves time, but also increases the amount of nonsense that lawyers need to wade through, or use AI to wade through, which reduces the efficiency. But it’s still a lot quicker to draft an agreement or a brief with AI than without.
OpenAI’s capitulation, and Anthropic’s refusal to do so, raise some other serious issues. First, Anthropic’s principled stance actually hastens what many observers already think is coming. Either the government owns or has access to the best AI products or it (1) builds its own, (2) buys an AI company, or simply takes the technology when it can’t buy it way in. Is that coming? Possibly.
Granted, the current U.S. government seems incapable of developing its own AI: Even if they could match the compensation provided to the top AI researchers, who would ever want to work for an administration this mendacious and capricious? Perhaps in the future we will get leadership actually capable of developing a huge, new AI program similar to the Manhattan Project. But I’m not holding my breath.
The second approach is more legally interesting: nationalization of the private companies building AI. There are two approaches here. First, the U.S. could make a deal with the owners of OpenAI or Anthropic (or both) to buy all the shares for hundreds of billions of dollars. One or the other of those companies would more than likely sell under investor pressure (it would likely be a breach of fiduciary duty for them to refuse so much money on political grounds). But that, again, runs into the same problem as the government developing its own AI. Most of the talent might just leave if the company is wholly owned by a government to a company which is not wholly owned by the government. There are other variations on this, like the government buying a small part of the company with outsized control rights. That is the model that was used in the government’s deal with Intel, where it now owns 10% of the company. It’s unclear if the government got control there. In contrast, there is the deal with U.S. Steel (now owned by Nippon Steel) where the government did not get any equity, but got a Chinese-style “golden share” which gives it control over some company decisions (like closing steel plants, I assume).
U.S. Steel and Intel were in trouble, and their boards were pretty eager to take the deal. That’s not true of any AI company worth buying: these AI companies are thriving, and the ones that are not thriving are really not worth controlling (xAI’s Grok, anyone?).
Which is to say that the U.S. government is probably not buying an AI company, or investing in one, any time soon. The solution then is more radical: nationalization. In other words, the government just takes the company without compensating the shareholders. We see that all the time all over the world, but rarely in the U.S.
In the U.S., nationalization has some legal issues. The last president to try it, Truman, got stopped by the Supreme Court. See Youngstown Sheet & Tube Co. v. Sawyer. Youngstown actually limits presidential power when there is no explicit authorization from Congress, which was necessary to seize the steel mills because nothing in the Constitution gives the President the power to seize private property even in the time of war. Congress, on the other hand, likely has that power under the Fifth Amendment’s Takings Clause, provided Congress gives “just compensation.” So in a situation where the Federal government wants to take an AI company (and Congress goes along with it), it can. It just needs to come up with the hundreds of billions of dollars needed to compensate the shareholders of the A.I. company. Despite all the handwringing, nationalization of AI is possible, but unlikely. Also, one might doubt its effectiveness. While steel production has a physical site, and you can’t easily move steel across the world, that is not true for the bits that make up AI. So even if the government owned the means of production in the U.S., there could be lots of A.I. in the wild, beyond the reach of the U.S. government but impacting America none-the-less.
What’s pretty amazing about the current debate is that no one in Trump’s government is even thinking about any of these options (at least not in a coherent and public-facing manner). There are actually good reasons for the Federal Government to be talking about how powerful A.I. is and that if the companies won’t agree to common-sense regulation or work with the government (on something other than autonomous human killing machines, one hopes), then more radical solutions are needed. But that is not the debate at the moment. Instead, in a fit of political retribution for showing spine, Trump’s people labeled Anthropic a “supply chain risk” and were busy ripping Anthropic’s products out of the Federal government. That of course matches the Trump modus operandi. Instead of interesting solutions, it’s just demanding loyalty and being vindictive, even where it just hurts himself (and the American people, by extension). And now, cue the lawyers, we’re in a lawsuit with the really fun name of “Anthropic v. U.S. Department of War.” That is what passes for A.I. regulations these days.
And we are in desperate need of regulation. Putting aside the human-killing machine problem from above, we’re getting a flurry of cases where A.I. has caused real-world harm. There are cases where the AI is alleged to have encouraged people to kill themselves, for example. Those lawsuits are tragic, but they are personal injury cases and, while certainly the negligence of the product design (whether the chatbot was negligently designed) is an issue, the more difficult questions are those of causation: did the AI cause the harm, or was this person already going to hurt themselves? Same idea with the social media addiction cases, currently on trial in Los Angeles where a verdict is due imminently. The question in those cases is one of design, but the companies are more likely to limit their liability on the issue of causation. Having done some personal injury work in the past (I rarely do it anymore), issues of personal responsibility are catnip for an American jury. We love to blame the victim, and lawyers love to play to those arguments.
No, these are more interesting because they involve the AI chatbots being used as advertised and causing injury to a third party. That’s not a tort theory as much as a product design issue — the person using the chatbot got exactly what they wanted, and the AI company advertised what it could do, but it harmed someone. The most interesting from a legal perspective is Nippon Life Insurance Company of America v. OpenAI. It’s the first case I’ve seen with a viable product design defect theory. OpenAI is very proud that ChatGPT can pass the bar and made sure everyone knew it. They basically said that their chatbot knows as much about the law as your lawyer (that might true sometimes, but not most of the time). So a plaintiff decided to fire her lawyers and challenge a settlement she had already signed, generating legally dubious arguments and dozens of filings the complaint says served “no legitimate legal or procedural purpose.” The user wasn’t harmed — the insurer who had to deal with all that paper was — and that distinction matters. A product used as intended that injures a third party is a design defect claim. Certainly it was not the insurer/plaintiff’s fault — they thought they had settled the case, only to be confronted with AI that told the plaintiff they had the chance to undo the settlement. Oops.
The lesson has implications well beyond the law. The through-line of this newsletter — from the autonomous weapons debate, to nationalization, to Nippon Life — is the same: AI companies are building extraordinarily powerful systems with remarkably little thought about what happens when those systems interact with governments, courts, or third parties who never consented to be part of the experiment. Anthropic deserves credit for thinking seriously about at least one of those problems. The rest of the industry, and frankly the government, has not caught up.
Regulation may come — I hope we don’t repeat the mistake of social media which we let grow unchecked. A.I. is different. The downside, even about 3 years into the A.I. era, are too large to ignore. The question is whether it arrives through thoughtful legislation, through the slow accumulation of civil verdicts that make certain design choices too expensive to defend, or through something more blunt, like the nationalization scenarios I’ve described above. Right now, we’re getting mostly the middle option: courts filling the vacuum that legislatures have left. That is not a terrible outcome for lawyers. It is a terrible outcome for everyone else.
Keep thinking, keep building,
Jesse
Hi, and welcome to my newsletter! I’m Jesse Strauss, Your Fractional General Counsel. I’m a lawyer with a private practice based in New York City, helping clients in the United States and globally with their U.S. legal needs. My expertise spans various areas, including raising funding rounds, employment issues, negotiating master service agreements, intellectual property, compliance, legal process management, and dispute resolution. My focus is on founding and nurturing great companies from seed to exit. Discover more at Your Fractional GC and book a complimentary 30-minute consultation. You can also follow me on Threads @lawyerjesse1977, on BlueSky @lawyerjesse.bsky.social, subscribe to my Substack here (follow me on notes), and follow me on LinkedIn here.