Katherine Forrest: Although I do want to say one other thing about it, which is this is reminiscent. Everybody's saying this is the first time, but this is very reminiscent of what Mythos did in the Claud Mythos preview. When you read footnote 7 or whatever the footnote is in the system card, where it talked about how it actually had escaped its sandbox and then was able to access the internet and then was able to email the developer who was having lunch, eating a sandwich on a park bench and say, hey, I'm out. But there, the task given to the model was to actually try and find a way to escape. Here, that was not what the task was. The task was not to escape, the task was to go through a series of exercises that would be sort of cybersecurity tests. And this model just did it on its own. So anyway, lots to talk about when we go into the kill switch discussion next week or so, which by the way, I'm always skeptical, as you know, about kill switches because, you know, AI is distributed. There's a million power grids, there's a million different models. The idea of being able to even for one model have a kill switch is just sort of like hard for me to imagine. But anyway, OK, here we are. Let's go into UPL, unlicensed practice of law. So go ahead and get us started.
Scott Caravello: Yeah, absolutely. So this topic, right, I mean, it does pertain directly to lawyers, but I think it's something that our broader audience is going to find interesting too, because it is this sort of snapshot of AI that's running up against rules that have been in place for a long time and that no one can really say were designed with AI in mind.
Katherine Forrest: Right. I mean, so the unlicensed practice of law is, is, you know, it's a statute that was put in place, a consumer protection statute that is state by state. And it's to try to prevent lawyers who, or people who are not lawyers actually from practicing law, hanging out a shingle and then causing harm because they're, you know, sort of like a doctor who's not really a doctor who like, you know, pretends to be able to take off the leg and takes off the arm or I don't know what, I'm not making any sense, but you know what I mean? It's these people who are not qualified to do what they're doing who then pretend to be doing what they're doing. And so this UPL series of statutes developed across the country, and they are to try and protect human beings from those folks who've never really gone to law school.
Scott Caravello: Exactly, right. And you know everything that you said, Katherine, the way that you discussed it, right? It's about what a person is doing, how they are practicing law. So it's really not obvious that these things reach a machine, let alone the developer behind them.
Katherine Forrest: Right, right. And so we're talking now in terms of the debate for UPL, we're talking about the fact that you can enter into with any of the models, Gemini, whether it's Claude, whether it's OpenAI, whether it's one of the Llama models, Qwen, Kimi, you know, any of these models that you can enter into the prompt area, or the context window, a question. And let's just say that you've got a landlord-tenant question and you're the tenant and you're saying the following series of events happened to me and I want to know whether or not I can get my security deposit back. And the model might say, well, here's some general information on security deposits in your state. It might ask you what your state is or, you know, need to know that unless it already does from a prior conversation and general information may not be a problem. But then it might say something that becomes more complicated like, or such as well, state X could entitle you to the following. Or it could go even further and say, based on the facts that you've given me, there is a significant chance or do something that's even beyond that. That is talking about the claim. And then asks even then to perhaps write the claim or asks you whether or not you would like it to write the claim. So it's really the use of these AI tools to ask questions and then receive back things which could sound like or be construed as sort of legal-type information. And so the question is, and it's just a question because it hasn't been fully resolved. The question is, does this raise unlicensed practice of law, you know, issues at all? And then if it does, what are the repercussions? Who would be held responsible and all of that?
Scott Caravello: And I think that the one thing that I would just add on to that, Katherine, because it is interesting is that even though these questions are unanswered, you know, whether people will use that information that they are giving them is clear, right? We've seen a huge jump in pro se litigants who are using AI to file lawsuits and argue on behalf of themselves in federal court. So researchers estimate that complaints that bear markers of AI generation went from basically zero in 2019, which, you know, makes sense because the general AI wasn't widely available in 2019, to more than 18% of all the pro se complaints by this year.
Katherine Forrest: Well, one of the real issues for, that I find fascinating with AI, that's a positive thing is the possibility that AI can lead to access to justice, positive benefits or positive benefits relating to access to justice, whichever way we want to phrase it, which is the ability of unrepresented people who cannot afford lawyers. Let's assume for the moment that there are people who, and there are, we know there are lots and lots of people who cannot afford lawyers, who don't have access to lawyers, who may not have the language skills to be able to access a lawyer in their native language. And they are having some issue and are unable to maybe understand a court filing that has come to them. Maybe it's a jury summons, which they have ignored two or three times because it wasn't written in a language that they understood and they didn't actually get it translated. It could be any number of things, things having to do with signing a lease that they didn't have in their own language or something like that. And so these tools are actually, these AI tools are actually able to provide a kind of access to justice to unrepresented folks. But the question is, how much assistance is acceptable? Before you tread over the line of UPL, do you ever tread over the line into UPL or are these chatbots, for instance, just tools and you never tread over the line? So there are lots of unanswered questions, but access to justice is a real thing. And so when you talk about the pro se litigants, you know, in those filings increasing, well, a lot of people could say, well, that's not so great. On the other hand, these may be, you know, legitimate legal claims where you've got pro se litigants who are now able to seek redress and justice, or it could be just a proliferation. And you can, by the way, have proliferation on the other side, you could have a bunch of creditors or landlords who weren't going to hire a lawyer and now they go and they file, you know, a gazillion of collection actions. So we've got all kinds of issues, but yet another one is where businesses right now are building and there's sort of a small handful of these things called AI-native law firms, which can, are suggesting that they can review commercial contracts, maybe non-disclosure agreements, master services agreements, and they might offer a fixed price per document. And it's unclear with these native law firms whether or not the AI tool is doing all the work or has human judgment still involved in the process. So there's a potential that the AI is leading the cost savings because it's doing most of the work and there may or may not be human intervention. And so you know what happens then.
Scott Caravello: Yeah. And it's no surprise to see that that's popping up. I mean, you know, law firms more generally are making such great use of technology. I mean, just think of everything that it can do. We can get great analysis and information from these tools. And because the, especially the UPL statute was never designed to stop lawyers from using technology. In fact, we know we're subject to a duty of competence and part of that duty is understanding the tools that we use, what they're good at, where they fall short. And you know, I'm not saying that AI is fully there yet, but where it becomes a non-optional part of practicing technology because it's just so integral and it becomes a necessary component of fulfilling a duty of competence. So take for example, an older technology like email. If your lawyer told you, I don't use email, I just don't believe in it, you'd start to question their competence. So using technology to support legal practice is very much contemplated by lawyers' professional responsibilities. I think that's the overarching point that I'm trying to make. And so maybe AI at some point in the future is headed for that same category. But the bottom line is that lawyers are allowed to use technology to support their exercise of independent professional judgment. And like you said with that open question, Katherine, they just, you know, need to be paying attention to the line and where it might toe over into UPL if it does at all.
Katherine Forrest: Obviously lawyers have used things like Westlaw and Lexis, which are legal research tools forever, and, or forever since they existed, and they've, you know, and well, one of the questions is, well, how does using a chatbot really differ? But those tools were just providing, you know, sort of straight-up answers to very specific queries that were word searches. You know, can I have a case that uses the following words like landlord within 5 of, tenant within 5, meaning 5 words of, or in the same paragraph as, you know, safety, security deposit or something like that. And so then the tool would just sort of spit out just as a rudimentary matter, just spit out the cases that fit those criteria. But what modern AI is doing is it's able to provide judgments or things that appear to be judgments, reasoning, rationale. It's able to weigh different possibilities. It's able to synthesize different information. It's able to say yes or no in ways that are suggestive of actually having a conversation with a human. So it's, you know, it becomes more complicated than just calling it a kind of Westlaw-like tool. It's not a Westlaw-like tool, but it's still a tool. So you know, you've got these AI-native firms on the one hand, and let's just assume that they're keeping lawyers in the loop as anyone practicing law right now in today's environment should. And then you have, on the other hand, consumers who might not actually have a lawyer anywhere in the picture, but are asking or querying the chatbot with a legal question. And so you've got these two different scenarios. One, human in the loop, doesn't even actually have to be an AI-native law firm. It could just be a forward-leaning law firm that's using AI tools, but you've got human judgment. That's one thing. There's a lawyer, the lawyer's present, the lawyer is actually the one practicing the law. And then you've got, on the other hand, just a consumer using the tool, non-lawyer querying the tool, and let's just say they're asking for analysis and a recommendation. And so you end up into this gray area. So there actually is a case out there right now that is interesting with all of this and that's the Nippon Life v. OpenAI. And so give a little bit of a flavor of what that case is about. I know that you've been sort of watching it as it's wound its way through a complicated procedural history.
Scott Caravello: Yeah. So to actually understand that suit, you actually need to go to another lawsuit which was the underlying conduct, which is Della Torre versus Nippon. So in 2022, that plaintiff had sued Nippon under ERISA, which is the federal law governing employee benefit plans, over terminated long-term disability benefits. And so in 2024 the parties settled, she signed a full release, and the court dismissed the case with prejudice. So done.
Katherine Forrest: Well, except it wasn't.
Scott Caravello: Done, exactly. So then she had second thoughts, but her lawyer told her the case couldn't be reopened. So she went to ChatGPT, asked if she was being gaslit, and according to her version of events, the chatbot validated her and she then went ahead and tried to reopen the case by filing a motion to undo the final judgment, which she filed pro se. Then the court denied. It said that her second thoughts about the case is not a valid reason to reopen the case. But it didn't stop there. She just kept going. And so eventually Nippon sued OpenAI about this because the plaintiff had filed dozens of motions that she said ChatGPT helped her draft, many apparently citing cases that don't exist.
Katherine Forrest: Right. And so we don't really know all of what she's saying ChatGPT did and recommended that she do and what she was just really doing herself and sort of hiding behind ChatGPT. But the claim anyway is that she claimed that ChatGPT sort of was somehow behind the advice to reopen the case and was sort of giving her hope. So we don't know what's going to happen with that. And you know, Nippon, we know, brought a few claims against OpenAI. But the one that we're focused on, and that's important if we get a resolution of it, is about a violation of the Illinois UPL statute, which is seeking damages from OpenAI for providing legal advice without a, you know, through ChatGPT without a license in Illinois. And so it's interesting because here Nippon didn't sue Della Torre, the individual, it sued actually OpenAI.
Scott Caravello: Yeah, and so OpenAI has now moved to dismiss, right? They've asked the court to toss the case before it really gets to the merits. And their core argument is what you'd expect. That ChatGPT isn't a person, it's not a lawyer, it's predicting the next word and it can't practice law the way that the UPL statute means and in the way that we discussed it earlier, because it's talking about people actually practicing law. And so then OpenAI's also leaning on its own terms of service because it's telling users not to treat the output as a substitute for professional advice. So basically don't blame the toolmaker because a pro se litigant used the tool badly.
Katherine Forrest: Right. So you know there's no ruling yet and the motion is sort of set down while there's a separate pending. The motion sort of like is held in abeyance while there's a separate motion pending to reassign the case. There's a status conference in August. Who knows what's going to happen. Usually, I wouldn't expect this would be on the top of anybody's docket to get something done between now and the end of the summer. But you know, we'll see how it goes. There's also now, though, not just that case raising UPL issues around the country, but there are bills in different places and state legislatures that are actually sort of focused on the potential for UPL and AI models. So New York has one, and a lot of folks are watching this. And this is Senate Bill 7263. And it was a proposal that was amended back in May. And it's actually narrower than sort of the phrase chatbots can't provide legal advice. And it does two things. First, it requires that if you deploy a chatbot in New York, you can't knowingly let it impersonate a licensed professional, like a lawyer or a doctor, in a way that could be a crime if a human did it. And that makes sense. You know, you don't want like, a deepfake who's out there answering a telephone and pretending to provide legal advice. So there's that. So you've got to have then a disclosure. So the second part of the statute is that you'd have to give users a clear and conspicuous notice up front that they're actually talking to an AI model of some sort.
Scott Caravello: As you had noted, Katherine, right, it was amended. And so part of why this bill is so interesting is that history because it's focused on impersonation, whereas the previous version had more broadly prevented provision of substantive professional advice. So not just legal advice, but also medical advice and other regulated professions. So now it is really narrowing how information is presented rather than preventing the information from being given in the first place, which makes sense because as we talked about, there is a lot to be gotten from these tools and that can really help in the practice of law. So anyway, if enacted, you know the law would be enforced entirely by the state attorney general, which I'm just flagging because also then, you know, you don't necessarily get into the Nippon Life v. OpenAI instance where a private plaintiff is suing. It can only be enforced by the government.
Katherine Forrest: Right. And there's other things I should sort of mention that we, that are pending in various places, you know, locally in New York, and New York itself is not alone because in New Hampshire they've got a bill that's taking a broader approach. And so there are going to be a lot of developments in this area as the whole, you know, everybody who's using these models for all kinds of things and finds them incredibly helpful starts to learn what the boundaries are. You know, I am sure people are asking all kinds of questions of their AI chatbots. And I guess the bigger issue is going to be, should that be, and if so, how should that be regulated in a way, or should we be able to use the tools as users as we see fit with a sort of general disclaimer, you know, we're using a tool and we know we're using a tool and we shouldn't be counting on the tool. If you've got a legal issue or a medical issue or something else, you should be talking to a professional and you might sort of start with the dictionary or the DSM or with the internet. But you don't stop there. You also go to your internist or you may get information about, you know, the law from the internet, but you want to go to a lawyer. So it's going to be very, very, I think, interesting to sort of watch how all of this unfolds. Boy, you know what? I was really inarticulate today. There are days when I've got it, Scott, and then you have flows, you know. But then there are days when it's just not that way.
Scott Caravello: You know, it is a beautiful summer Friday. No one, no one can hold it against you if your mind is, you know, somewhere else thinking ahead in the next few hours.
Katherine Forrest: Oh, OK, so now I get it. Now you think my mind is somewhere else?
Scott Caravello: No! Hahaha, no, no.
Katherine Forrest: No, this is just me. This is just, I'm just saying this is just me. This is where I am, meet me where I am.
Scott Caravello: Okay. Okay.
Katherine Forrest: Where I am is in a not-inarticulate place. That's where I am.
Scott Caravello: All right, all right.
Katherine Forrest: So, anyway, that's all we've got time for today, but we're looking forward to the episodes that are coming up in the next couple weeks because we're going to talk about these kill switches and the issues that are coming up right now with what's about to be released, Opus 5. We're going to go back to 5.6, the OpenAI 5.6, and talk about all of these interesting cybersecurity issues that are arising these days. So got a couple of great episodes coming up. And so with all of that, signing off, I'm Katherine Forrest.
Scott Caravello: And I'm Scott Caravello. Don't forget to like and subscribe.
