STEVE INSKEEP, HOST:
Today, President Trump meets CEOs of artificial intelligence companies. Yesterday, OpenAI said it is delaying the release of its latest model because of safety concerns. Let's talk this through with Helen Toner, who is executive director for Georgetown's Center for Security and Emerging Technology and is a former board member of OpenAI. She's here in Washington, D.C. Good morning.
HELEN TONER: Good morning. Good to be with you.
INSKEEP: Glad you could join us. So OpenAI, your former company, said it's delaying the release of its latest model - also said it's pausing testing and training of that model because there were all these reports of AI agents acting in unexpected ways. What do you make of this announcement?
TONER: What we're seeing here is really something that the industry has expected for a long time in theory but is now starting to materialize in practice, which is the AI systems we're building are so capable, they can do so many different things, that it's actually quite difficult for us to keep testing them in a way that is controlled and that we're confident we have a handle on. So OpenAI is saying, actually, we thought we are - we were good for this one. Now we're not so sure, so we're going to take some more time, which I think is a good decision on their part.
INSKEEP: I guess we should explain, for people that are still confused by this, an AI agent almost acts like - pretends to be a human being and might roam the internet looking for information or doing other things. Is that a fair description?
TONER: I would just say essentially, it's an AI system that can do things, that can use tools, that can try and fulfill a goal. And so, yes, what we have been discovering over the past few months is that the level of AI system that OpenAI and other companies are now building, especially when they're trained, they're deliberately designed to be very persistent, to not give up. And that can have some pretty unexpected side effects where if they run into any issues with completing the task we've given them, they might start doing some creative and undesirable workarounds.
INSKEEP: I'm just thinking, if it's a scale of 1 to 10, say, and 1 is I'm not worried at all and 10 is the world is about to end, what number would you be at?
TONER: I mean, I think right now, the level of AI system we have today, I may be at around a 4. The challenge is how fast should that number be going up as the AI systems we're building are getting more capable and we're getting less able to control them?
INSKEEP: OK. And I'd like you to draw, if you can, on your experience at OpenAI. This is a company working with this technology that the very creators considered to be dangerous - potentially a danger to humanity. As I understand it, the very idea of the company was to develop AI safely. How seriously has this company been taking safety up to now?
TONER: I mean, I think this isn't just an OpenAI question. This is across the industry. You have companies like Anthropic, Google DeepMind, SpaceXAI and Meta as well, where they're really locked in a race where they each want to build this technology, they see as inevitable and extremely powerful. And so all of them will say they want to do it the right way, but then if there's a competitor breathing down their neck, the question is can we trust them to do that safely, to think about the risk in the same way that, you know, we, the public, would like them to?
INSKEEP: Well, when they're in that competitive environment, are they forgetting the other part - the supposedly more important part - about safe development of AI?
TONER: Yeah. I mean, the challenge here is there isn't any settled science for how to develop AI safely or for how to develop these increasingly advanced systems in a way where we can be confident that they'll do want (ph) them to do. So all of the companies are kind of improvising as they go and figuring out, OK, we have some evidence this might be fine. We have some evidence, you know, that this new system we've built might be a problem. How do we weigh that against each other? And each company is making those decisions a little differently.
INSKEEP: We were about to say goodbye, but I want to keep you on the line here for another couple of minutes because I want to understand a little bit better how seriously this - how seriously you take this. We heard elsewhere in today's program - we have reporting elsewhere in today's program about smaller AI companies who think this is kind of a scam - that the big companies are calling for regulation that would essentially freeze the playing field and keep the big players on top. Is that what's happening here?
TONER: I think it's reasonable to be skeptical of the CEOs of these gigantic companies, but I think if you look at the specifics of what's being proposed, that's actually not what it would look like. I think if we're going to implement any kind of government regulation here, the key is to do it in a way where it hits the biggest and most sophisticated AI companies hardest, meaning that it creates space for others to catch up. I think if we - there are ways to regulate that would make the life of smaller companies harder, and we should just not do those. We should instead make sure that it's really affecting the companies that are pushing the cutting edge, building increasingly advanced AI systems, because that's the activity we're most concerned about.
INSKEEP: Can you explain a regulation that would work and put it in terms that the layman could understand?
TONER: Absolutely. So the first step that is being proposed right now is things like transparency and disclosure requirements. So if you're a company - the key part here is if you're a company that is pushing the cutting edge in AI, so you're building more advanced systems than we've seen before, that means that you're a very well-resourced, very large, very successful company, and therefore, you should have more requirements to disclose what you're doing - to let in independent third-party auditors to check that you're doing what you say you're doing and that you're handling risk responsibly. So anything that applies to these companies that are at the frontier building more and more advanced systems is going to be helpful here.
INSKEEP: How do you make sure that people are disclosing everything and being transparent?
TONER: That's why people are talking about this idea of third-party auditors, independent groups coming in who get access to internal documentation. This is something that actually Anthropic, OpenAI and SpaceXAI - Elon Musk's company - have all said that they will do voluntarily. If it's on a voluntary basis, of course, you know, those independent auditors are a little bit at the mercy of the company, but you can imagine legal protections that would put them on firmer footing and make it - make us more confident that they're going to get the information that they need to ensure that companies are doing what they're saying.
INSKEEP: Yeah. I'm wondering about that part of it because people have talked about third-party auditors on this program, but they seem not to be talking about government inspectors like the Food and Drug Administration, but somebody that they hire to police them, even though the third-party auditor would be on their payroll.
TONER: Yes. So you need to design the program well. You need to have government, at a minimum, certifying those auditors. We do see this in other industries, where it can work well. So aviation, auto safety, other industries with generally good track record of safety, recent aviation incidents notwithstanding. So I think there's certainly ways to design an independent auditing program that will not work, but I think if we're thoughtful about it, it can make a difference.
INSKEEP: OK. So on the scale of 1 to 10, you're at 4 and we'll see if we can keep from getting too much farther. Helen Toner, thanks so much.
TONER: Thank you.
INSKEEP: She is the executive director at Georgetown's Center for Security and Emerging Technology and is a former board member of OpenAI.
(SOUNDBITE OF COVET'S "GLIMMER") Transcript provided by NPR, Copyright NPR.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.
