Hvorfor tradisjonelle rammeverk for KI-risiko bryter sammen

David Rice: A legal executive at a Fortune 500 company put it this way: her company wasn’t on fire, but it felt like everybody was holding a flamethrower. That’s where most organizations are right now, aware of the risk, not sure where to aim the hose, and the danger isn’t coming from outside. It’s internal momentum that nobody knows how to slow down.

On today’s show, I’m talking with Reid Blackman, author of The Ethical Nightmare Challenge, about why traditional governance frameworks aren’t built for what’s happening now. Your employees are already using AI in ways you haven’t anticipated. You can’t predict how they’ll use AI agents, and the only lever most leaders know how to pull is restricting access, which isn’t really a solution.

Reid makes a point that cuts through a lot of the noise around this. Moving fast and taking AI ethics seriously are not a trade-off. The companies treating it as a binary choice are creating the exact catastrophic trust failures they’re trying to avoid. And you don’t have to build a comprehensive AI ethics program overnight.

You can start at the project level, two or three teams, a ten-week pilot, even just one honest conversation about where the nightmares might come from. So what’s the first question worth asking in your organization? How about, have we stopped to identify what the ethical, reputational, and legal nightmares might be from how we’re adopting AI?

Most organizations, if honest, will answer no, and that’s where you start. So today, we’re going to cover why traditional risk governance isn’t equipped for generative AI or agents. What makes agentic AI a fundamentally different kind of governance challenge, why speed and ethics aren’t a trade-off, and what move fast and break things actually costs. And finally, how to start the ethical nightmare challenge at the project level.

I’m David Rice, and this is People Managing People. And if your organization hasn’t stopped to ask what could go wrong and who gets hurt, this conversation shows you why that question can’t wait much longer. Let’s get into it.

All right, Reid. Well, welcome to the show. It’s good to have you.

Reid Blackman: Thanks for having me.

David Rice: Yeah, I got the book here. I like the book. I think it’s a good read for everybody out there thinking about sort of the, the ethical challenges around AI.

Reid Blackman: It’s not my cup of tea, but I’m glad you like it.

David Rice: You describe a, a legal executive in the text, and you’re talking about at a Fortune 500 company who said her company wasn’t on fire, but it felt like everybody was holding a flamethrower, which I like that.

Reid Blackman: Yeah, yeah, it’s a quote. It’s a direct quote.

David Rice: It’s a remarkable image, right? ‘Cause it captures exactly how a lot of leaders are feeling right now. They’re aware of the risk, but they’re not really sure where to aim the hose on this whole AI thing. So I’m curious, you know, in your opinion what’s the structural reason most organizations are in that position?

And why, why doesn’t traditional risk governance close that gap?

Reid Blackman: That’s a big question. Okay, so the reason they feel that way is because There’s a lot of reasons. One of the main ones, though, is that they don’t know what kind of mischief their employees can get up to. Because the thing about things like, started with generative AI and then AI agents, which we can talk about, is that they can be used in so many different contexts by so many different people for so many different purposes that you can’t predict.

You simply cannot predict all the different ways that the tool is going to be used. And that’s why, and you also know, oh, right, these things can engage in certain kinds of privacy-violating behavior, they can be biased or discriminatory in various ways, they can hallucinate or just make up information. So there are many– There’s been law firms, the Big Four accounting firms in the news for handing in to, you know, client work with AI-hallucinated material.

And so when you put together the fact that you’ve got all the ways that things can go wrong with AI, combined with the fact that you can’t predict how all your employees are going to use it, of course, you’re gonna be nervous about the, you know, everyone’s holding the flamethrower now. If there’s an answer in your why doesn’t the standard approach work yet, but I’ll pause there for a second.

David Rice: I think what sort of strikes me about the image is it puts the danger in the hands of the people who are supposed to be steering, right? It’s not an external threat. It’s that internal momentum that nobody knows how to slow down at this point. I think that’s actually just, you know, it’s a leadership and culture problem that we, we talk about all the time, right?

It’s, as much as it’s a governance problem.

Reid Blackman: The only way they know of slowing down is by restricting access and not using the stuff, and just not using the tools. So they can ban stuff-

David Rice: Which is not really a solution.

Reid Blackman: No. It’s a stopgap measure at best. Right. And this gets even worse with agentic AI. So when you’re asking employees, “Hey, use these latest agentic systems, use Claude Code,” or something along those lines, and, “Build your own agent, build your own assistant, build your own dashboard,” now things get really out of control, because who knows what they might get up to in terms of what they’re gonna build and what they’re gonna use it for, and how are they gonna use it in their workflows?

And are they going to put the proper kinds of checks, and are they gonna know what the risks are, and how do they exercise their professional judgment in this very gray area context?

David Rice: It’s so difficult, right? Agentic is maybe a little bit different because of the way it has to be built. But I think, like, when we think about LLMs and GenAI in the workplace, I think what I also find interesting about the flamethrower analogy, right, is it’s HR walks in the room and the thing is already lit and spraying everywhere.

Because, because everybody’s already adapting to it and using it and incorporating it into their workflows in ways that you don’t even know or haven’t even thought of. And so now trying to get your hands around that is its own challenge. And then now they’re– you come along with this thing where it’s like, “Well, we’re gonna custom-build what?”

It’s a really interesting time. I’m curious because what I keep coming back to is a lot of this is a choice, right? We are implementing this in different ways, but who in the C-suite is kind of like authorized to pump the brakes? And say, you know, the pressure to accelerate is not coming from here.

Right now it is coming from everywhere. It feels like boards, investors, C-suite, the competitors. So Who is gonna own that moment to say, “Hold on. We gotta slow down. We gotta ask a few ethical questions here”?

Reid Blackman: Yeah, yeah.

David Rice: And it might feel a little career limiting, but we wanna do this responsibly.

Reid Blackman: Let me just define AI agents really quickly because I think that’ll help for the audience because that’s where things are headed.

So the way that I think about an AI agent, I, I definitely don’t think I’m alone here, is that an AI agent is typically has at the center of it an LLM, like a chatbot that you interact with, and then the two conditions have to be met to be an agent. Number one, it has to get connected to other tools. So you connect your LLM to the internet, and you connect it to maybe enterprise databases or enterprise software like your CRM.

It gets connected to other AIs perhaps. So condition number one is that your LLM has access to these different databases and software and other AIs, et cetera. Condition number two is that it exhibits some degree of autonomy, which means you give it the goal you want. So rather than you saying, “Hey, listen, first I want you to go to this database and pull this information, then I want you to run it through Narrow AI number two, and I want you to do this internet search, and then I want you to…”

You know, that’s a pain. What you really wanna be able to say is, “Hey, I want you to email my five top customers to say blah, blah, blah, based on yada, yada,” whatever it is. And it, as it were, quote-unquote, “figures out” for itself what tools to call, call on, what internet searches it should perform, if any, what databases it should access, if any, what other AIs it should use, what other pieces of software it should connect to.

And because it’s figuring out for itself how to pursue the user-specified goals or ends, it’s autonomous with respect to the means to those ends. It’s autonomous with respect to how it carries out the goal that you gave it. So if your LLM meets those two conditions, it’s connected to tools and it’s free, as it were, to figure out for itself how to pursue the goals that you give it, then you have an AI agent on your hands.

And you begin to understand all the complexity here ’cause there’s… What the agent looks like is very context-specific. So who’s using the agent? What tools did they connect it to? You know, what access does it have to what data, and to what other pieces of software? In what order does it call on those tools or use those tools?

What decisions is it authorized to make? And so on and so forth. So you get a lot of complexity here. And so not only do you have the sort of unpredictability of how people will use LLMs just by themselves, just straight-up generative AI, how will they use AI agents is even more unpredictable. Now I’ve said that, and I forgot your actual question.

David Rice: Well, I think the second part was you know, why isn’t traditional risk governance closing that gap? And I think you kinda, you speak to the complexity there, right? Like-

Reid Blackman: Yeah …

David Rice: AI agents, like we’re not writing clear job descriptions for them in some cases, and drift is a real thing in that we start to see them doing things that we never intended.

So yeah, I think the complexity of it and the technological savvy maybe is still gonna come a long ways. But I mean, you, you… I don’t know. You tell me.

Reid Blackman: It’s all those things. So when I think about what traditional AI governance looks like, and I say this as someone who has- Practiced traditional responsible AI or AI governance.

So, you know, I’ve been in this business for eight years advising large organizations for the past at least… No, no, actually, all eight years I’ve been advising large organizations on how to do AI governance or responsible AI or AI ethics, whatever I call it. And there’s a number of problems with it. And it’s not just me, it’s also my colleagues, my competitors, my first book articulated and defended that standard approach to AI governance.

And there’s several things that don’t work anymore. It’s, I think the approach is just broken. One, it’s very top-down and policy driven, and if you’re in a large organization, you know how long it takes to pass the policy. For the board of directors to ultimately pass an enterprise-wide policy, at least a year.

Some places it’s two years. It takes a long time. The fastest, absolute fastest I’ve ever seen is nine months, and that was by very competent, very smart, fast-acting decision makers. But that still took nine months. And meanwhile, the technology’s leaping ahead. In fact, that flamethrower company that you’re talking about, that’s a Fortune 500 CPG company, and she said the thing about the flamethrower, she was talking about generative AI as compared to narrow AI, rewrote their policy, blah, blah, blah.

It was a very robust policy, probably the most robust policy we’ve ever passed with any of our clients. I was a big fan of it. I thought, “Wow, this is amazing And then we started implementing it because, of course, once you sh- publish the policy, nobody cares. No one is rushing, waking up in the morning and jumping out of bed eager to read the new policy.

So now you’ve gotta do this awareness campaign. Maybe there’s three people in the company who are, who are into it, but for the most part, you’ve gotta build awareness about the policy and how it… You have to translate it to their department. You have to translate it to their role and their workflows.

This is a multi-month, really multi-year process, and we were doing that with that client, and then Agentic AI comes out, and oh my God, now we’re implementing an out-of-date policy, and we’ve gotta get the C-suite and the board DAC to approve a n- You know, and you can see this is a never-ending mess. And this is just AI.

I mean, we can talk later if you want about, you know, now there’s quantum computers and blockchain, so the technological landscape is just getting very complex very quickly, and this top-down policy approach is very, very slow. And then to your other point about the complexity of AI agents, a document that’s entailed by the policy is gonna have a bunch of procedures for compliance.

So let’s just say we’ve got 20 procedures on our hands. With AI agents, given their complexity and the context specificity, maybe 10 of those procedures are, yeah, that’s exact- You need to do those 10 procedures. The next 10 are gonna be totally irrelevant for your particular agentic context. There’s gonna be another 10 that’s not on the list at all that are really relevant to you, given how you’re, you know, what you’re building and how you’re using it.

And so this sort of let’s assume from the start that we know all the controls, that we know all the procedures that people need to comply with in order to make sure that things are on the up and up. That model is just broken given the complexity of Agentic AI.

David Rice: Well, you just said something interesting.

It’s let’s just assume that we know all the, the risks and the complexities and roles. We know that we don’t, right? So like-

Reid Blackman: Yeah, 100%.

David Rice: I saw that like AI incident data from the MIT, I think it’s AI risk repository shows like verified failure was nearly tripling over a four-year period between 2021 and 2025, and that’s just the stuff that gets reported, right? So-

Reid Blackman: Yep …

David Rice: you know, most boards are treating AI ethics as like this compliance checkbox. A lot of people think of it as like an IT issue, but at what point does that posture become Essentially a fiduciary failure, right? What does accountability actually look like when an AI system causes real harm?

Reid Blackman: Yes. So I think my answer to this question has changed over the years. It used to be that the sort of the people that I find objectionably salesy in this space have always said things like, “Oh, AI ethics and AI governance doesn’t slow you down. It makes you go faster, like brakes on a car lets you go faster.”

And I always thought, that’s just such BS. I’d love for that to be true. I can sell more stuff, but it’s just false. It’s just BS, so can we stop it? But I do think that changes with agentic AI. I actually do think that. And I think that’s because it’s just so freaking obvious that AI agents could go off the rails.

But you- you “Oh, with agentive AI, they’re gonna summarize emails and then send emails and generate documents.” And yeah, I can see how things can go a little bit wonky. But when you say, “Oh, we’re gonna create these AI systems that sort of make decisions at a scale and speed that humans could never match across dozens and dozens of pieces of software It’s easy now to be like, “Oh, that sounds like it could go real bad, real fast.”

You, you don’t have to be a technologist to be like, “Oh yeah, that seems crazy.” Autonomy and all that power that we’ve given to the agent, all the access to data and the tools, and potentially to customers and clients that you’re giving that LLM, that seems a bit crazy. So rather than brakes, I think a little bit more about the steering wheel now.

And if you don’t know how to steer agents in the right kind of way, it’s obvious things are gonna go sideways. It’s obviously things are gonna go bad. And so I think AI agents have made it so obvious that what I’m beginning to see, and it’s still the beginning, is that more senior leaders, and more specifically, more people in the C-suite who are not on the tech side of the house…

So traditionally it was chief data officers, chief analytics officers, chief information officers, et cetera, but now CHROs, COOs, CEOs obviously, CMOs have been now like, “Oh yeah, this stuff is crazy.”

David Rice: It’s interesting, right? One of the things I, I always think about is it’s not really just a legal problem, and we talk about it from an HR lens often, you know, it’s a culture problem, which it is, and there’s a lot of trust stuff that goes in there.

But there’s also a business continuity problem here. And so, I don’t know, I mean, is it the chief people officer that needs to be the one raising that before it ends up in legal? I mean, we look at things like the Workday- Yeah … the Eightfold lawsuits, right? You know, those aren’t fringe operators.

That’s mainstream HR tech. And are you carrying liability from just adopting those tools, and then, that you didn’t knowingly take on? And so, so it’s like a wake-up call about a vendor accountability and, and how we do procurement. I’m wondering too on the board piece, because I think there’s a real gap between what boards say about AI governance and what they actually know.

Reid Blackman: They have no idea.

David Rice: That was gonna be my question, is do they have the literacy right now to even- No … ask the right questions?

Reid Blackman: No. Whenever I talk to a board, I’m frankly blown away by how little they know. It’s honestly, it’s still day one. They’re, boards of Fortune 500 companies, they’re not…

Let’s just say they didn’t grow up with the internet or a personal computer or anything like that, right? They’re typically older. They know very little about AI. I think that they tend to be intimidated by it. I actually think one of the problems that we run into is a lot of non-technologists are intimidated by the technology I think to your earlier point, they think it’s an IT thing or it’s a data scientist thing.

They’re not technologists. It’s really complicated. I’m not a mathematician. I don’t know how to code. That’s for them. And so I think that fear that I’m not gonna be able to understand this stuff makes them keep AI generally at arm’s length, but especially when it comes to you need to think about how to avoid the nightmares, the really bad situations, the bad scenarios with AI.

They’re like, “That’s not my purview.” And I think that’s disastrous, especially now. Because one of the things that people keep celebrating about AI, especially generative and agentic AI, is that it democratizes AI. It puts AI in the hands of everyone, which is, it’s, it’s awesome. I mean, I’m… Even though I focus on the ethical nightmare challenge, I’m focused on the nightmares of AI, I’m not a skeptic.

I think AI is awesome, and I use it every day. But we need to get people comfortable if we’re gonna democratize it Which we need to do in order to really get the full benefit out of it. We need to get everyone talking about it and thinking about it in a way that’s not intimidating. We need to lower the bar to entry to having meaningful conversations about the risks and how to avoid them.

David Rice: Yeah. I mean, we’re only between six months and a year into HR people actually taking it on now and really starting to feel a little bit more comfortable using it and getting something meaningful from it. I mean, I can remember just like a year ago, having a lot of conversations with people who felt like, you know, “I’m not really qualified to use it.”

I mean, they were in the HR space, a-and challenging that narrative amongst people in the space because, yeah, we could see the potential of what they could build and what they could do, but it was intimidating at first for a lot of them, and I imagine it’s the same for boards especially, you know, to– they, they may have arrived there through very different means than by absorbing technology na- you know, naturally.

Reid Blackman: Yeah. Yeah, yeah. Yeah, yeah. A couple things. One, just a, a very short story, which is that I was giving a presentation to… a more general presentation. It wasn’t to the board, it was for all sorts of employees. Again, Fortune five hundred, this was a biotech company, and I was just explaining stuff about AI to risk and bias and things like that, and I gave an example of AI being used in HR context.

I think the idea was maybe I was just talking about narrow AI, so it was just scoring resumes. And there was some compliance person on the call, and she’s furiously typing into the chat box, “To be clear, the policy is that no one is allowed to use AI in any way for hiring,” it’s blah, blah, blah. She didn’t really forget about it.

So I’ve seen people get really worked up about, “Don’t use AI in HR.” That’s one thing. The second thing to say is, one of the problems with the standard approach that I talked about earlier, which is very policy-driven, top-down, et cetera, et cetera, is that it requires more or less someone from the C-suite on the tech side of the house to lead the charge because they’re the only ones who feel comfortable.

They– If anyone’s gonna own it, it’s probably gonna be, like, a chief information officer or chief information security officer, chief data officer, et cetera. And the HR people are standardly… Well, they’re never the go-to ones. That said, they’re often one of the first departments in which the policy is implemented because they hold so much sensitive data and make such high-stakes decisions.

But what we need now is, generally, I think we need a much more rapidly implementable and scalable solution, and I think the solution, this is what I developed in my book, we need a solution that can start anywhere. So I think we need a solution where the chief product officer can be like, “We need to figure out what AI governance looks like within product.”

We need a way for a CHRO to say, “We need to figure out what AI governance looks like for HR, and I don’t know what they’re doing over there necessarily, but we’re not gonna wait for an enterprise-wide policy. That’s gonna take too long. We need to do our own problem-solving around what does it look like for HR to avoid those nightmares?”

And so, as a matter of fact, this is for the neutral characterization of what’s needed. We need a rapidly implementable and scalable approach that can enter at multiple places in the organization and doesn’t wait for the C-suite as a whole or the board to do something, because you’ll be waiting forever.

That’s the sort of neutral point. The sort of less neutral point is my own view, which is something like, we need the department heads to rally around, “Hey, let’s identify what the potential ethical, reputational, and legal nightmares are for our department. What resources does HR, for instance, need in order to avoid those nightmares, and how do we need to train our HR personnel to use those resources effectively?”

I think we need something that’s rapidly implementable like that, and it can’t be that we’re waiting for legal to get involved.

David Rice: Yeah, I, I would agree. I mean, the sort of conventional responsible AI framework, I think, you know, is built for narrow, predictable systems. And, and you were, you know, arguing in the book that that’s already obsolete, and most organizations haven’t caught up.

But I’m curious about the organizational psychology behind the lag, right? Is it denial? Is it sort of speed pressure, or is it that the people setting the strategy generally kinda don’t understand what they’re deploying?

Reid Blackman: So the lag. There’s a number of ways to answer this. Here’s one way to answer it So we’ve worked with a bunch of clients, some we’re very successful with, some we’re less successful with.

A lot of different organizations hit different walls at different times. We’re working with someone in financial services at, you know, a bank, and they’re the head of AI innovation, and we’re building this responsible AI program. But then model risk management, which is a different department, they get wind of that, and they’re like, “Wait, you’re doing what?

No, no, that’s our domain.” And then there’s a political infighting, blah, blah, blah. That’s one example. Another example is we get a policy passed, we try to implement in a particular department, and the department head is like, “Wait, you wanna do what? I don’t wanna do that.” So even though you passed the policy, now you’ve hit a different wall of during implementation where some head of a department is especially in, if they’re in, in IT, they’re like, “No, that’s not what we’re doing.”

So y- how do we read this? And this leads, of course, to overall, it, you hit a wall, it goes slower than you thought it was gonna go because you’ve gotta get buy-in. The first thing that one of my colleagues said to me is, “You see, at the end of the day, they don’t care about ethics.” And I thought, “Actually, you know what?

I don’t think that’s the problem.” Shockingly, actually, for the most part, we’re dealing with good people who wanna do the right thing, period. And of course, they wanna do the right thing by their brand and their organization, and they want, you know… Some of this is rep- a lot of this is reputational risk management.

And I thought that, you know what? I think one of the reasons why we’re hitting a bunch of walls is that we’re relying on the chief data officer, the chief information officer, the chief information security officer, whatever, we’re relying upon them to engage in organizational change, and it’s a big, heavy lift, and they have no idea how to do it because they’re technologists.

They’re in the C-suite on the tech side because they were good at managing tech teams. They were never taught how to reach out to the CHRO, the COO, the CMO, the et cetera, et cetera. And so doing that kind of heavy lift org change was never taught to the leaders. And so that’s why there’s the lag that you’re talking about.

That was my first explanation. So then I thought, you know what we need to do? We need to help our clients do that org change. We need to train those people how to do org change. That was my first sort of pass or my first reaction. But then I did some more thinking, more writing, blah, blah, blah, and I thought, you know what?

Actually, the problem is not that it’s a big political lift and they need help with that thing, it’s that we’re giving them too big of a political lift to begin with. We need to figure out a solution that doesn’t require generating this insane amount of senior executive alignment in order to make good progress.

It’s too big. We need to start smaller. We need to figure out how to get in, in bits and pieces and expand from there, as opposed to this very top-down CEO-level alignment driving blah, blah, blah. I think that’s one of the major reasons you’ve seen the lag. It’s the standard approach to AI is very slow, in no small part because it’s very slow to Create the political will and momentum to actually pass a policy, get it implemented, integrate into workflows, et cetera, et cetera.

David Rice: It’s interesting, right? ‘Cause I think like with any major technological shift, right, where it outpaces institutional response, we have like frameworks that exist, and they, they feel better than nothing, right? So we keep using them. Yeah. But even when some of the underlying assumptions don’t hold anymore, we still are like, “Well, everybody knows that,” and there’s a comfort level and a sort of feeling of security around that.

And there’s a people dimension here as well, right, where like a lot of leaders who built these frameworks for, maybe it was for gen AI or predictive AI that they were using before, they were built in good faith. They invested their credibility in those. There’s like an ego and identity piece, acknowledging that they need to be scrapped maybe feels a little bit off to them or they’re not comfortable with that.

And so it’s interesting ’cause you got like a human problem inside of a technical conversation. But for every chief people officer and chief operating officer listening to this, I think it’s worth saying if your governance system was designed before agentic systems were on the radar, it’s gonna be pretty quickly out of date.

It’s not gonna be protecting you anymore. And it’s not a criticism of what you did. It’s just the reality of how fast this thing moves. It’s like what everybody’s dealing with, right? Nobody can keep their skills on pace with the rate of change, and it’s the same thing here, like policies are out of date.

Reid Blackman: Yeah. I’ve said, if you wanna take, you know– I’ve been a practitioner and an advisor on that standard approach. I’ve implemented it in various organizations with my team. I wrote about it in that first book. But if you wanna take the sort of, I think, the fourth and seventh chapter of the first book and use them as kindling or, you know, starter fire, fire starter-

that’s fine. You know, it just, it was good then. I don’t know that we got it wrong, but we got it wrong enough. We didn’t, if you like, as people like to say, future-proof it enough ’cause we didn’t know. I mean, I didn’t know that ChatGPT was coming along. I didn’t know that AI agents were coming along until February of ’24.

That’s when I s- you know, we first started getting glimpses into it. So the old thing was reasonable. You said this earlier, the old approach, the standard approach, was reasonable for narrow AI, which is why I wrote that book. It’s just not built for the agentic world. Forget it.

David Rice: I mean, I think about it, I’m like, you know, 2022, and you talk to me about AI, I’m like, “Oh, you gotta be a data scientist to figure it out or to understand what’s really going on.”

And now it’s there’s a million things going on. Who knows? You know, like anybody and everybody’s using it, and we’re just gonna see that proliferate. It’s only gonna get more and more complicated. It’s only gonna bring more and more people in trying to do things that you don’t even, we can’t even predict.

So yeah, it’s,

Reid Blackman: So I think that’s right, but I think it also highlights an important point, which is that we need to increase clear communication. So right now, as you said, everyone and their mother is trying to figure out AI. They’re using it or they’re thinking about using it, they’re dabbling with it, they’re AI curious, whatever, and we need to be able to talk to each other about it.

But the data scientists don’t know how to talk to people in HR or COOs. They– And vice versa. The people in HR probably don’t know how to talk to the product people about it. And with all this complexity, if we can’t It is just a complex landscape. There’s nothing we can do about that. But if the way that we talk about that landscape is confused, and we’re using a different language, it makes it a hell of a lot worse.

And so the question is then, in the face of all the complexity, how do we create a clear language with which to talk about this stuff? We need that communication not only so that we can get a grip on the landscape, but we need, of course, for collaboration. Because, again, part of the point of agents is that they can do these cross-departmental, cross-functional things, that they’re– they can do multiple– lots of stuff.

They can reach across the aisles and speak to other departments, which means that departments have to collaborate with each other on building and, and monitoring and controlling these AI agents. And so we need the communication so that people can– clear communication so people can understand the complex landscape, but also that they can talk to each other and therefore collaborate about, “Hey, how do we avoid the really bad stuff?”

That’s one of the reasons why the book is called The Ethical Nightmare Challenge and why I use the word nightmare, because everybody knows what a nightmare is. It’s not techie, it’s not jargony. It’s, “Hey, what are the nightmare scenarios here?” People in HR, they can answer that question. It’d be a real nightmare if we figured out that we were using AI in such a way that we were systematically not interviewing women over forty years old or something along those lines, right?

That you can say what those nightmares are. And if I just say, “What are those nightmares?” You can know what those are. And then you can also say, an HR person can say to a data scientist, “So, okay, we like what you’re doing, but here’s one of the nightmares that we’re really worried about with the software, the AI solution that you’re providing us.”

Okay, now we’re talking about nightmares. We’re talking about concrete outcomes that we want to avoid, as opposed to very, like wonky standard operating procedures and principles about data minimization and stuff like that. That’s so in the weeds wonky, and the HR person’s gonna be like, “Data minimization, I guess, but what are the other fifteen techie things?”

Forget that stuff. Here’s the really bad stuff. How are we making sure that we’re not gonna run into those problems?

David Rice: You mentioned in there, like the nightmare scenarios, and the book asks leaders to imagine the worst plausible outcomes of their AI systems before they happen, right? That sounds obvious, but it clearly isn’t common practice.

So what happens in those sessions that leaders aren’t expecting and, and what is a sort of review about how well an organization really knows its own AI stack?

Reid Blackman: They don’t. The nightmare stuff has been eye-opening for a lot of people. And you’re right, it, it is obvious. Like, how do we not start there?

I mean, I know why I didn’t start there personally for two reasons. One, my background is I was a philosophy professor. I specialize in ethics. Ethicists usually talk about the good and the right. We talk about the wrong stuff and permissible, impermissible, but talking about nightmare scenarios is typically not our thing.

We’re trying to articulate what does the good look like, and then how do we get closer to that, something along those lines. And then just from a, a risk perspective, risk, it’s usually very procedure-oriented, and this is what I learned working with one of my colleagues who was a chief risk officer and chief compliance officer.

But it was always very procedural “Hey, did you– are you making sure to only collect as much data as you need and not more than that?” You know, that sort of like very And so I don’t know, even though it’s obvious, it’s just like the way that ethicists think and the way that risk people think just didn’t lend itself to, let’s just start with identifying the worst case scenarios.

Even though, by the way, it’s pretty standard in engineering. Standard that if you’re building a plane, you look for ways that the plane can go break, and then you try to fix those possibilities. So there are areas of risk management where that is standard, but in the AI space, it just hasn’t been the thing to do And so when I started thinking about the ethical nightmare challenge, which starts with step one is tell me what the ethical nightmares that pertain to your organization are, your AI ethical nightmares are.

It seemed obvious to me once I finally said it, and I felt validated when I saw that it was going on in other kinds of engineering practices. Now, as far as what it opens up, here’s the way that those board-level conversations used to go. You would start with your values, not your nightmares, of course.

So you start with, “Oh, we’re for fairness and transparency and accountability and privacy,” and, you know, a laundry list of generic, you know, moral values or ethical values or something along those lines. Sometimes it’s called principles, sometimes it’s called pillars, but whatever. And then you would say, “Well, what the hell does that mean, we’re for fairness?”

And everyone stares at it, and they’re just like, “I don’t know.” And so if you’re going to actually make that value statement a real thing, and I advocated for this for years, you translate those values into procedures. So if you’re for fairness, that means we’re going to engage in the procedure of checking for bias at each stage of the AI life cycle.

We’ll identify potentially discriminated against stakeholders. For transparency, we will engage in the procedure of ensuring that we communicate to end users that they’re interacting with an AI, and so on and so forth. So this all gets translated to procedure. And now this conversation becomes, are we going to engage in these procedures or not?

And at the C-suite and board level, this is very boring for them. It’s very wonky for them. It’s too in the weeds. I totally get it. But that’s all we ever did with those value stuff. What else are you gonna do with it? ‘Cause ask them, ask the CHRO and the COO and the CMO and the CEO and the board, what does fair AI look like?

You’ve got a billion answers, and they’re all contradictory. So it’s just a bad, bad place to start. So now when I say to leaders, “Okay, what are the nightmare scenarios?” They have a much better grip on what that looks like. It’d be a real freaking nightmare because blah, blah, blah, and our investors would be pissed off and blah, blah.

They know how to specify what really bad looks like. And that’s one thing that I love about nightmares. It’s one thing I like ’cause everyone understands what it means. Number two is that it’s way easier, which means it’s way easier to drive alignment. We were talking about driving alignment before. Way easier to drive alignment around the really bad stuff as opposed to what the ideal looks like.

It’s not perfect. You’ll still get disagreement, of course, but it’s just, it’s just much less friction. Not absent, just less, a lot less.

David Rice: And it’s also motivational. It conveys the right kind of urgency. No one’s “We better get on these procedures

Reid Blackman: now. I’m real excited about how these procedures realize the value of fairness, so let’s– I can’t wait.

Let’s put a lot of momentum behind that, a lot of effort behind that.” No one does that. No one cares. But Oh, wow, these nightmare scenarios, now that we think about them and we think about the degree to which they’re likely and how we might run into them, yeah, that’s– now I’m staying up at night. The nightmares are keeping me up at night in the right kind of way.

And so now, yes, let’s definitely do something about this.

David Rice: What’s cool about the nightmare conversation, I think, is like it gives you a structured way to, to move towards intellectual honesty, right? Or at least a form of it. Because like you said, the values conversation, asking what does it mean, and a lot of people just look at you, you know, it’s like-

Reid Blackman: Yeah.

David Rice: Yeah … I don’t, I don’t know, you know? It’s– Organizations are not necessarily good at that level of honesty when there’s momentum and investment behind something in particular. I don’t wanna be the person in the room that says, “Yeah, this is cool that we’re doing this, but what if this actually does this?”

And it’s the real harm, right? ‘Cause it makes everybody uncomfortable. But somebody in the room has gotta say it, and there’s like this thing, you know, we talk all the time about psychological safety, but we need to hear as well because the willingness to imagine worst case scenarios requires sort of a, a culture where you can say that uncomfortable thing without it being seen as like the negative Nancy or the doomsayer, so to speak.

And so, I think it’s cool that your process for Ethical Nightmare Challenge is like a diagnostic for your culture as well.

Reid Blackman: Yeah, I think that’s a nice way of putting it. It’s also, in my estimation, a way of building that ethical culture where it doesn’t require… I mean, it requires psychological safety in the most minimal sense ’cause it be like, “This is just what we do here.”

I wanna highlight part of the solution. So we talked about the Ethical Nightmare Challenge, and that really consists of three questions. So the first is, as we’ve, we’ve talked about now, what are the AI ethical nightmares of your organization? What resources do you have to avoid those nightmares?

And how will you train your people to use those resources effectively? So those are the three questions of the Ethical Nightmare Challenge. One thing that I really like about those three questions is that they’re highly portable, so you can ask them at any level of the organization. So you can ask it for, at the solution level.

Hey, for this particular AI solution, what are the AI ethical nightmares? What resources do we have for avoiding those nightmares? How do we need to train developers or end users to avoid the nightmares for this solution? You could bump it up a level to, let’s say, the department or division level. Say it’s HR.

What are the ethical nightmares of HR? What resources does HR have? What training do HR personnel need to avoid those nightmares? And you could ask it at the enterprise level as well, et cetera, et cetera. You know, for organization as a whole. Yeah. So that’s what I like about those three questions. But then there’s a crucial thing, which is, well, if you like, level one here is I think everyone should be trained.

I mean, this is me being less neutral, but everyone should be trained on those three questions. They should be trained on what the heck AI is, what are the nightmare scenarios that are baked into AI. So now we’re talking about biases and hallucinations and automation bias and cascading failures in agentic systems, blah, blah.

And then they should be given this three-question framework. But when it comes to other things like procuring solutions from vendors or developing AI solutions internally, now there’s what I call ENC, Ethical Nightmare Challenge or ENC teams, and the mission of an ENC team is to answer those three questions.

Again, let’s go with an HR solution. Let’s say it’s a case of procurement. Here’s what I’d like to see. I want an ENC team who’s gonna answer those three questions, obviously gonna be comprised of a cross-functional group. So you’re gonna have someone from HR, you’re gonna have someone from data science, probably gonna have somebody from legal, risk, and compliance, or one from each of those, or however your organization is constituted.

Ideally, you’d even have on your ENC team a member from the vendor from which you’re procuring, and that way, what you’re trying to figure out, what resources do we need to avoid the nightmare scenarios, some stuff is gonna fall on the vendor, some stuff is gonna fall on you internally. So Now, if you just take these cross-functional experts and throw them into a room and say, “Solve the problem, answer the questions,” forget it.

So the other thing that I developed in the book and that we deliver to clients now is a seven-step method. So here’s the seven steps. Here’s a worksheet that the team, this cross-functional team, is gonna fill out. What are the nightmares? Score how likely they are, how bad the impact would be, so you get an overall risk score or nightmare score, so you…

now you can prioritize nightmare avoidance. And so now, if you can create an ENC team on an as-needed basis, what we were talking about earlier was the difficulty in scaling policy implementation. This is just create an ENC team on an as-needed basis where you need them. So if you need one in HR ’cause they’re doing a bunch of AI and marketing is not, create an HR ENC team, but not a marketing one or vice versa.

You need an A- ENC team for this AI solution but not that one because this one is obviously high risk and this one is really minor, okay, great, then do that. It takes care of that rapidly implementable thing. The other thing, though, was that by creating these ENC teams where they’re needed, you’re creating…

you used the word culture earlier. I wanna say something that’s slightly, slightly different, but there’s a Venn diagram where they overlap. You’re creating organizational capacity for nightmare avoidance or ethical nightmare avoidance. The more readily people in the organization know the language of what are the nightmares, what are the resources, what’s the training, and the more readily that you’re able to form ENC teams where and when you need them, the more it’s just, oh, we have an organizational capacity to engage in AI nightmare avoidance at scale, and that seeps into the culture.

It can’t not, right? ‘Cause we’re just… The fact that we’re all talking the same language and engaging in the same practices does a lot to build what the culture is.

David Rice: And from a people operations standpoint this feels like something that should definitely happen before deployment and not after an incident occurs.

But I talk to a lot of HR leaders out there who are… feel like they’re reacting all the time. They’re not necessarily anticipating. And so I, I think the challenge, and I’d be interested to get your sort of tips for people on this, is how to build anticipation into their initial processes when that’s maybe not been the habit that they’ve built over time.

Reid Blackman: There’s a couple things to say here. One thing to say is that when you’re talking about project-level ENC teams, they are engaged in a proactive, let’s think about what might go sideways before things go sideways endeavor. That’s their thing. Their mission is to identify the potential nightmares and create the resources and training to avoid them.

And part of that, by the way, is creating the tools by which they can monitor the AI once it is deployed, because you can’t just sort of like set it and forget it. You gotta keep a close eye on this stuff, and they have to figure out what does keeping a close eye on it look like for this particular solution.

That’s one way to be proactive. The other way to be proactive, which might be more relevant to your audience, is that suppose you have a department-level ENC team, and let’s say it’s an CHRO, you know, or HR department-level ENC team. They’re asking those three questions, “What are the nightmares we need to avoid?

What resources do we have? What training?” That is two things. One, it’s proactive. It’s not just let’s wait until things go sideways, but what can we do now to avoid those nightmares? The second thing it’s that I’m asking Those leaders, CHROs, COOs, also CMOs, say C blah, blah, blah, heads of departments, to reconceive their role in nightmare avoidance.

All E&C teams are fundamentally problem-solvers, not engaged in procedural compliance. We talked already about why procedural compliance in this context doesn’t work. We need good problem-solvers. We need people to figure out, “Hey, where the hell are the icebergs, and how the hell do we steer away from them?”

And that’s a problem-solving endeavor, and the answer might be different for each thing. So one way to be proactive is to engage in those kinds of questions where you’re trying to think, “I’m at the department level. I need to think about how do I avoid the nightmares, but more specifically, I need to think about how do I enable my project-level E&C teams to avoid their AI solution-specific nightmares.”

So the way that I see this is it’s cascading enablement as opposed to cascading rule-giving. The department heads shouldn’t be there just to give the rules. They should be there to help solve the problem. It might be a matter of giving some rules, but it might be a matter of giving additional specialized training.

It might be a matter of creating a nightmare inventory for an HR. It might be a matter that project E&C teams can draw from. It might be a matter of creating a library of nightmare-avoiding strategies and tactics that project leaders draw from. It might be organizing monthly or quarterly sort of roundtable discussions among the project leaders around this stuff.

There’s lots of things that a department-level E&C team can do, and all that stuff is both problem-solving and proactive.

David Rice: Agentic AI, as we’ve been talking about, changes the risk calculus in ways I, I don’t think most leaders have internalized yet. You know, systems don’t, they don’t just respond.

They initiate, they chain decisions together, and they, they act in a world or in the world on behalf of the organization. But for a COO or chief people officer, those people are now working alongside autonomous agents, the accountability factor is changing. What does that mean for when something goes wrong?

Where does the org need to own that? Who has accountability for these agents?

Reid Blackman: One, I think E&C teams are the first layer. But the thing is that if you take a step back and you ask a sort of broader accountability question, who’s responsible when things go sideways, it’s not necessarily obvious.

I think about this like a helicopter crash. So a helicopter crashes. Who’s responsible? Who’s accountable? We gotta do investigation. Was it a design flaw? Was it a manufacturing flaw? Was it a maintenance flaw, problem with how the maintenance crew worked? Was it the pilot error? Was it, whatever, air traffic control error?

There’s so many places, and that’s because you’re dealing with a very complicated piece of machinery where lots of teams had their hands on lots of different aspects of how this thing was built. It’s the same thing with AI agents. We conveniently conceive of them as like people, as agents, as a coherent entity.

But in fact, they are sets of tools cobbled together more or less intelligently. So did things go wrong because this tool to which it was connected went sideways and that echoed throughout the system? In that case, who made that tool? Might have been someone else, could be a third party. Was it who made the LLM?

So was it the foundation model company? Was it a, a Google or a Microsoft or an OpenAI or an Anthropic? Then again, those foundation models are modified in various ways by internal developers. Let’s say in a Fortune five hundred company, they do things to it. They add adapters, they might fine-tune. Is it their fault?

There’s no more general answer to whose fault is it when AI agents go sideways than there is a general answer to whose fault is it when helicopters go sideways.

David Rice: Well, it’s interesting ’cause like the agentic shift and the thing that’s maybe a little bit unsettling about it for some people is that it dissolves the sort of human decision point that we’re comfortable owning, but that’s where accountability lives, right?

It’s like where the decision was made.

Reid Blackman: You can put authority or res- accountability on the person who greenlit it to some extent. There’s negligent or reckless greenlighting of AI solutions, right? So clearly, whoever’s going to be responsible for saying, “Yes, deploy,” or, “No, don’t deploy,” a lot of accountability is gonna fall on their shoulders, but they are approving something that is fundamentally unpredictable.

And there’s ways that things can go sideways that are not reasonably foreseeable by the person who greenlit the AI, in which case it wouldn’t be right to hold them accountable.

David Rice: Well, and then you’ve got a lot of organizations where people aren’t even fully aware of what they’re working alongside of, which I think is a really interesting thing that’s happened, because if they’re not aware that they’re working with it or being evaluated by it, in some cases, there’s a transparency and consent dimension there goes beyond compliance really.

It’s a– This is– We keep talking about trust and sort of people being disillusioned and disengaged, and I’m like, well, this is just like pouring gasoline all over the fire when you do this. It blows my mind that it’s even happened that way at this particular time. Yeah. But I, I guess I shouldn’t- Yeah be surprised, but.

Reid Blackman: Well, you know, I’ll say one last thing. Executives are afraid to have this conversation with their employees. They think that if we talk about the risks and we talk about the bad stuff, that they’re not gonna adopt it. I’ve received pushback from, I’m thinking in particular of a CP in AI governance at a Fortune five hundred financial services company who’d be like, “Ah, I like the word nightmares, but I think some people might not like it.

It might be too negative for our crowd. Can we change it?” And I get where they’re coming from, but I just think it’s, it’s misguided on at least two fronts. One, it’s misguided because, look, they’re real problems, and you don’t solve the problems and you don’t avoid the problems by putting your head in the sand and just saying “Don’t worry, everyone.

Everything is great.” You will run into the problem. So just… You can’t both be a hard-nosed capitalist, worry about money, dah, dah, dah, and then, but don’t say the bad stuff. We’re too delicate for that. That seems like a weird posture. Anyway, so you’ve got to name your problem if you’re gonna solve it. So that, that’s one reason why I think it’s a mistake.

The other is that I just think it’s psychologically misguided. You got these people who are anxious and scared about AI. They don’t know what the hell it is. They don’t understand it, and I think that that lack of understanding leads to a tremendous amount of anxiety. And they, they know about bias, they know about hallucinations, or they’ve heard about stuff, but they know, they know things can go sideways and I don’t think you say to them, “Don’t worry about it.

It’s great,” and then they’re– that’s gonna increase adoption. I think if you wanna increase adoption, it’s psychologically astute to say, “We want you guys to use this stuff more. We recognize that there are issues with it. We recognize you’re anxious about it. We wanna make sure that you feel comfortable, you know, with what this thing is, and we wanna give you a way for thinking about how to use it responsibly in a way that’s aligned with your professional judgment.”

That’s a much, I think, more psychologically plausible way of increasing adoption than pretending that the nightmares don’t exist.

David Rice: You telling me that story about, you know, that some people don’t like the word nightmares, it’s… This goes back to something that I come back to all the time, and it’s something I’d really like to see leaders address more, and that is we’ve got to stop softening language to try to make things more palatable.

It’s just a bad habit we have in our society. I had the, the instant recently that I’ve been– I keep coming back to, I keep thinking about is somewhere along the line, we decided that homeless people are no longer called that. They’re unhoused. I’m like, what does that do to help the discourse around how we can get these people a better place to sleep?

It does nothing. You know what I mean? Yeah, yeah, yeah. It is not useful language or change of language. There are moments where the language does have to change and evolve, that’s fine. There’s a lot of times where we try to soften it, and it’s like in this case, you’re saying not a nightmare, and it’s would you rather use disaster?

What do you wanna use?

Reid Blackman: Right. Catastrophe?

David Rice: Yeah. What are we talking about here?

Reid Blackman: I know. It’s wild to me, especially because I come from philosophy, where it’s we wanna explain the issue head-on, plain language, be blunt. But there’s this sort of you know, everyone smiles, AI is amazing, and it is amazing, but we don’t have to smile the whole time.

We can recognize there’s there’s some screwed up things about it.

David Rice: I laugh at the social media feed all the time, right? There’s funny stuff on there. But I can also recognize that this thing is awful for us and has made people- Yeah … big public discourse a nightmare, right? We have to recognize- 100% the reality of what it is.

Reid Blackman: Yeah, yeah, I know. Crazy. Yeah, there’s this weird thing where we sort of like, I don’t know, it’s almost like we infantilize employees. You know, I have young kids. I think I do a pretty good job of not talking to them like they’re babies and little kids. I mean, I do that too, but there’s some stuff I can hold back on, of course, but we know about overparenting or helicopter parenting.

Ooh, don’t… Oh, the kid, he’s very delicate, you know? We’ve done that with employees. We’re like, “Ooh, you know, they get scared off by AI. Don’t say bad words.” What are you talking about? They’re adults. They’re fine.

David Rice: And they’ve used tools before. They’ve used new technology before. If you don’t give them helpful ways on how to use it and direction, they’re gonna figure it out themselves, and you might not like what they figure out.

Reid Blackman: Yeah. Yes. This goes back to, I used to talk about a lot of the discourse around AI, sometimes a little bit less now, but especially when generative AI first came out, it, it sounded like the conversation about abstinence. It’s “Don’t do it. Don’t touch it. We have a strict policy in our organization.

There is no generative AI. You’re not allowed to use it for anything.” I’m like- Does that work for teenagers in accidents?

David Rice: The first thing the employees did was fire up chatgpt.com. Yeah, of course. It’s of course.

Reid Blackman: Get ready for some babies, okay? You better…

David Rice: Well, you make the case that ethical nightmares, you know, they rarely come from malicious intent, and I think that’s pretty true, right?

They come from a lot of well-meaning people moving fast inside of systems that maybe they don’t fully understand or that weren’t designed to go that fast in a lot of cases. And that’s a, a bit of an uncomfortable truth for leaders who have built their AI strategy around speed and competitive advantage, right?

Those are things that we’re always focused on. But truthfully, it’s uncomfortable for a lot of the people doing the work. I’m curious, like, how do you make the argument to a CEO who believes the bigger risk is falling behind, and they wanna see those productivity gains, and they wanna see everybody moving fast.

How do you make that case to them?

Reid Blackman: I mean, I don’t think there’s any contradiction. Again, I, I always scoffed at the, “Oh, responsible AI or AI governance is the brakes on the car,” blah, blah, blah. But I do think that, you know, to my point earlier, that is the world we’re in now, where AI agents can clearly go so far off the rails so fast, and okay, if you wanna move fast and compete, fine, but then fully educate your people.

You wanna move fast and do all this stuff with a kind of half education. Only, only educate them on one side of the coin, the opportunity side, and not the risk side at all. That just seems insane to me. I think it’s obviously insane. And in fact, I think that because we know that AI agents will go off the rails if not created and overseen and used in the right kind of way, if you tell everyone to run out, you know, the second they hear the starting gun, they’re gonna stumble out of the blocks or at least stumble sometime along the track.

We just know that. So you gotta teach them how, how not to stumble. I just think that’s sound kind of obvious. If they don’t see that point, they don’t understand how easily AI just goes sideways. So teach them how it can go sideways and teach them how to exercise their professional judgment, and I think that’s the only way to get them to go faster.

David Rice: I think this is one of the more important reframes of this whole conversation, right, for leaders that are listening, is the narrative that a lot of executives have bought into or have, have internalized is that ethical AI failure is something that happens to bad actors, companies that cut corners or they don’t care.

I think what the book is saying that’s really interesting is this happens to organizations that care deeply, but they’re moving faster than their judgment can keep up with. That sort of changes the intervention.

Reid Blackman: That’s a nice way of putting it. That’s a nice way of putting it. Moving faster than the speed of human judgment.

David Rice: Well, it changes the intervention too, right? ‘Cause it’s not about installing ethics as a set of rules necessarily. Hundred percent. It’s about building the capacity within the organization to slow down at the right moments. In a speed-obsessed culture, that’s genuinely counter-cultural, right?

Reid Blackman: Yeah. Well, that’s how you get move fast and break things, which maybe you’re two things. A, you’re amoral, and B, your revenue will outpace any fines or reputational damage, right? So I’m thinking about Meta. But maybe you’re not that. Maybe you’re just a standard Fort– you know, non-tech Fortune five hundred company, or maybe you are a tech Fortune five.

But and you think, “You know what? Actually, we’re not gonna just move fast and break things and just throw money at the problem. We do care about our brand.” Then you probably shouldn’t move that way. That seems like a really silly way to move.

David Rice: I like that you challenged the speed versus ethics frame.

It’s a false choice, right?

Reid Blackman: Yeah. And just anecdotally, just anecdotally, I think I am ahead of the curve of AI adoption. Most people I talk to, I’m like, “Oh, what do you do? Oh, you only…” And I’ll say, “Oh, I do this, I do this, I do this.” I mean, I use it for all sorts of things. That’s incredible. And I’m the ethics guy.

And I think because I understand the ways that things can go wrong, and I understand I have to make sure that I verify the information, or I understand that I know it told me to do that, but I don’t think that’s right. I know to resist my own inclination to trust the computer. That’s what has allowed me to do so much with AI.

So again, it just, it just, it just strikes me as misguided to think that either you move fast and you break things or you can’t move fast. That’s just not true.

David Rice: No, I, I like to push back up on the move fast, break things ideology all the time, right? I think the real risk is having a big catastrophic public trust-destroying failure that sets the business back.

I was on a podcast recently, we were talking about that, and I was like, “How about we– can we just put it out there? Move at a normal speed and make things better.”

Reid Blackman: Well, I think, I think that is- shift the thinking. Yes.

I will say this. People are looking they’re like, “Reid, why should companies do this?”

And what they’re looking for when they ask me that is a complete knockdown argument for any and every company and every skeptic, and that’s just not possible. I’m not persuading Zuckerberg. I’m just not doing it. It’s not gonna happen. That company survived Cambridge Analytica. Any other company would be done after Cambridge Analytica, right?

They’re donezo. But Facebook’s Facebook, or now it’s Meta. Then I think it was Facebook. And they have so much money that You know, you wanna fine them a million, 10 million, trying to fine them $100 million? Okay, couch cushions, no problem. Most companies are not like that, so I don’t think there’s a, there’s a drag out knock down argument, the argument that will rule them all for why you have to take AI governance seriously.

That doesn’t exist. But I think for any company that is not like that, there’s just obviously good reasons not to move fast and break things. Your brand can’t handle it, and that’s put the ethical reasons to the side. Ethically, you shouldn’t break things, or not, you know, not things that ought not to be broken.

But anyway, yeah, I think there’s plenty of reasons. I think it’s pretty obvious.

David Rice: The last question for you is, you know, if a C-suite leader has been nodding along to this conversation, they recognize the exposure, right? They know their current frameworks probably aren’t enough. What does the first serious conversation inside their organization need to look and sound and who needs to be in the room?

Reid Blackman: Well, one of the things that I try to stress is that it doesn’t ha- there doesn’t have to be a whole bunch of different people in the room. The ethical nightmare challenge is you can start it at, at the department level, at the division level, at the project level. I think if there’s going to be one question they should ask, it’s have we stopped and identified what the ethical, reputational, legal nightmares might come with our adoption of AI?

Now, if they say yes, then great, but I’ll bet you they’ll say no. And then if that doesn’t… They need to create some sort of, “Should we do that?” Then I don’t know what’s going to get them to move.

David Rice: Yeah, I mean, you know, thinking back at the, what we’ve talked about here, and I think one thing that kind of lands it in a practical place was you were, you were talking about how it can start in different places.

And I love that you just said even at the project level. Because sometimes people hear this conversation, and they feel the weight of it, right? They don’t know where to take the first step, but they know that it’s heavy. But what you’re describing is permission to just start small. Not, you don’t have to build a comprehensive AI ethics program overnight, but you do have to start by having at least one honest conversation about where the business is with this.

Reid Blackman: So here’s something that we do with our clients now that we c- we could never do before with the standard approach. We can pilot The Ethical Nightmare Challenge. We can say, “Hey, we’ve got the training. Identify two, three teams.” Could be project-level teams, could be two project-level teams in a, in a department.

“And we’re just gonna try out. We’re gonna train these people. We’re gonna run through this method with them, and we’re gonna get to work doing nightmare avoidance in that 10-week pilot.” Start there. If for some reason it doesn’t work or you don’t need, turns out you don’t need it, okay, we’re done here.

It’s such a tiny thing to start with you know, a couple project-level teams, and see what they, see what they think about it. Oh, yeah, that actually did work. Do we need to scale it? No, we don’t. Do we need to scale it? Yes, we do. Okay, great. You know, so you can just go from there, but you scale it as you need it.

You don’t, you don’t have to start with this big heavy lift. So as compared to, “Hey, let’s pilot this enterprise-wide policy,” that’s not a thing that you do, right? That’s not, that’s not on the menu. Let’s design this thing for a year plus and then begin implementation of a pilot. What are you talking about?

No. You can and you should start small.

David Rice: Well, excellent, Reid. Well, thank you for coming on to the show today. It’s been great chatting, and I enjoyed this conversation.

Reid Blackman: Yeah, likewise. Thanks for having me.

David Rice: All right, well, listeners, be sure to check out Reid Blackman, his book, The Ethical Nightmare Challenge. Do give it a good look and some consideration. There’s a lot of good questions in here, a lot of things that we all need to be thinking about.

If you haven’t done so already, head on over to peoplemanagingpeople.com/subscribe. Get signed up for the newsletter. You’ll get this podcast and everything else that we create straight to your inbox.

And until next time, let’s make a habit of asking, “What could go wrong here, and who gets hurt?” Rather than, “How fast can we ship this?” See you next time.