The AI Decisions Are Wrong (And Your Data Isn't the Problem)
Guest: James Taylor, Founder and Executive Partner of Blue Polaris
Host: Kumar Dattatreyan
The Meridian Point
Episode Date: June 16, 2026
---
KUMAR: Hi, good afternoon, folks. This is Kumar Dattatreyan with The Meridian Point. Today we're joined by James Taylor, the founder and executive partner of Blue Polaris, formerly called Decision Management Solutions.
For over two decades, James has been one of the most influential voices in a discipline most organizations don't even know they need: decision management. He's the author of Digital Decisioning, Real-World Decision Modeling with DMN, and Smart (Enough) Systems. He's a former VP at FICO who helped create the decision management category, and a co-submitter to the DMN standard. Today his firm works across insurance, banking, healthcare, manufacturing, and travel to bring structure to the messiest part of any business: how decisions actually get made.
In an era where everyone is racing to bolt large language models onto every workflow, James has been quietly arguing for something more durable: a multi-method approach that puts decisions, not technology, at the center. He's here to talk about what digital decisioning really means, why most AI projects miss the point, and what it takes to build systems that can explain themselves.
All right, without further ado, here's James. Thanks so much for joining us today, James.
JAMES: Well, thanks for having me, Kumar. It's great to be here.
KUMAR: Of course. So you've spent a couple of decades building this discipline around something most organizations treat as an afterthought, their own decision making. When you started at FICO and began shaping what became decision management, what were you seeing inside companies that convinced you to pursue this path?
JAMES: It's interesting. What I noticed, focusing on retail credit and retail banking, was that there was a set of applications with different names: originations, account management, fraud, collections. They all used the same technology stack and all worked the same way. At the heart of these systems was this technology stack being applied to the decision making, not to anything else.
Sure, they had workflow or they had case management, but what they had mostly was this very focused core that made a high-volume transactional operational decision. Should we pay this credit card transaction? Is it fraudulent? Should we lend money to this person? It seemed to me that these applications had a set of common characteristics. They had a set of common technology. They were a set.
We started talking about them as though they were a set. Look, there's this decision management approach. It's a way of combining these technologies and delivering value across a wide range of solutions. They were really decision-centric. They used this mix of technologies. All of them, without exception, had this focus on continuous improvement. This set of things seemed to us to be a repeatable pattern, a thing you could describe, you could talk about, you could give it a label.
Over the years, what's changed is really only the price performance. It used to be you could only afford to do this for very high-risk, high-value, and very high-volume transactions. Now the price performance has improved. The technology has gotten easier to use. But the core is still this focus on the set of technologies being applied to a decision. That's the essence of decision management.
KUMAR: Okay. But isn't that similar to, or the same as, business process management? You've got some decisions that get made, that are automated in some way. What's the difference? What's the same?
JAMES: Sure. The thing we often observe is that as you run through a process, it's a sequence of steps with different people being assigned to it. There are moments in that process where you have to make a decision, and that decision is made at a point in time. It turns out that if you only think of it as part of the process, you tend to break it up into pieces and smear it through the process. It's much more productive to think of the decision as its own artifact, with its own place in the process.
What we've noticed is that there are some processes that don't really have very much decision-making. They just have some routing logic or things that the process owns. But there are other processes where there's a decision made in it that changes on a completely different change cycle. It's got a completely different set of owners.
Think about a loan origination process. I have to make a decision about whether I'm going to lend you the money. Every time I change my risk strategy, every time the market moves, every time the rates change, every time competitors change their pricing, I have to change my decision as to who gets to be lent money and who doesn't. My process doesn't change. They still get the same documents from you. They still go to the same people. They still look at them. They still check the same things. All of that's the same. All that changes is the decision making.
Think about credit card fraud. The process of that hasn't changed in decades. All that's changed is how we decide this is a fraudulent transaction. So these things have an independent existence from the processes from which they are called. The only way to manage that is to separate them. Think of them as a separate artifact. Think of them as orthogonal to the process itself.
KUMAR: And how does that show up in the systems that you build?
JAMES: Generally what it shows up as is it simplifies the process dramatically. If you think about a process where I've embedded decisioning, I typically have lots of steps in my process where I do a piece of the decisioning. So I might say, well, is your insurance policy enforced? Does it cover the thing you're claiming for? Were you somewhere where we could pay the claim? You work through these steps, and each of these steps is a piece of the decision making.
But if I say I've got a decision, which is should we pay this claim, and it's an artifact, it's a thing, then I have a much simpler process. I assemble data about the claim. I decide if I'm going to pay it. And then I act on the decision. I have this dramatic reduction in the apparent complexity of my process. I essentially outsource that complexity to the decision.
Secondly, I can change the decision without changing my process. Changing a process typically involves retraining people, retesting, all sorts of things. Companies that change their process typically have a whole room of people who run through the testing process. They run it as a separate test and they do all this evaluation to see if it's going to work. That's fine because you don't change the process all that often. But the decision-making changes all the time.
The best example of this was actually out here in California. The DMV has to calculate how much you're going to have to pay to license your vehicle. That changes all the time because politicians tinker with it all the time. The process hasn't changed since the fifties. They send you a bill, you send them a check, they send you a sticker. Process is the same. It's always been the same. Nothing has changed about this process at all, except that the logic for calculating the fees.
You see this over and over again. It simplifies your process. It makes it much more agile, because you can change the decision making independently of changing the process. The other thing it does is it actually makes it much easier to apply machine learning and AI and analytics to it. Because if you have a process, generally if you apply analytics to it, what that does is help you identify problems in the process. It helps you understand the process. But within a decision-making context, you can make decisions based on the analytics. If I predict the risk of something being late or something being fraudulent, then I can act differently within my decision-making.
That actually turns out to be quite hard to do unless you separate that from the process. We often say that decisioning makes your process smarter, simpler, and more agile.
KUMAR: That's starting to crystallize for me, the separation of the decision from the process. Almost thinking starting from the end in mind. What are the decisions we need to make, independent of the process, and encapsulating them in some way so that you can always change the decision or the decision logic.
How is this different from a business rules engine or something like that?
JAMES: Business rules management systems are really technology platforms. When you separate out a decision, there are lots of business rules in it typically. You've got rules about compliance. You've got rules about best practices and policy. Even if you want to apply machine learning and AI to that, you still have a regulatory framework or a contractual framework within which you've got to operate.
What business rules management systems do is allow you to manage that logic within that decision in a very transparent way that's easy to expose to the business and easy to manage. You can theoretically code a decision in code. The problem is that then it's not really any more agile than the process it's being called from. Just like you can code processes too, a business rules management system or a decision platform plays the same role that a business process management system does in keeping your processes out, or a database management system does. You don't have to use one. You can write code to do all these things. But they provide a set of management infrastructure and versioning and control and visual metaphors that make it easier to manage.
Most decisions tend to be very rule heavy. They have a lot of rules in them. Even if you're maximizing your use of optimization or machine learning, you generally still have policy, regulatory, or contractual constraints. It's very unusual that you can make a decision based purely on what the machine learning tells you.
KUMAR: Good point. Business analysts have probably never heard of DMN. What does the decision model reveal that maybe a process diagram, or a use case, or a user story can't?
JAMES: DMN stands for the Decision Model and Notation. Just like BPMN is the Business Process Model and Notation, it's a modeling notation for decisions. It's a lot simpler than BPMN because the internal mechanics of a decision tend to be more about the structure of a decision, and the components are simpler. We often tell people, you need to understand really at the heart three shapes and two lines, and you can build decision models. They build a very simple yet very robust model.
It's a way of breaking down a decision into pieces, then understanding what technology and what approach helps you solve each piece of that decision making, and then explicitly defining how they fit together using dependencies. So instead of saying, as in a process, first I do A, then I do B, then I do C, then I do D, we say: in order to do A, I have to decide B, C, and D. I don't really care what order you do them in, but I can't decide A until all three of them have been decided.
You build this more top-down decomposition of your decision making, which turns out to be much more effective at managing the complexity of a decision. We use sub-decisions, and you can break it down. If you've ever tried to look at a business decision and been asked the question, as many business users are, "is this all the rules you need to make this decision?", that's a more or less impossible question.
KUMAR: Yeah, it's silly. I don't know why people bother to ask it.
JAMES: Right. But if I break the problem down into thirty or forty sub-decisions, okay, we have this decision in here that says, do you live somewhere where we're allowed to lend you money? Here are the rules I have for that decision. Are they all the rules? Well, that's a relatively straightforward thing. We don't lend money in Alaska. We don't lend money in Hawaii. We don't lend money to people who are here temporarily. Okay, so those rules are it. Fine. So it's much easier, if you break it down into pieces, to identify that you're finished.
It's much easier to say, do I really need these to be rules, or can I use a machine learning model or an AI model instead? Or is it just something where I rely on Kumar's judgment and say, you know what, I don't really want to try to automate that. I want to have that one decision in the model be made by a human.
KUMAR: Interesting.
JAMES: It's also very good at identifying what data you need. Because it's a dependency network, it forces you to say, what data do I need to make the decision? Not, what data do I have? The process often has a bunch of data in it that's just flowing through the process. If I fill out an application for something, there's a whole ton of data in there that's only really needed by the last step in the process where I'm going to print you a loan doc. When I'm getting to a decision halfway through, I probably don't need all that data. Often I need like half as many data fields to make the decision. That starts to clarify which data has to be available when. It clarifies how good that data has to be, because I know how I'm going to decide with it. It often enables you to dramatically simplify the beginning of a process by saying, actually, I don't need to capture a whole bunch of this data until I've decided to lend you the money.
If you've applied for a HELOC or a mortgage or anything else, there's all this stuff they ask you at the beginning. All this huge paperwork. I guarantee you, seventy-five percent of that data isn't used to decide if they're going to give you a loan. It's just there in case they give you a loan. If they give you a loan, they will need that data to print your loan docs.
KUMAR: Interesting. Why they capture it at the beginning, I've never understood.
JAMES: I would capture the data I need to make the decision, make the decision, and then if I say yes, capture the rest of the data.
KUMAR: This reminds me of a video, I don't know, some old video from Western Digital. I think I saw it as part of some Lean Six Sigma training. They had simplified a process that took thirty days. Someone would apply for a loan or for something, and it would take thirty days, all the wait times and the takt times of that process, and where the document ended up, and finally they would get their loan. They cut that down to like fifteen minutes, because they simplified to what they needed to make the decision. Then somebody, a human, had to make the decision, and then they would get all that information once they'd made the decision. This was back in the nineties or something. The video is old, but that's what it reminded me of.
JAMES: It was doable then. The problem was we didn't really have repeatable things for it. I've been doing this, as you said in the intro, a long time. What we noticed when I was first doing it is that all we could really do is encourage people to think about the decision, to do it first, and to think about it cogently and to not get distracted by the context and everything else. That was hard to do. It was really only with the advent of decision modeling, which is now ten, thirteen, fourteen years ago, that we were able to go, oh, okay, this is a notation that lets us do that in an effective way. Before that, we were just writing requirements documents or user stories or whatever it was, and you just had to try to be focused on the decision as a separate thing.
With the advent of decision modeling, it's much more robust. It makes it really impossible for you to do anything other than to build a stateless, side-effect-free decision. You pass data in, it tells you what the decision is, and that's all it does. The modeling intention doesn't let you do anything else, so it forces you to think through it the way Western Digital did. They could do it. It was just hard.
KUMAR: Right. It does make a lot of sense. In our prep conversation, you made an observation: companies have digitized their data, they've digitized their content, they've digitized their processes, but they're still using people to make all or most of the decisions. Why do you think decision making has been the last thing to get digitized?
JAMES: It's interesting. We had a great example of an insurance company we worked with. We started there in their claims group, and they said, we have one hundred percent digital claims. We were like, wow, that's astonishing. That's great. And they're like, well, why do you need us? Then they walked through what they did, and what they meant is they had digitized one hundred percent of the claims and they flowed through the system entirely electronically. But every single claim was looked at by a claims adjuster. We were like, oh, we would call that zero percent digitization. Someone's involved in every transaction.
I think it's partly because it used to be quite hard to automate the decision if you hadn't already digitized the data. You've got to run the decision-making logic and the machine learning models against your data. If you haven't digitized the data, you are out of luck. Then there's a natural flow from digitizing the data for a transaction to making sure I can move that transaction through the steps. Once you've committed to digitizing the data, this sort of process follows from it.
But people, I think, are a little bit reluctant with decision making. First of all, they didn't feel we had good tools for thinking about it. Secondly, the tech used to be expensive, so you only really saw it being applied where the multiplicative effect really worked for you. If you take the number of times you have to make the decision and you multiply it by the value of making the decision, how much risk there is or opportunity, if that's not a really big number, it's very hard to justify the investment in the tech stack. Now the tech stack prices have come down. The price of compute has gone down. A lot of these things have changed. So now we are building decisioning systems for decisions that are only made a few tens of times a week, because we can do it cheaply enough that it's worth it. Whereas it used to be that if you weren't making the decision thousands of times a day, we'd be like, it's hard to do the ROI calculation.
To your point, I think also we figured out how to model data. As part of the relational database transition, entity diagrams, relational models, Codd and Date and so on, that made it much more practical to build data models and therefore think about how to digitize our data in a consistent way. BPMN came along a long time ago. So there's a little bit of back and forth between the way we think about the problem, the need to digitize it, and our ability to model it. That became self-reinforcing with data and then with process. We're beginning to see that with decision. We're starting to see people who think they have an AI problem realize they have a decisioning problem. More people adopting decisioning that didn't used to adopt it. There's a little bit of a chicken-and-egg problem here as well.
KUMAR: You mentioned machine learning, language models, and inserting them in the right place at the right time. What does that look like?
JAMES: This is one of the interesting things. If you follow the industry as long as I have, one of the things you notice is that when I started talking about machine learning and predictive analytics, eighty percent of models were not put into production. And here I am, twenty years later, and guess what? Eighty percent of models don't make it into production. You're like, really? We're building a lot more models. We invest a lot more money in them. We have a lot more good stories. But the percentage hit rate hasn't gotten much better.
Your question is, well, why? The problem is that people build the models forward. They look at the data they have, they see what models they can build from it, they build those models, and they present them to people who they think have a problem that model will help them solve. If you're a follower of Stephen Covey, you'd say, well, actually, you should begin with the end in mind. What is it these people are trying to do, and how can you help them with the model? We often say, look, if you're going to want to succeed with these tools, you have to begin with the decision in mind. You have to say, this is the decision these people are trying to make. This is what a good decision looks like, or what a better decision would look like. How do I help them make a better decision? Because at the end of the day, if they don't make a better decision, it doesn't matter how clever my model was.
KUMAR: How do you quantify a good decision?
JAMES: It depends on the decision. What we like to do is tie the decisions to key performance indicators that they have an impact on, or metrics. We would say, what are the business metrics? These have to be business metrics. They can't be anything else. You have a business metric or a business KPI, and you say, okay, what metrics are you trying to improve? On-time arrivals, or risk, and so on. Okay, what are the constraints on that? Well, I want to reduce the amount of bad debt in my portfolio, but I still want it below this threshold. After that, I don't really care. I just want to maximize profit. So I want to maximize profit for my portfolio of credit as long as I don't get more than a certain amount of bad debt. Classic credit.
Okay, great. Now I know how to measure better decisions. My decisions would either reduce the amount of money I make or increase my bad debt above that threshold. So I've got these metrics. But what decisions do I make that have an impact on those metrics? Well, I have to decide. If I'm a transportation company, on on-time arrivals, I've got to decide what equipment I'm going to use, what staff I'm going to use. I have to decide when they're going to leave. I've got a bunch of decisions I make that have an impact on that. If I want to move that metric, I've got to change one of those decisions. If I do the same thing the same way, I won't get a different result. I have to change one of the decisions that has an impact on the metric.
KUMAR: When you're designing these decision trees, or decisions, whatever you call them, can they sort of learn from the decisions they make based on the metrics that are being measured? And can that be automated as well?
JAMES: Yes, they can. What we see generally is that when we build a whole decision, there's a continuous improvement component we're presenting. We're generating data that you can tie to the outcomes. You can say, which decisions do I need to change? How do I need to change my model and my implementation? Within that, you can often use self-learning models, neural networks, or other kinds of self-learning models that are learning how to make their bit of the decision more effectively.
It's relatively unusual to feed the whole model into an automated learning loop, because to some extent, if you've got a decision that's complicated enough to do this, then it's almost certainly got regulatory, policy, or contractual constraints. So it can't just change to be more effective. It's got to operate inside its constraints.
A few years back, I wrote a piece with Michael Ross for the Harvard Business Review, and we talked about different ways to manage these kinds of things. There's a sort of human-in-the-loop where there's a decision being presented to Kumar. No decision will be made until Kumar presses the button. It's not a great approach for anything other than very low-volume decisions, to be honest. Otherwise, you're just going to say yes to all.
The next level up is what we call human-on-the-loop, which means you're not making each decision, but you're part of the review cycle. You see what the decisions were. You make decisions about, okay, that was good, that was bad. I'd like to tweak this, simulate that. Okay, that looks good to me, we'll put that into production. You're controlling the behavior of the decision, but one step removed. You're not part of each individual decision. You're controlling the set.
Then there's human-out-of-the-loop, where you've set boundary conditions and constraints and goals, and it just learns for itself. Those are actually fairly rare as decisions go.
KUMAR: You mean the third one?
JAMES: The third one, yes. You think about ad buying on Google and things like that. Some of these get really highly automated. I've got a budget, I've got a target, click-throughs is the only thing I'm measuring, which is very immediate. There's no long-term wait. But think about trying to do that with loans. I don't know if the loan was a good loan until you pay it off, until you see how long it takes you to pay it off. That could take months. That's much too slow to have an automated feedback loop. You often end up with proxies and all sorts of complications that are much better suited to having a human on the loop, even if I'm automating each decision.
Even when you get very high levels of automation of the decision itself, up to and including a hundred percent, generally there's a mix of self-learning models that are part of that decision, that are learning how to do their bit more effectively, and then a human-driven review loop of the decision as a whole.
KUMAR: That makes sense. But in my brain, it seems like if you built each model to optimize the decisioning based on some metrics that it has access to, the whole is going to improve. Whatever the business KPIs that you're measuring. That would be kind of where you want to be. Sort of human on the loop, but not necessarily in the loop for sure. That frees up the humans to do other things, but also this is now a system that's consistently and constantly improving itself and how it operates.
JAMES: We often say that there are three things that really make this work. The first is, you have to begin with a decision in mind. You have to be decision-centric. You have to think about decisions, model decisions, understand the decision as a thing.
The second thing is, most decisions involve a mix of technologies. You have to be willing to say, this decision needs a lot of rules and a little bit of machine learning. This one needs a lot of machine learning, a little bit of rules. This one needs a large language model. You have to pick and choose based on your understanding of each individual decision. There's no one-size-fits-all.
The third essential piece is continuous improvement. Our experience is decisions are almost never static. The competitors change, the market changes, the environment changes, regulations change, court rulings happen, customers change their behavior. If you don't watch that and continually tweak your decisioning, it gets worse and worse. So it's really important to build that in.
KUMAR: You're absolutely right. I'm going on a different tangent here. Your earlier statement, you know, build with the decision in mind, or start with the end in mind. Doing so simplifies things quite significantly. For companies that say our data isn't good enough to do this, your answer would be what?
JAMES: How do you know? You almost certainly haven't ever sat down and attempted to describe the decision-making in any kind of detail. You're asserting that your data is not good enough to support a particular decision-making approach, and you haven't really thought through it.
First of all, the data that's bad quality may not actually make any difference to the decision. We build a lot of decision models, and I would say every time we build one, there's data we're told at the beginning is absolutely categorically needed for the decision, that we can't find anywhere in the decision model that makes any difference. And there's almost always data that we need to make the decision that wasn't in that list. So we change the view of what data is necessary to make the decision.
The second thing is, if you don't know what the decision is, then how do you know whether the data is good enough or not? Sometimes we build decisions where a particular field value might have dozens and dozens of cutoffs. Between A and B, between B and C, between C and D, and there's dozens and dozens. If that data is not accurate, that's obviously a problem because you've got fifty segments based on it. But sometimes the decision is, if it's less than fifty, do this. If it's more than fifty, do that. Well, okay, how accurate does that have to be? I can say it's probably less than fifty or probably more than fifty. I might be able to do that a lot more often than I can say exactly what it is. There's some data quality person saying, but we need to know if it's forty-eight point six. I'm like, actually, no, I just need to know if it's less than fifty. I don't even care how much less than fifty in the context of this decision.
We're always like, build the decision, understand the decision first. Once you understand the decision, you understand what analytics might help you, what machine learning might help you. You also have a sense of how accurate each piece of data has to be, because you understand what you're using it for.
KUMAR: Interesting. Certainly, it seems like a no-brainer, a way to think about decisioning and the things we build. A lot of operational work in large companies, including where I'm currently at, runs on email, spreadsheets, human interfaces between systems and copy-paste information from one to another. I'm wondering what you would say to a place like this that wants to automate some decisions, or put a more modern interface around the decisioning, on these old legacy systems.
JAMES: What we have found is there's a couple of approaches. First of all, we're doing a lot of work, as you'd expect, to convert legacy code and legacy systems to modern decisioning platforms. If you have a system that works, however badly, you can move it to a new technology stack. That's definitely an option.
But the other one, the one you specifically asked about, is what happens if I don't really have a system? I don't really have automation today, and I still want to think about the decision-making. What we've noticed is that large language models are really good at reading documents, ingesting unstructured information or less-structured information.
What we have found is often possible is you can define the decision-making and automate that decision making. Now you've got essentially a decision agent that's got an API. It needs a certain set of data, and given that data, it will make a decision. Then you can use AI to explain that in natural language. So now I've got an API, I pass data to it, I get a natural language explanation of the answer back.
It turns out you can train models to ingest your documents and pass that data to the decision agent. I know what data I'm looking for, because the decision engine needs certain data. I can trawl through a very long document looking for those bits of data. I need this, I need that, I need these things, I need a list of these, and see how much of it you can find.
We had this the other day with a commercial insurer who said, we've got this. You built this decision model for us, you automated it for us, that's great. But our underwriting system simply doesn't have that data in it. It's a crappy old system. We just don't have that data. So we can't set it up to call the engine. I said, well, but most of the customers calling you have a competitive quote, or they have an existing policy. That's a PDF. You could easily build something. You could build an agent, a framework, that says, okay, upload a document. I will scan through that document. I know what data I'm looking for because I know what data the decision engine needs. I will find all the data that matches that decision structure, the input structure, that I can find in this document. Then I'll call the decision agent. It will run the logic, the machine learning models, everything else against that data. It will do the best job it can of making a decision, which might be complete, might not be. Then I'll turn that back into natural language and say, okay, I looked at that document you sent me, here's how much of the decision we could make, described in natural language, and here's why we couldn't finish the decision if we couldn't finish it.
Now you've got something that's automating that decision in a very effective way, even though the data, frankly, is crap. This combination of AI interacting with humans and gathering data from documents and so on with automated decisions is a really interesting space. It creates a real opportunity to provide more automation without having to first automate all the data. I think that's an interesting area where we're just starting to see people realize this.
KUMAR: It's a whole, I can see so much opportunity there with a lot of companies, older, big companies that have a lot of old legacy systems that are like, oh my god, we've got to modernize. We've got to move it to a platform. We've got to fix our data.
JAMES: Yes. One of the interesting things here to remember, of course, is if I've got a process today where someone emails me a document and I decide what to do and I send an email in response, then by definition, that email and the attached documents contain enough data to make the decision.
KUMAR: So it's just applying intelligence to that.
JAMES: Yes. Exactly.
KUMAR: This is definitely a human-in-the-loop type of system.
JAMES: That would be human-in-the-loop, yes. Because the problem is, there's data in the environment. There's data in people's heads. There's data on yellow stickies around the computer or on the wall. That isn't going to come out of that email and that attachment. Obviously you can whittle away at that. But to begin with, to some extent you want to add value first and then encourage people to change things.
It's interesting because it used to be that for sure you had to digitize the data first before you could do anything else. Now large language models are really good at reading documents. So now what matters is, do you know what data you need out of the documents? If you know that, now you can extract it.
KUMAR: Yeah, definitely. I can see so many ways to apply these concepts to where I am now, and certainly even for my own work. Anything I haven't asked you about the work that you do that you think would be relevant to bring up?
JAMES: One of the things I was going to say is we get a lot of questions about AI. Why can't I just replace these decision engines with AI? Pile up all the requirements documents and the policies and the regulations that are supposed to be followed, and tell the AI, make a decision of X, follow all those policies and guidelines.
To some extent, obviously you can. You have two problems. First of all, it may not make the same decision twice. It will vary somewhat because AI is probabilistic. Secondly, it can be quite hard to prove to someone, a regulator, an internal audit team, or even a customer, that you followed the guidelines. Even if you look at explainable AI, mostly what it does is it comes up with a plausible explanation of how it might have decided. It doesn't really know how it decided. So this is a little bit tricky. People like the financial services regulators, the CFPB and people like that, have already said, we're not making any exceptions to the guidelines just because you used AI.
We think there's tremendous opportunity in essentially using AI to munge all that sort of requirement and generate a more formal definition of the decision making. Then run that as a separate artifact that's very prescriptive, and say, okay, I've made the decision, and then use AI to explain it. So now you're using AI to build the first-cut version of that, so it's quicker to build. You're using AI to ingest documents because that's where your data is. And you're using AI to explain the results. It feels to a customer or to a call center rep like you're using a chatbot. I put unstructured text in, it was built fast, and it gives me an answer in natural language. But under the covers, it's got an exact prescriptive log of how it decided what it decided, what AI it used, what machine learning models it used, how it used them, and everything else.
It's going to be an interesting mix for AI. The people we see succeeding with AI regard it as another technology they can mix with their machine learning for predictions, with their rules, with their decisioning. They regard it as another tool in their toolbox. It's got some real strengths. Then the people we see basically flailing around are the ones who are like, I'm going to throw the baby out with the bathwater and start again with large language models. We're like, why would you do that? You already know a lot. There are some things that they make dramatically better. And other things, they don't.
We did some research the other day on different kinds of coding tasks. The research is pretty clear. Some of them, it makes a big difference, like test data. It's really good at generating test cases and everything else. Refactoring code, there was some evidence that it was harder to refactor the code with an AI assist than it was without. So it's not a panacea. As always with new technology, this one is dramatic. It's going to make a big difference. But I don't think it's going to be a blanket replacement for all the things we're doing.
KUMAR: It would be kind of silly to try to put everything into an AI and expect that it's going to come up with the right decisions a hundred percent of the time, or even consistent decisions a hundred percent of the time. For companies that are governed, they've got policies and things to follow. You wouldn't entrust one person to do that either. That's why you have rules. That's why people work on specific areas of the company. That's why you have four eyes, why you have review cycles, all these things. All those things don't go away. I think you start to see legislation that's starting to say, like in Canada, there's one that says if a decision affects you, then you can ask them to say what data did they use to make the decision and how did they make the decision. You have to be able to generate a letter that says, here's how we decided. I don't think an AI, at the moment at least, is going to give you that. You need a little bit more structure to that.
JAMES: Absolutely.
KUMAR: We're running out of time. I just have one more question for you. You helped build this category from almost nothing, decision management. What's the single hardest thing about convincing the world that this new discipline exists?
JAMES: It's the D word, decision. On the one hand, everyone knows what decisions mean. It's not like it's a complicated word. The problem is that everyone knows what decisions mean. They think I mean the decisions they make, and generally I don't.
What I mean is transactional operational decisions, the high-volume decisions you make about a single transaction or a single customer, a single account, a single vehicle. That's a hard thing. They're used to making decisions about the fleet or the portfolio. I'm not talking about that. I'm talking about making a decision. Should I pay this? Should I give this person this loan? They're thinking, loan decisions are what's our loan policy. I'm like, you still have to make that decision. I'm just trying to apply it to this transaction.
So often those decisions are poorly defined. They're missing. They're not understood. When you go to a website, it has to decide what to display to you. If you've logged in before and you have a cookie, it should decide differently from if I go as a new user. But it doesn't. That's a missing decision. There are these places throughout organizations where there's these transactional, interactional decisions that could be made, that aren't made, aren't well understood, and are fundamentally different from how people think of the word decision. That's the biggest problem.
KUMAR: Wonderful. Decision management. I've been thinking about it for a while. This is our first conversation. For those of you listening, the book that James Taylor wrote is called Real-World Decision Modeling with DMN. That's going to be in the show notes below. Thank you so much for joining me today, James. I really appreciated the conversation. I'm sure the audience that listened to it live and those that are going to listen to this later will too.
JAMES: Well, thanks very much for having me, Kumar.
KUMAR: Of course. Thank you. Bye-bye. See you all next time.
