The Meridian Point - Episode 174
AI Is Not a Magic Bullet: What Enterprise AI Gets Wrong
Guest: Ashwini Kumar
Host: Kumar Dattatreyan
---
Kumar: Hi, everyone. Kumar Dattatreyan here with The Meridian Point. Today we're joined by Ashwini Kumar, a product manager and AI practitioner who has spent the last six years working at the intersection of machine learning and enterprise software, long before generative AI made that cool. From manufacturing and energy to healthcare consulting to major telecom, Ashwini has been in the room when companies make their big AI bets. And he's watched those bets play out in real time. What makes him a compelling voice on this topic is that he doesn't just understand the technology. He understands the people, the workflows, and the organizational dynamics that either make AI implementation succeed or quietly fall apart. He's here to tell us what's really happening inside enterprises that are racing to adopt AI and why the gap between the boardroom promise and the frontline reality is a lot bigger than most companies want to admit. So without further ado, let me bring Ashwini on stage here. Hi, Ashwini.
Ashwini: Hello.
Kumar: Great to have you on the show. I was just mentioning to you right before we started this recording that the picture behind you is the exact same one that my mom has on her wall. Coincidence, or maybe it's meant to be.
Ashwini: They're mass produced in India, to the point, yeah.
Kumar: Exactly. So Ashwini, you've been in AI and machine learning for about six years now, which means you were doing this work before ChatGPT and all of these chatbot tools came out. Of course, ChatGPT suddenly made everyone an AI expert, right? Or they think they are. What's it been like watching the rest of the world catch up, and what do most newcomers to this conversation not understand?
Ashwini: Well, it was interesting being in machine learning and then seeing ChatGPT come out. It really created a lot of buzz around what it could do. I kind of sometimes consider it the front end of AI. Something that anybody can talk to. Machine learning before that had a very specific purpose. It's used well for recommendation engines and things like that. It's used pretty widely.
But then, things like Netflix run on machine learning. Because of patterns and neural networks and things like that. Manufacturing processes. We worked with a roof tile company of all things, because to make a roof tile is a very complex mixture of asphalt and other materials. They always have a recipe they have to mix just right or the mixture gets thrown out. So in chemical manufacturing, it's very key to have very precise, predictable outcomes and measurements.
Back then it was more about, and it's still the case today, interpretability and trusting the data. It was a little bit easier to trust machine learning, but did it produce data that they would understand? Because it's the same thing today. It's about how they do their work, how they do their experiments. Going back to machine learning in manufacturing, how they do their chemical processes and does that correlate to what the models are telling them. Sometimes it did, sometimes it did not. It was hard to make that correlation sometimes, especially with oil companies. Very complex systems.
We also worked with mechanical engineers on CFD and designing wings, trying to optimize wing design simulations. That was three-dimensional data, mountains of data. You're putting that through a model that can run the simulations, which would take days to run. Then they have to go back and adjust this and that. It was expensive to run that amount of data through a model. It was not easy.
Kumar: So what's changed since then with ChatGPT and, the way you put it, the front end to this?
Ashwini: Yeah. I mean, you can use machine learning and then have LLMs interpret what the models are telling you if that's their use case. But I think it's brought AI to the public. Not machine learning per se, but AI. It's brought it to the public to interact with it and understand it and use it in a way that they can understand. That has really brought a lot of money into it too. More money than, I don't have the numbers in front of me, but a lot of the VC money went into LLMs after that. Machine learning didn't get put aside, but the money wasn't flowing that way anymore.
Kumar: Right. Billions and billions of dollars in investment. I think we're reaching almost a trillion. Who knows?
Ashwini: Yeah, for sure.
Kumar: So you've been in rooms where major enterprises make the decision to go all in on AI. In your experience, what does that decision look like and where does it start to go wrong? If it does go wrong.
Ashwini: Well, I'd say it's in its juvenile state. The AI, as far as what we know of it today, is based on LLMs generally. And companies want to deploy this pretty rapidly. There's that pressure at the executive level. There's the pressure to implement it, to be more productive, increase efficiency.
Now, the other thing I've noticed is that it becomes sort of a pet thing for some executives. It becomes a thing that they can put on their pedestal. Like, I implemented AI. But they push it sometimes so hard that they don't think about what problems it's solving. Is it solving those problems? What are some of the outcomes? Is it being successful? They've invested so much money into the project that it almost cannot fail. It might fail, but they don't want it to. Sometimes they tend to fudge the numbers a little bit to make it look good.
Kumar: Interesting. In our prep call leading up to this conversation, you made a point. When companies employ AI to existing manual workflows, all they're really doing is automating a human process that may have already been broken. Can you unpack that for us? What should they be doing instead?
Ashwini: Well, first and foremost, you shouldn't apply an AI process as a blanket way to solve what you're doing as a human workflow. You should take a look at the workflow and see if there are things in there that can be first automated. Look at what may be a tedious, repetitive task. Then look at what may not need to be done. Maybe there's a human involvement in a step that really could be done another way.
A lot of times what I've seen is they try to use AI to automate what the human was doing, but that may not always be the best approach. Sometimes the human does extra things they don't need to be doing. Sometimes these things can be shortcut. There might be things you need to add to it. A human might be jumping around in different systems to get information. An automation or AI doesn't need to jump around. Well, it does jump around, but much faster. So there might be some steps that a human does that just don't make sense.
Kumar: That's interesting. A question popped in my mind. Is it conceivable or inconceivable that an AI would make a different decision given the same circumstances?
Ashwini: It is possible, yes, that it could make a different decision. There is a technology I haven't implemented yet called Semantic Kernel that looks for bottlenecks. It's an agent you can build based on that. I haven't implemented it, I've just read about it. In theory it sounds interesting. It can look in your workflow, as long as they're digitized, and look for bottlenecks and try to fix them.
Kumar: Interesting.
Ashwini: If you make it autonomous. But I think you'd probably want a human in the loop for that.
Kumar: Yeah, that is interesting. The reason I ask is that for certain types of processes that are repetitive but rely on really strict rules on how to apply them, you may not want to use an AI. Because if it can make a different decision based on whatever it comes up with, then for things like a decision on a loan or a mortgage or something more critical, maybe you want the human in the loop to make those decisions.
Ashwini: Generally you do. You want to make the process to gather the information much quicker and then let the human make the ultimate decision. Because there are nuances, there are cognitive things that a human must do. All of the processes should augment what a human does. Not necessarily always replace it. Maybe the tedious, repetitive stuff, yes.
Kumar: Yeah, that makes sense. The point I need to make is a lot of times you need to look at what can be automated first, right? Because implementing AI is not cheap. It's expensive.
Ashwini: Yeah, I read a report that some company, for coding, Claude Code or whatever it was, they're going back to making developers do it manually, the old way, just because it was too expensive to use these tools. It was Microsoft, yeah. They're pulling the licenses for Claude Code.
Kumar: Interesting. And the thinking is that it's just too expensive? So until costs come down, maybe that's not the right place. Who knows?
Ashwini: Yeah, but I think they want to push Copilot too. Because they're a paid company. So I think that's part of it.
Kumar: Yeah. I think this industry is changing so fast, maturing so fast, that it's hard to predict where it's going to be in a year, even six months.
Ashwini: Yeah. Did you hear about Starbucks? They recently pulled their AI implementation. I just read about it.
Kumar: I did not.
Ashwini: Yeah, it's interesting. They were implementing AI for inventory management and things like that, and it just fell off. Starbucks has so many locations. You mess one thing up in inventory, it can screw up the whole chain.
Kumar: Yeah. I have an interview coming up with another gentleman who is in decision management. That'll be interesting to see what his take is on AI in the decision loop. How decisions get made.
In our prep conversation, you described a situation where a sales rep found a data error in an AI platform. And because of that, they went back to their old system. This new AI thing just became a tool that was essentially dead to them. How do you rebuild trust once it's broken?
Ashwini: It's difficult. The way you build trust is they have to use the tool you're implementing in a way that's easy for them. A lot of sales reps that I worked with on this one particular platform, they had a way of getting the information they needed pretty quickly. They knew where to go, they knew where to get it, and they were pretty fast at it. Now, AI was faster, but there were times when they would look and the data was wrong. Either the data wasn't structured correctly, and it's also based on RAG. RAG still drifts a little bit sometimes. It'll give you a different answer every time. The data might be the same but it might phrase it differently.
Kumar: Can you explain what RAG is for the audience?
Ashwini: RAG is Retrieval-Augmented Generation. It's basically a platform where you ingest your data for the company. It's used quite a lot in most enterprises. Then you use the LLM to query against the data that it ingested. It uses its own model training to understand that data.
Kumar: And what do you mean by drift?
Ashwini: Sometimes the model might look at, depending on how you trained it and how you designed the data you fed it from the RAG, it might give a different response to what you expect. It says this thing one time, and then through a couple of months you've done some adjustments to it, and then it goes in another direction on how it answers. In the industry, it's commonly known as overfitting, so it drifts a little bit. You don't want it to do that. You always have to monitor it and make sure it's not drifting. You're always adjusting it. These things are not one and done.
Kumar: Interesting. The promise of RAG was that it wouldn't hallucinate.
Ashwini: Right. But it still does a little bit.
Kumar: Well, humans hallucinate all the time.
Ashwini: Oh, totally. Yeah, yeah.
Kumar: But RAG still hallucinates a lot less?
Ashwini: Yeah. Machine learning hardly at all. It doesn't really hallucinate, but it may not have the full confidence. It has maybe ninety-five percent confidence. And then you have agentic RAG, which is much, much better. You have agents that work on top of the RAG that are designed for specific tasks. When we implemented agentic RAG, we found that it was much more useful for what we were trying to do. We were generating proposals. It was much better at writing the proposals. It was much better at formatting them. It was much better at writing them in context.
Kumar: Is that because the data was a much smaller subset of what the agent needed to do its work?
Ashwini: Well, no. From what I understand, each agent, given a specific task, is able to focus just on that part. Like an agent that focuses on formatting. An agent that focuses on the context of the RFP. An agent that focuses on designing the final format of the proposal. And then you have an orchestration agent that tells each agent what to do.
Kumar: So I guess success in AI projects comes down to limiting the scope and not trying to do too much?
Ashwini: Yes. Generally, a lot of these projects try to tackle too much. I'm not saying that broadly. I'm just saying in my experience. When you have a very small task you're trying to focus on that you can measure the impact for, it's easier to measure that.
As my example, if all you're trying to do is generate a proposal against an RFP, it's easier to measure how long it took you before. How good is this proposal compared to the ones we used before? Things like that. So that would be the ROI of that AI implementation. It should be directly measurable that it helped reduce the time to produce whatever the artifact was or the value that you're getting out of it.
When you make it too big, it's hard to attribute that. It's hard to capture the KPIs, what you're trying to measure. It's really difficult.
Kumar: It's always a difficult thing to capture. So when you go in, when you're helping a company that's thinking, okay, we need to implement AI, everyone else is doing it, we've got to do it too. How do you help them think through use cases that are good to pilot an AI project on?
Ashwini: So what you just said is a very common problem. They want to implement AI because their competitors are implementing AI. They're in a rush to do it. So you have to help them step back to understand what the problem is that users are experiencing.
Typically what I've seen is they tend to make assumptions on what their teams are experiencing, like the team of sales reps, that may or may not hit the mark. So we need to be very careful. I always try to push, let's talk, let's try to dig into some of these problems that we're trying to solve. Let's dig a little deeper before we start doing this.
I've seen it where they just say, let's come up with these five solutions and let's go. Let's assign teams to them. And then the teams build the solution and don't even know what the problem is they're solving specifically. And then the five different solutions don't even talk to each other.
Kumar: Yeah, that's reminiscent of just any kind of project, right? We do that all the time. But it's expensive with AI. It becomes more expensive with AI because of all the infrastructure you have to invest in, the tools you have to invest in. I can certainly see that.
That's one of the things you talked about in our prep conversation. You described a scenario where a company built five different separate AI platforms, none of which talk to each other. I guess it mirrors the structure of the organization. Maybe they built it for each division. I don't know how that worked, but that is maybe a relatively common thing. We've got to build an AI for this and an AI for that, and don't really think about how to connect them together.
Ashwini: Yeah, they think of different problems and then they think of a different solution set for each. But some of it sometimes is based on assumptions on how they're solving it for their team.
Kumar: So what does that tell you about how the organization thinks?
Ashwini: Well, they're not a true product-oriented company. I always come back to the product thinking. What is the problem? How are we solving it? Jobs to be done still applies. It's still a huge thing. It's big in AI because we're trying to solve a particular task and make it much easier to do than before and much more productive.
Kumar: Yeah, for sure.
Ashwini: But I think, and this is kind of a broad statement, the bigger the company, the harder it is to implement. Because you're trying to do it company-wide and it just becomes much more daunting. And then when your competitors, you're reading about them doing it and supposedly having good results, who knows? You don't want them to beat you at it. And you want to look good.
Kumar: Yeah, absolutely. So there's this data out there that ninety-five percent of AI projects fail in some way or fail to deliver meaningful returns. From what you've seen, does that track with your experience?
Ashwini: Yeah, I'd say it's a little higher than I would expect from what I've seen at the major companies.
Kumar: So what would you say is a way for companies to get more honest with what they're looking to achieve? I guess we've kind of talked about it. Break it down into smaller chunks and smaller pieces, perhaps. Anything else that comes to mind?
Ashwini: Well, I think they need to think of AI as another solution to the problems, not the solution. It's another tool in your tool set. And the most expensive one. But don't forget about automation. Don't forget about triggers. Don't forget about other solutions that may work also. Don't just apply AI to it because it sounds cool.
Kumar: That's a good point. Use it judiciously, especially because it is expensive. But it can really enhance and augment the products that you're building, the services you're building, if used for the right purpose in the right context.
Ashwini: Yeah. A lot of companies, we're all still learning. We're all still adapting. We're all changing. That ninety-five percent is definitely going to go down.
Kumar: Yeah.
Ashwini: But it's going to cost a lot of money to get there.
Kumar: Yeah. Like any new technology that goes through its phase of adoption. So if I remember, you're also building your own AI-powered product on the side. A proposal generator. Is that correct?
Ashwini: Correct. I'm building two now, but yeah. I'm building a system that takes RFPs and writes proposals against them. I've worked for a couple of consulting companies and some of the directors, that's all they did. Write proposals all the time. Especially the bigger consulting companies.
Kumar: So what are you learning from this experience?
Ashwini: How agentic RAG really makes a difference. When we put the agents in and we did the agents correctly, it made a huge difference. We're able to generate the proposal in one run. And then you don't have to prompt it very much after that. Currently, if you go into Claude, you can generate a proposal, but you still have to prompt it each section, each paragraph. But we've got it to where all our agents are doing all the work. And they generate generally pretty nice proposals. But you're still able to prompt against it and refine it.
Kumar: Do you find that you have to make final edits before you send these proposals out?
Ashwini: Yeah, you generally do. A little bit. It just depends on what you're looking at. But generally, you should know your industry. And if you know your industry, you do tend to have to glance at it and adjust a little bit. We're getting better and better at it depending on the industry. And we're refining it more and more. The thing is, there's only about four of us and we're able to develop a pretty powerful tool.
Kumar: That's nice. What would you say the ROI is? If you have to do these proposals by hand compared to using the system?
Ashwini: Well, typically with the consulting companies we're working with, one proposal can take two weeks generally to write because it's got to get passed around. We're able to generate the first draft probably in a day, two days. And then do a review in three days maybe. So we've cut the time down significantly. And if you can do the proposal much faster, you can do more proposals. And hopefully generate more revenue if you win a proposal.
Kumar: Yeah. When I worked on the consulting side, I remember proposals would take one or two weeks at companies that were really good at it. The companies I worked at, it would take a month to write a proposal and there were tons of reviews, especially for government work. So it would take a month at least to get the first draft to go through review. Something like this, if you can get a first draft in a couple of days that can start to go through the review process, that could really be something.
Ashwini: Yeah, it's a huge time saver. I worked with a defense consulting firm. My director, the guy above me, that's all he did. Just work on proposals all the time.
Kumar: Yeah. That would be such a boring job.
Ashwini: Yeah. Something that would be good for AI.
Kumar: Yes. It's a good application of AI. It's time-saving. So this spurred a thought in my head. Have you sort of developed a set of characteristics of what would be good for AI and what would not be so good for AI?
Ashwini: Yeah. It's just been learning for me, just like everybody else. What I've seen work well, what I've seen probably doesn't work perfectly or great. In theory, many of the use cases I've seen should work pretty well. But when you're dealing with legacy companies, however you want to phrase that, that have processes and workflows in place that have been around for twenty, thirty years, trying to all of a sudden within a year apply AI to all those systems and change people's habits is tricky to do. It's change management.
I've noticed that when you implement AI, there's always a top twenty percent of people that are just all about it. They're super tech savvy. They'll try it, they'll jump on it, they'll use it. Because you must have engagement on it. If you don't have engagement, implementation goes flat, it dies, and there's no ROI.
You have that top twenty percent, and then you have the middle which is indifferent. They're okay with it, they use it a little bit. And the bottom that doesn't want anything to do with it.
Kumar: Yeah, that's about right. All right, so we're getting towards the end of our time together. Anything that I didn't ask you that you'd like to share?
Ashwini: Yeah. One thing I know is that in playing with Claude Code and everything like that, things are changing. Like you said, things are changing so fast. A year ago, Claude Code was not what it is today. Not even six months ago. And I'm actually loving playing with it.
I think the hard part with AI is getting it to work, and the hard part is getting people to trust and rely on the data. I think that is a key thing. But I always sum it up as the reality of AI and the hope of AI. There's still a gap there.
Kumar: Yeah, I'm sure that will persist. Reality is going to move further and further to the right in terms of what we can do. But there's always going to be something that we're hoping for that it doesn't do yet. So that gap may always be there. Maybe it'll narrow in time. That's a good point you make.
I thought I would end with a couple of fun questions, if you're open to it. Say you're at a technology conference. What's an AI hot take that would get you in trouble?
Ashwini: An AI hot take? Something that would really get them to jump on me? Interesting question. I would say, kind of what I've mentioned before, I would say something like: everybody believes AI is a magic bullet, but it's not. It is not the magic bullet we all think it is.
Kumar: That would probably give you trouble with some, but not with others.
Ashwini: Yeah, probably not.
Kumar: Good. So say a C-suite executive comes to you and says, we need an AI strategy. What's the first thing you would tell this person?
Ashwini: I would go back to my product thinking. What is your vision of your AI and what do you see it doing? Let's talk about that first. It goes back to product vision. It's almost an AI vision. Get them talking about it, because they may be very lofty, they may be spot on, or they may be somewhere in the middle. You need to see where they're at first and how they see AI in solving the problems they're trying to solve.
Kumar: Yeah. Before you can build this strategy, you need to understand what the vision is, or even, do they have one?
Ashwini: Yeah, they may or may not. Their vision or their AI vision, I call it an AI vision. It might just be that they're trying to keep up with the Joneses. Or they're like, we have to do something with this but we don't know what that is.
Kumar: Yeah. And the industry you worked in, telecommunications, they're very competitive.
Ashwini: Most industries are. I think every company doesn't want their competitor getting ahead of them in AI. Which is why there's so much money going into it.
Kumar: Yes. All right. Well, this was fun. Thank you so much for joining me today. Hope you enjoyed it. I did learn something new.
Ashwini: Thanks.
Kumar: Thanks again. We will talk soon. I'm sure there'll be something new coming out in two months that we'll talk about.
Ashwini: Yeah, I'm sure. Because the industry changes all the time.
Kumar: All the time. All right, thanks for watching. Bye bye.
Ashwini: Thank you.
