AI at work: When competence becomes an illusion

Host: [00:00:00] Welcome to Talent Talks, the L&D podcast about the future of work and the talent driving it forward. Today, learning leaders are expected to do more, do it better, and do it faster than ever before to strengthen teams, scale training, and to show results. Because when learning works, businesses move forward.

Together, we’ll explore what that really means for L&D in growing organizations, what to rethink, what to refine, and what to double down on. I’m your host, Gina Lionatos, and this is Talent Talks.

Talent Talks is brought to you by TalentLMS, the easy-to-use training platform that helps growing businesses launch faster and see results sooner. [00:01:00] Learn more at talentlms.com

On today’s episode…

Josh Cavalier: “It’s hard work, there’s no doubt about it, because things are very disruptive right now. But if you can acquire those attributes and apply them properly, there’s just so much potential moving forward. I do believe we’re in a golden age of performance and learning, and we’re gonna see some really amazing things as we move into ’27.” 

Host: Is AI making our teams more capable or simply just doing the work for them? My guest today is Josh Cavalier. He’s an AI and L&D expert who’s spent decades helping organizations rethink how people learn and perform at work. Together, we explore what happens when AI becomes part of almost every task we do, from redefining the ways we measure performance to learning how to spot the potential for knowledge gaps.

Stay with us.[00:02:00] 

Host: Josh, welcome to Talent Talks, and thank you so much for being with us today.

Josh Cavalier: Yeah, it’s great to be here.

Host: Really excited to jump into all things AI in L&D. But before we get into AI specifically, Josh, I want to start with the idea of learning debt. Now, I don’t know if it’s a term that you’ve come across of late, but learning debt is essentially what happens when people have to keep performing and moving on without having the time or the opportunity to build the knowledge and the skills that they need.

And over time, of course, those gaps can really start to add up. In a recent study into learning debt here at TalentLMS, we found that 59% of employees are using AI for tasks that they were never really trained to do, and half say that AI helps them complete tasks even when they don’t [00:03:00] understand the process.

Does this surprise you, Josh?

Josh Cavalier: Not at all.

Host: What does it tell you about the relationship between AI-assisted performance and genuine competence? Well, what I do know is that people just wanna get the work done in whatever way possible, and sometimes that doesn’t mean taking on long-term knowledge and getting things as you need it in the moment of need.

But there is something to be said about auto-completion and the allure of doing deep work and gaining that knowledge versus just doing, what in effect is window dressing, and being efficient but not necessarily effective. And I think that’s like one of the issues with AI is that you can have so many associates drive towards work completion.

They’re getting things done, but in essence, there are two [00:04:00] things. One, they’re creating friction for the business, and two, they’re not really gaining that knowledge that can be applied down the road to address business problems, to think through potential opportunities, and essentially they’re just auto-completing.

And that is also gonna be an issue for new hires because with new hires, they can look extremely productive, but they are not understanding anything about the reasons why we’re doing the work we’re doing. So yeah, it’s very… On the surface it looks like there are a lot of advantages to leveraging AI, but the undercurrent is there’s a lot of friction, and there’s a lot of risk that’s being taken on.

Host: I think it’s really a matter of where in the timeline the friction happens, right? It can happen at the beginning when you’re doing something for the first time and learning your way through it versus removing the friction at the beginning and, you know, getting some automated assistance, but then [00:05:00] down the track creating the friction, right?

And it’s really a matter of what’s harder to manage down the line.

Josh Cavalier: Yeah. Exactly. I mean, there’s nothing wrong, especially for a new hire- them understanding a little bit deeper level about how we do things here and versus somebody that’s been at the job for 10 or 20 years to really this, they wanna get their work done.

And maybe it’s a new system process, or it’s a new standard operating procedure, and they just, they wanna keep on moving things forward, and then AI’s wonderful for that. But again, I do have reservations for new hires or people that are just getting started in the industry within their domain of knowledge, and having systems just auto-complete without really having a mentor or a coach to go in and show that individual the reasons why.

You know, why are we doing the work the way that we’re doing it? So that they understand and have perspective, so when things break down or there’s new opportunities, they know how to address it properly. [00:06:00]

Host: So just following on from that, is being able to evaluate, challenge, and improve an AI-generated answer becoming perhaps as important or maybe even more important than being able to produce the original answer yourself?

What’s your take here?

Josh Cavalier: Well, each of that is a craft. So can you go ahead and wield the tool properly to give you an output that hits a quality floor? And then as an expert or a professional, can you evaluate that from the machine to make sure that the work moves forward? What I do know is that somebody has to take responsibility for it, and typically when an AI fails, it’s gonna be a leader who is managing the individual contributor.

So leadership has a vested interest in making sure that the people who are doing the work understand the risk, the responsibility, the ability to go ahead and have judgment and discernment [00:07:00] and to, you know, leverage these tools properly, and to raise a concern if there are issues, with quality.

You know, and I know I do have reservations because as we, the technology moves forward, automation is really driving hard in a lot of workflows beyond augmentation, you know, to where you have massive human in the loop. We’re just talking about work completion at scale. And there’s been some recent announcements, especially this week in our industry, on the tool side to where this is becoming evident and that IDs are, you know, essentially becoming orchestrators as opposed to creators.

And so you really gotta have your game on because now you’re leaning on a vendor and what their outputs are. You know, is that gonna do the job? And without having all of that background, without having, you know, years and years of experience, you don’t know what the answer is. And so you’re gonna go ahead and take that output at face value and just send it [00:08:00] along.

And again, that’s gonna cause some issues downstream.

Host: What do you think would be the best kind of ways to encourage the development of evaluating, challenging, and improving those AI-generated outputs in your experience? Have you found that there’s something that so far is working really well? Do you think you have to make a jumbo mistake in order for it to become a learning? It is a great way to learn.

Josh Cavalier: But we wanna minimize that as much as possible. I really think it’s a matter of getting your peers involved and having them cross-examine the work, ’cause that is a learning moment right there. Not that we have a lot of time to do that, but as workflows collapse, you can still have the rigor involved there.

Obviously, it’s more difficult if you’re a solo person or a solo entrepreneur in this field, but with teams as the tools and as the architectures begin to be more robust behind the firewall, and you can start doing work at scale, that’s [00:09:00] a time to actually take a pause, come together as a group, evaluate outputs, evaluate prompts, check out the models, and see is this valid?

Like, can we, like, lean into this? Not only just lean in, but an opportunity to, you know, advance in some new and interesting ways with performance and training experiences that we’ve never had before. So yeah, I think it comes back to people, you know, working together to move everything forward.

Host: Yeah, that’s a great point, I think.

It’s just the challenge of finding the time, isn’t it? And perhaps, perhaps the trick there is a small group of peers where, you know, it’s more of an exchange rather than, you know, two peers helping, helping out another peer in that instance. Maybe it’s a bit more of a collaborative exchange where everybody’s kind of helping each other sense check, sanity check, and I guess really evaluate the output of each person’s work as an example.

Josh Cavalier: Yeah. And that can be intentional. You know, and I talk to leaders about how they should show up, you know, for their [00:10:00] individual contributors with AI. It’s about creating safe spaces. It’s about creating environments to where, whether it be on a Friday afternoon, you get together and you talk about the work that’s been completed by AI, you try some– you share some prompts, you all take a course together and you debrief, whatever the case may be.

This technology is moving so fast. One week you’re very skeptical of the quality of those outputs that AI’s producing, but then a week later everything changes and you have workflows that are collapsing. But you have to pause and take a moment and measure, well is this really a benefit?

Because there’s still a supervision tax. You know, humans still have to evaluate. And so just because you have more content that you’re creating, somebody still has to rubber-stamp it. And so are you generating more friction and more work for an individual that they don’t wanna be doing?

Host: Absolutely. I think documents have [00:11:00] certainly started getting a lot longer. Emails have started getting a lot longer, haven’t they?

Josh Cavalier: I’m all for brevity, believe me.

Host: So from what we’ve discussed, it seems that if you roll out a tool, some kind of generative AI tool, there are certainly going to be blind spots, right?

Where these tools end up limiting capability rather than, unfortunately, rather than enhancing it. So how do we make sure, Josh, that AI can actually accelerate learning and deepen understanding rather than just checking the box and getting the tasks done one after the other?

Josh Cavalier: Right. And there’s a massive opportunity before us because I know throughout my career there’s been very set ways in regards to how training and performance experiences are created and consumed.

And of course, eLearning has always been like a, you know, a big one, and then we moved it onto mobile. But what I see is the opportunity to look at this [00:12:00] technology and the way that it can be leveraged in new and innovative ways that we’ve never been able to do before at scale because it either didn’t make sense from a development standpoint, time, money, it just cost too much.

And I think about early in my career, I would always try to solution set with… if I had infinite time and infinite money, what would be the best possible experience for this performer? What is that? And then you, you pull back from that, right? Based upon all the restrictions that you have. Well now, many of those solutions are becoming part of the conversation because you don’t need to go on site to create a training video.

You know, you can actually go ahead and put together something that is based on photographs or a short video that somebody takes on site, and then you can effectively put that content together, saving tens of thousands of dollars, weeks and weeks of work. You have to be careful because you can get really stuck in a rut with your tool set [00:13:00] and what’s viable.

But, you know, be curious and want to explore and push some boundaries and create, you know, opportunities to where you can test or pilot these new training and performance experiences, ’cause they’re only going to continue. Like, they’re only going to accelerate. And I think if you put the blinders up and you’re like, “Okay, well this is our solution set, this is our tool set, this is our platform, this is how we’re recording, you know, these scores or whatever,” I think you’re missing the boat.

You’re missing the opportunity that this technology can allow IDs to program and to create interactive experiences within minutes that would take weeks to build.

Host: It’s such a catch-22 because at the end of the day, it’s only going to create more time and more opportunity for each and every professional down the track, but I think it’s just sometimes it is that getting over the hurdle of carving out the time and dedicating the time.

It’s seeing it as an investment [00:14:00] almost because knowing that it will, you know, you will have a return on that investment of time down the track. 

And something that I really love about what AI is bringing into the space, and we’ve seen it on our side at TalentLMS, is the way that we’re able to leverage it for also for the learner experience.

So creating, like, safe, interactive places to practice and to simulate before getting into a high-stakes situation. You know, getting the practice in, using AI to kind of give you back some feedback and kind of come back on the safe, private kind of practice space. We call it Learning Playground.

We think it’s a little bit more of a place where people should be able to kind of stretch their legs in a safe way and kind of have that safety to it. But I think that this is also an amazing tool that before this, you know, you would’ve had to get into working breakout sessions, get everybody into the office or into virtual breakout rooms, [00:15:00] and now it can just happen, you know, in a one-on-one, a one-to-none environment actually, which is pretty amazing.

Josh Cavalier: Yeah, and that technology’s only going to increase in capability and what can be tracked, what those outputs of those tracked interactions can yield, and how it can go in and even dynamically change future interactions. That’s what excites me, you know, is that there’s this notion of either self-healing or adaptation that’s possible.

Especially when you start incorporating business data into it, and you get some real-time updates in regards to those interactions so that when there is a situation that changes with frontline sales, that you’ll be able to go ahead and take that new reality, especially now that supply chains are so volatile, that if you had to change products and sell a different product on a Monday morning ’cause you couldn’t get access to a product, you can effectively go in and build that sim [00:16:00] At such speed that someone, a salesperson could be pulling into a parking lot, go in and, and go over, you know, benefits and about this new product and get some reps before they walk in the door with a customer.

Like, that’s all viable today, and that’s the exciting part, is, you know, the, you know, a platform like yours, that’s an enabler.

Host: Josh, I know that you’ve spoken and developed the AI Task Enablement Scale to kind of think about different levels of AI involvement in a task, which I think is such a great way of kind of breaking down and helping create, you know, a common language within an organization, for example.

How do you think organizations use a framework like that to decide where AI should support people, where it should take more of the work, and where humans still need to be firmly in control?

Josh Cavalier: Yeah, just for context, that scale is one through seven, where one is human and seven is completely [00:17:00] autonomous AI.

And then we have those, those shades of gray in between, where two you’re prompting, and three you’re building like a GPT or a copilot agent or a Google Gem or Gemini Gem. And then you, you keep going up from there to where now you start working with like Claude Cowork or Claude Code and Codex and Gemini Anti-Gravity, and now you’re collaborating at scale.

And then as risk is mitigated, you start pulling away the human, and then the agent or AI becomes front and center. But even within there, there are shades of gray. And so what’s been fascinating, and this is what… You know, when I was thinking about this scale, I was like, “Well, how does it show up behind the corporate firewall? What does this look like?”

And you can go in and you can map those levels right to AI access. So do I have access to Copilot, and if it’s a Copilot for Work or Copilot for Web? Do I have access to make a Copilot agent? Do I have access to [00:18:00] Copilot Studio? Each has an affordance in regards to the level of automation within a flow of work, and some are gonna be down in the tactics, and that’s all they’re gonna be.

They’re gonna be at level two, level three, which is fine. I mean, you can, you can have productivity, you can have advancement of work at those levels. But then as we start taking looking, you know, looking at how AI can reconstitute work at scale and is risk mitigated, then we start getting into bigger AI.

Now we’re talking about full-on automations, we’re talking about agents that talk to each other, and, you know, that type of work, though, requires massive human in the loop, governance, security. There’s a big spend as far as tokens. You know, so there’s this balance that’s happening between what is possible and what is the cost, both with human labor and with token burn.

Host: I guess there’s also another side to this, which is the reality of workload and [00:19:00] expectations, right? If someone has 10 deadlines and AI can get them 80% of the way there in five minutes, why would they spend an hour thinking through the problem themselves? So I think this also becomes more of a work design problem rather than just an L&D problem.

What role do you think L&D can play here in helping solve those kinds of issues? Can L&D play a role in those kinds of issues?

Josh Cavalier: Well, they can, but it depends upon the skill set that they have. Because to work higher up within the work, that takes a whole different set of skills. It’s one thing to go ahead and produce training and performance experiences to change behavior.

It’s another to go ahead and work up and understand business process management, to be a performance consultant, to work directly with the business in understanding the capabilities of AI to know exactly where that delta is at. Like, where is the intersection, and how can we reconfigure work?

Believe me, I’ve been a part of many system [00:20:00] implementations where as we’re building the training going, “This is a horrible system. I don’t know how we’re gonna be able to make people effective on this when it’s just not gonna work.” And at that point, if we really had a line into developers or the business and give them feedback, that’d be very valuable feedback.

But essentially, organizations aren’t set up that way. So can IDs help with the reconstitution, reconfiguration of work? Most definitely. But do they have the skills required to be a part of that conversation? Most do not. Matter of fact, when I go in and I am facilitating, and each week I facilitate, you know, AI workshops for IDs, and I am shocked how many do not understand performance consulting.

They’re so, and to your point, they’re so focused on the task at hand to be that order taker and just crank out the training. And it’s a massive missed opportunity to work [00:21:00] higher up with the business to understand exactly what’s happening from root cause analysis. Is it truly a performance or training issue, or is it environmental or, you know, something else going on with the business?

So yeah, there’s a lot of work to be done in that area if IDs want to be a part of that conversation.

Host: Josh, I’ve heard you talk about the idea of, I hope I get this right, human-machine performance analysts and this being a future role in organizations. I’d really love to know more about this concept.

Josh Cavalier: Yeah. So when I wrote my book, I, in my mind, I was thinking about the evolution of an ID with those competencies of performance consulting, business process management, even total quality management if you wanna take it that far. You know, a little bit of Six Sigma thrown in there. And being able to go ahead and get the data from the business that would help determine [00:22:00] at the work level what is the relationship between human and AI across the whole entire series of tasks within a process.

That was my thought. And as I’ve, you know, got more time to ponder that, I do believe that this is a business analyst role, and I don’t necessarily know if it’s for every ID. You know, I think we’re going to have kind of a separation in the industry to where you’re going to get instructional designers who are very passionate about orchestrating those types of learning and performance experiences at scale and continuing to drive towards innovation and new ways of creating those and getting them down in the flow of work.

But the human machine performance analyst is gonna work tighter with the business as a business analyst, but they’re gonna have to understand performance consulting. They’re gonna have to understand business process management. They’re gonna have to understand all these little [00:23:00] nuances around AI and the spend that’s there to really have impact.

And what’s fascinating is that my peers at Talent Rewire, they are constantly scanning the job boards for these new roles that are appearing in organizations, and my gosh, you call it by whatever name you want, they are starting to crop up.

Because, you know, first there’s the realization like, “Oh, well this isn’t—like we can’t just park AI in a single function. It impacts everything.” And then, well, what does it mean when every associate gets access to a certain level of AI from a cost standpoint? What kind of affordance does it, do they—what is the impact on work? What is the impact on the individual when it comes to like the supervision tax? Like there are all these little nuances about the work itself.

Can we completely reconstitute the work? Is the risk mitigated? You know, and these are the things that the human [00:24:00] machine performance analyst is gonna bring forward to the business or to a leader and gather all that data and then run simulations. If we—you know, here’s the thing. You have a lot of organizations that out of the gate they were taking big, here in the States we call it a big swing towards the fence, which is a baseball analogy.

And they’re gonna go ahead and take all tier one call center people and completely replace it with AI. Go, make it happen. Well, they eventually bring those people back, or if you take like an Ikea, they reconstitute those individuals as designers to the tune of 1.2 billion top-line growth, substantial.

So somebody’s gotta go ahead and provide guidance on that. Does it make sense from a cost standpoint over a long period of time if we move these individuals out of this role into a new role, or we cost them out altogether? Like, what does that mean? [00:25:00] There goes all that tacit knowledge out the door, and that can have a huge impact on the business.

So the human machine performance analyst sits between many different functions to gather data from the business to determine, one, what is the current cost or the net value of an existing workflow? And then what is the opportunity to completely transform work, and how does it impact us as a business from a people standpoint, and what does that look like in regards to the cost of AI and the systems that we have?

Host: It’s definitely a huge undertaking, and I think these kinds of roles, at least at a starting point, really require a deep and broad understanding of the business, right, of the organization, because it can’t really be approached in like discipline-specific way. Because like you said it, the efficiencies can be quite great that they can be company-wide, but then also, you know, the learnings and the way things are implemented cannot [00:26:00] necessarily be drilled down, I think, at least at the first level, to a department or a team.

I think it needs to be a real big-picture thinker and someone who has the real big-picture understanding of how the business is operating.

Josh Cavalier: Exactly.

Host: Josh, I know that you’ll agree with me that for Learning and Development professionals, whether you’re an instructional designer, whether you’re, you know, specifically on the field or maybe you’re working with the business at the higher level to understand the objectives first and kind of, you know, create a learning program off the back of that, regardless of the role you might be playing within an L&D team, I think, you know, we could all agree that roles are moving away from necessarily the creating of content and becoming more and more about how to best curate it.

You know, you yourself spoke about how the speed with which things can now happen when it comes to creating learning content in all different types of formats, which is really amazing, [00:27:00] and roles are constantly being redefined and reshaped. That can be pretty disconcerting for people. So if you’re an L&D pro listening to this, perhaps they might be in charge of a small team, or they may just be a, you know, small and mighty team of one.

What would your advice be to someone who wants to be one step ahead in their profession and really create value in an AI-driven workspace and a, and a workspace that, that aims to kind of achieve that efficiency and augmentation that AI seems to now be offering them. What do you think would be the one, two, three things that they should kind of be focusing on as a starting point from your point of view?

Josh Cavalier: Well, one, they have to understand the craft. Like, if you are not aware of empirical studies or learning science or adult learning principles, fill that gap first. Like go please round yourself out and [00:28:00] understand the intricacies of this profession. That’d be first and foremost. Second would be to have a growth mindset, and if you’re locked in…

And, I, you know, I totally get it. Individuals get so passionate about the work that they do, the craft that they are able to perform and all the details and their decades of understanding and learning and applying, and then they get rocked because now AI can go ahead and build a course in five minutes. Well, you know, having that growth mindset, that really comes into play because now you can take a look at that as an opportunity and not as a limiting factor.

Which gets me to number three, which is adaptability, is being adaptable and being able to, at the end of the day, how does this best support the business? How does this best support the individuals that I am, you know, assisting [00:29:00] day in and day out? That’s the key. And, you know, this technology, ’cause it is moving so fast, is going to give us all these new affordances of supporting the business and supporting performers, individuals where they’re at.

You can get caught up in doing the work, you know? Head down, we got deadlines, and you’re, you’re driving towards work completion. It takes intentional effort to hit the pause button, poke your head up out of the sand, look around and say, “Whoa, we need to go ahead and try this on the next project,” or, “We should run a small little pilot and be intentional about it and put it on the schedule.”

That’s where the real velocity’s gonna happen are for both teams or individuals that can operate at that level. And it’s hard work, there’s no doubt about it, because things are very disruptive right now. But if you can, you know, acquire those attributes and and apply them [00:30:00] properly, there’s just so much potential moving forward.

I mean, I do believe we’re in a golden age of performance and learning, and we’re gonna see some really amazing things even… as we move into ’27.

Host: I’m nodding my head emphatically because in those moments where I, even I on a personal level, when I’ve been able to carve out… not been able to, made the very conscious decision to carve out that little bit of time to dedicate to doing something I would normally do in a slightly different way leveraging AI, the benefit to it has been so great, mainly because of what it has opened up for me time-wise in terms of, you know, the things that I may take great pride in them, but knowing that a lot of that, to be fair, is stuff that can be automated, that creates the space for the real beauty of many years of experience and love for a craft to kind of really come out and shine in new and different ways.

So I’m nodding absolutely emphatically to everything that you’re saying [00:31:00] because I see it in day-to-day work as well.

Josh Cavalier: Yeah, I know. I get excited.

Host: So finally, Josh, as we wrap up our conversation, one last question, some final thoughts, if you will. AI use is inevitable, so how can companies make sure that employees use it in a way that doesn’t hide knowledge gaps or create this learning debt?

What are some guiding principles that you’d put into place to ensure that capability is really being built rather than being slowly lost?

Josh Cavalier: That’s a difficult one, because you get this capability kinda dangled out in front of you, and you just wanna get the work done. And I touched upon it earlier, it’s leadership.

You know, leaders are gonna have to come in and be intentional, and notice that when these workflows collapse or work can be accelerated, are we losing our edge? Like, should we get together and talk about how this work is [00:32:00] done even though AI is doing mostly the work? I think that’s gonna happen.

Because you’re going to get individuals that eventually are going to be scaling and doing a lot more work that are gonna be a higher paid, much more, important, you know, part of the business because they are coordinating so much. But what happens to that person if they leave the company, right?

Who’s gonna go ahead and replace them? And what kind of skills do they need to pick things right up, right? And how do you build that? And again, that takes planning, that’s being intentional with how you show up, and creating spaces so that internal knowledge and knowledge of the craft is maintained and built upon, right?

So again, we can get caught up in doing the work, but how are we improving everyone within the team in the age of AI, and what does that look like?

Host: Absolutely. [00:33:00] Josh, thank you so much for sharing your insights today. It’s been such a joy, and I think there are so many real-life current learnings that we can take from the conversation.

Like you said, this is an age that we are living through in real time, so we are all learning during the same time, and it’s just about the pace with which, you know, we’re ready to kind of take it on and move forward.

Josh Cavalier: Thank you so much. I appreciate the opportunity.

Host: Thanks for tuning in. In the next episode, we’re asking why bad habits die hard and what you can do to make sure knowledge isn’t just learned but applied. You can find Talent Talks on all podcast platforms. Subscribe now so you don’t miss an episode.