What 34,000 Coaching Sessions Taught Us

I used to think I was a decent cook. Not Michelin-star decent, but "can follow a recipe and not poison anyone" decent. Then my wife asked me to make dinner without a recipe, using whatever was in the fridge, and I stood there for 15 minutes staring at a courgette like it owed me money.

That's the difference between practice and performance. One happens in controlled conditions. The other happens when the pressure's on, the ingredients are wrong, and someone's hungry right now.

The entire AI coaching industry built for the recipe. We built for the fridge.

The data told us.

34,000 Real Coaching Events

Here's what most people don't know about Replicate Labs: we run a free AI coaching tool. No paywall, no trial expiry, no credit card. Just high-quality AI sales coaching, available to anyone. If you want the broader context first, our complete guide to AI sales coaching lays out what it is and how it differs from training.

Over the past 12 months, that free tool has generated more than 34,000 coaching events. Not logins. Not pageviews. Real coaching interactions where a rep sat down, described a problem, and got help.

That's a tonne of data. And when you sit with a tonne of data for long enough, patterns start to scream at you.

The loudest pattern? Reps don't use AI coaching the way the industry thinks they do.

The Assumption Everyone Made

If you look at AI sales coaching right now, here's what you'll find: roleplay. Everywhere. Pitch practice. Objection handling simulations. Discovery call rehearsals. Cold call simulators.

The assumption is simple: reps need practice. Give them a safe space to rehearse, they'll get better, they'll close more deals.

Sounds reasonable. It's also wrong.

When we looked at how reps actually use our platform, the split wasn't even close. Live deal coaching outpaced roleplay by 3 to 1. Three times more reps came to the tool with a real deal, a real objection, a real email they needed to send today, than came to practise a hypothetical scenario.

Think about that for a second.

The Industry Built the Wrong Product

I'm not saying roleplay has zero value. It doesn't. For new reps learning a methodology, for onboarding, for building baseline confidence, roleplay has a role.

But the industry didn't build roleplay as one feature. They built entire platforms around it. Raised hundreds of millions around it. Built their positioning, their pricing, their entire go-to-market around the idea that what reps want is practice.

Reps don't want practice. They want answers to today's problem.

The rep sitting in a car park before a discovery call doesn't want to roleplay. They want to know: "What should I ask this CFO who ghosted me after the second meeting?" The AE who just got a brutal objection on a call doesn't want a simulation. They want to know: "How do I recover this deal before my forecast review on Friday?"

That's the gap. The industry built for preparation. Reps need intervention.

Why This Matters More Than You Think

Here's where it gets interesting. When we dug deeper into the 34,000 events, we found something else.

Reps who used live deal coaching came back more often. The repeat usage rate for deal-specific coaching was 2.4x higher than for roleplay. Not because roleplay was bad. Because the rep could immediately see the value. They asked a question about a real deal. They got an answer they could use that afternoon. They came back the next day with the next deal.

Roleplay doesn't have that feedback loop. You practise a cold call. Maybe you feel more confident. But you don't know if it worked until days or weeks later, and by then you've forgotten you even did the roleplay.

Immediate relevance drives adoption. Not features. Not gamification. Relevance.

Sound familiar? It should. It's the same reason most sales training fails. It's disconnected from the moment it's needed.

The speed of adoption is the real signal

Here's the pattern in the data that I can't stop thinking about. It isn't about how many people used the tool. It's about how fast they went from first touch to hooked.

A high proportion of the reps who come to us for one genuine coaching moment on day one come back on later days. Not because anyone asked them to. Because the first session paid off, on a real deal, immediately. And once a rep gets a taste of that, they don't ration it. They batter the product. People run through a free plan we designed to last weeks in a couple of hours, because the moment it helps them on Monday they're back with Tuesday's deal before lunch.

Sit with that for a second, because it's the opposite of how enablement usually behaves. Compliance-driven training and enablement tools are blood from a stone. You mandate the login, you chase the completion, you build a dashboard to prove people did the thing, and they still do the minimum. You are pushing usage uphill the entire time.

This pulls. Nobody is mandated onto a deal coaching session. Nobody is compliance-checked into it. Reps come on their own, fast, repeatedly, because a real deal is on the line and the tool actually moved it. When usage accelerates like that with no stick behind it, that is not adoption. That is reps telling you, with their behaviour, that you built the thing they actually needed.

The Uncomfortable Truth for L&D

I've spent enough time in enablement to know this won't be popular with everyone. L&D leaders have built their careers around training programmes. Roleplay platforms justify training budgets. Practice metrics justify headcount.

But the data doesn't care about org charts.

Most reps get next to no coaching between formal training sessions. That's not a training problem. That's a coaching availability problem. And roleplay platforms don't solve it, because reps don't voluntarily practise when they have 47 other things screaming for their attention.

What they will do is ask for help when they're stuck. Every single time. The 34,000 events are the proof: nobody was assigned a deal coaching session, nobody was compliance-checked into it. Reps came on their own because a real deal was on the line. That is the only kind of usage that tells you the truth.

So What Did We Get Wrong?

Here's the honest confession: we almost made the same mistake. Early on, we leaned heavily into roleplay features because that's what the market said it wanted. That's what competitors were building. That's what RFPs asked for.

The data saved us. 34,000 coaching events told us, in no uncertain terms: build for the deal in front of the rep, not the scenario in the training calendar.

We listened. Not because we're smarter than everyone else. Because we had data they didn't, from a free tool that removed every barrier to honest usage.

When there's no contract, no manager watching, no compliance requirement, reps show you what they actually want. And what they want is performance support, not practice.

What the two sessions actually look like

Let me be concrete about the difference, because "roleplay versus deal coaching" sounds like a small distinction and it isn't. Here are the same Tuesday afternoon, two reps, two ways of using the tool.

The roleplay session is the kind the industry built platforms around. It is not a waste of time, exactly. But the buyer was invented, the objection was generic, and the rep walks back into a pipeline that contains none of it.

The deal coaching session is the same tool, used at the moment of need. The coach doesn't run a simulation. It works the real problem: what the champion said before they went quiet, the fact the renewal was never confirmed as a budgeted line item, the exact message that reopens the conversation without sounding desperate.

That second session is why the repeat rate is 2.4x higher. The rep got something they used within 16 hours, on a deal that pays their mortgage. The first session asked them to get better in general. The second one helped them win something specific. Reps will always come back for the second kind, and they will mostly skip the first.

So what this means for what you build, or buy

If you run enablement, the practical takeaway is not "delete all roleplay." Roleplay still earns its place for new-hire onboarding and for drilling a brand-new methodology before a rep is trusted on live pipeline. Keep it there.

But stop making it the centre of gravity. If the primary thing your coaching tool does is generate practice scenarios, you have built a gym for people who need a corner during the fight. Reorder it. The default surface a rep sees should be "what deal are you working, and what's stuck", not "pick a scenario to rehearse." The roleplay becomes the thing a coach reaches for when a real gap shows up, not the front door.

And if you are buying, run the simple test. Ask the vendor what percentage of sessions on their platform are reps bringing a real deal versus reps doing a scripted scenario. If they don't measure that split, they don't know what their product is actually used for. If they do measure it and the answer is mostly scenarios, you are buying a practice tool and you should price it like one.

The Takeaway

The AI coaching industry is having its moment. Money is pouring in. Everyone's building. But most of it is being built on an assumption that 34,000 data points say is wrong.

Reps don't want a gym. They want a coach in their corner during the fight.

Build for the fight.

---

Ready to see what 34,000 reps already figured out? Replicate Labs gives you AI coaching that helps with the deal in front of you, not a hypothetical one. Reps, managers, entire teams: get started free at replicatelabs.ai. No credit card. No trial clock. Just coaching that actually gets used.