Documentation · Replicate Labs

The product, page by page. Every page is also served as Markdown, so the same words reach the models a team works in.

Roleplay

Reading your score

What to take from a graded roleplay, what to ignore, and how to turn a weak criterion into an actual improvement.

After a roleplay you get a graded read: a score, a breakdown by criterion, and feedback pointing at moments in the conversation.

The number is the least useful part of it.

#Three habits

Read the evidenceA criterion marked down with a quoted exchange is a coachable thing. The overall figure is not. Go to the quotes first and the total last, or ideally not at all.
Look for the repeatOne weak criterion in one roleplay is noise. The same criterion weak across four is a skill gap worth an hour. Judge yourself on the pattern, not the session.
Do not chase the numberPlaying to the rubric produces someone who is good at roleplays. Work the criterion, not the score.
How to read a graded conversation. The score exists to point you at the criteria; it is a signpost, not the destination.

#Turning a weak criterion into something useful

The most valuable twenty minutes after a roleplay is not another roleplay. It is a conversation with your coach about the specific thing that let you down.

1Name it preciselyNot "my discovery was weak". "I was marked down on confirming impact."
2Take the quoteBring the actual exchange the grader flagged. "Here is what I said. Show me what it should have sounded like."
3Get the wordsAsk for the two or three questions you should have asked, phrased the way you would say them.
4Re-runNow the second attempt is practising something different rather than the same thing more fluently.
Score to improvement. Without step two the conversation stays abstract and nothing changes.

#A low score that is not about you

Three cases worth recognising before you take it personally.

  • You ran four minutes. The grader had four minutes of evidence. It is not a read on your ability.
  • The scorecard was edited between attempts. You are being graded against a different standard. See Scorecards.
  • You practised the wrong scenario. If the roleplay's scorecard grades a discovery conversation and you ran it as a negotiation, the criteria will mark you down for not doing something you were never trying to do.

#What a good trajectory looks like

Not a rising line. What you want is a narrowing spread: your weakest criterion coming up towards the rest, and the same criterion no longer being the weak one three sessions later.

A rising average with the same criterion at the bottom every time means you are getting better at the things you were already good at.

Last updated 3 September 2026

View as Markdown