# Building scorecards Writing items, levels and feedback instructions that grade consistently, and the two mistakes that make a scorecard useless. Source: https://replicatelabs.ai/docs/admin/scorecard-config Last updated: 2026-09-03 --- A scorecard is the standard a conversation is graded against. Getting it right is the highest-leverage configuration work on the account, because every roleplay result and every management read downstream inherits its quality. ## The structure
ItemOne capability. A scorecard is a set of them. Aim for the handful that actually decide outcomes in your business rather than a complete taxonomy of selling.
LevelsUnder each item, what each grade looks like as observable behaviour.
Feedback instructionWhat the rep is told at each level. This is where your organisation's coaching voice lives.
Item, levels, feedback. Most of the quality is in the levels.
## Levels must be behavioural This is the whole craft. A level written as an adjective cannot be graded consistently by anyone, human or otherwise.
Scorecard configuration showing capability items and their score ranges
Scorecards on an account. Several short scorecards, each grading one kind of conversation, beat one long one that grades all of them badly.
Not gradeable

Item: Discovery quality
Level 3: Good discovery
Level 2: Adequate discovery
Level 1: Weak discovery

Gradeable

Item: Quantified the problem
Level 3: Got a number from the buyer and confirmed how it was arrived at
Level 2: Got a number, did not test it
Level 1: Established a problem exists, no number attempted

The test: could two different people, reading only the level descriptions, watch the same conversation and land on the same grade? If not, rewrite until they could. ## Feedback instructions Write what you would say to the rep, in your organisation's voice, at that level. Not the definition of the level again. - **At the top level**, name what they did so it is repeatable. "You asked for the figure and then asked how they got to it. That second question is what makes the number usable." - **At the middle**, name the one thing missing. Exactly one. - **At the bottom**, give the words. A rep at level one usually does not know what the behaviour sounds like, and telling them to do it more does not help. ## Two mistakes
Too many items

A twenty item scorecard produces a grade nobody reads and feedback nobody acts on, and it makes every roleplay feel like an exam. Six to ten items covering the capabilities that actually decide your deals will change more behaviour than a complete one.

Editing it once results exist

Scores are only comparable while the scorecard is unchanged. Edit items or levels and every earlier grade was produced against a different standard, with no offset that repairs it. Tune the scorecard hard in the first few weeks, then freeze it, and record the date you froze it so any report spanning that line can say so.

## Match the scorecard to the conversation A scorecard grades one kind of conversation. Binding a discovery scorecard to a negotiation roleplay marks reps down for not doing something they were never attempting, which reads to them as the tool being broken and is a fast way to lose a rollout. If your team runs four distinct conversation types, you need four scorecards, not one long one. ## Where to start Do not write one from scratch. Take the qualification standard your organisation already uses, the one in your deal review template or your methodology material, and turn each requirement into an item. If a requirement cannot be turned into observable behaviour, it was never a standard, and finding that out is useful on its own.