The finding
The biggest single gain in our AI call-scoring research came from people. Giving the scorer the one-line reasons trained assessors gave each mark lifted its average fit with human scores from 0.62 to 0.72. Written textbook examples of good behavior added nothing, according to our field guide Can AI judge a sales call?.
The study ran from August to October 2026: more than 5,000 tests, more than 250,000 scoring decisions and 15 AI models, against blind marking by trained assessors from PeerLab on real enterprise discovery roleplays and a 20-skill capability scorecard. The assessors never saw a machine's answer. The calibrated results cover one scorecard and one cohort.
This piece is one finding from the sales coaching research hub.
What a marker reason is
A marker reason is the short note a trained assessor writes to explain why a call earned a level on a skill. The scorecard says what a level means in general. The reason says what the assessor saw in this call that earned it.
The field guide's example is one level of strategic negotiation. The scorecard reads: "Explores trade-offs explicitly (e.g. speed vs. cost, scope vs. certainty, risk vs. return)". An expert's reason for awarding that level on a real call read: "Surfaces budget source, CFO gate and decision timeline."
No model would guess that from the scorecard alone. The experts read "explores trade-offs" as budget and sign-off groundwork. That is how this team's standard works in practice, and it lives in the assessors' heads until someone writes it down. An AI scorer working from the level text alone starts at 0.62, as the table below shows.
Where the reasons helped most
| Skill | Fit with human scores, scorecard only | With marker reasons |
|---|---|---|
| Strategic negotiation | 0.42 | 0.66 |
| Personas and drivers | 0.35 | 0.66 |
| Frames impactfully | 0.30 | 0.54 |
| Emotional control | 0.43 | 0.61 |
| Gaining commitment | 0.64 | 0.83 |
| Average across all 20 skills | 0.62 | 0.72 |
Source: field guide. These are skill-level fits, separate from the call-ranking measure.
Look at which skills moved most: personas and drivers (0.35 to 0.66), strategic negotiation, framing, emotional control and gaining commitment. All of them need human judgment and some emotional intelligence, the kind a model does not bring on its own. A checkable behavior, such as whether the rep asked permission before a cold-call pitch, needs far less calibration. The guide's advice follows from that: spend your expert marking where judgment lives.
One more result made the method practical. Reasons from other markers worked as well as one marker's own. The scorer learns the scorecard as the group of assessors applies it, and does not end up imitating one person.
Why textbook examples did nothing
Our reading of the result is that a textbook example describes ideal behavior in the abstract, what a perfect trade-off question sounds like. A model already knows roughly what good selling looks like in the abstract. What it lacks is the local standard, the line between a 3 and a 4 as this team's experts draw it on real calls with real buyers.
Marker reasons carry that line. They are written against real calls, at a specific level, in the assessor's own words.
The guide also found that the scorecard's own wording should stay intact. Rewriting the scorecard as "observable checks" dropped its rank agreement (how closely it put reps in the experts' order) from the production baseline of 0.74 to 0.62. That is a ranking measure, separate from the skill-level fit in the table above. Telling the model how to score, in the system prompt, made agreement worse too, a story told in five prompt fixes that made AI call scoring worse. The reasons improved scoring where instructions had failed.
The method has to use them
Marker reasons lifted the checklist model, the scorer that asks one narrow yes/no question at a time. Given the same reasons, GPT-6 Sol ranked reps at 0.73, exactly where it started. The guide's summary: people supply the meaning, and the narrow questions are what let a model use it.
- 01The scorecard's own level text
- 02Up to five assessors' reasonsGiven that level on other calls
- 03One yes/no question per level80 questions per call
- 04A blind GPT-6 Luna passScores the whole call without the reasons
- 05Rankings averaged0.89 rank agreement with the experts
The checklist model, TypeSafe's Jev, answers one narrow question at a time: did this rep reach at least this level on this skill? Each question carries the scorecard's own level text and up to five reasons assessors gave that level on other calls. The question template from the guide reads:
Skill '<SKILL>' (1-5). Level <L> reads: <LEVEL TEXT>
Trained assessors gave level <L> to other calls for reasons like:
- <reason>
- <reason>
- <reason>
Does this call's Salesperson perform AT LEAST at level <L> on '<SKILL>',
judged the way those assessors judged?
Four questions per skill (levels 2 to 5, since level 1 is always true) across 20 skills makes 80 questions per call, sent in one request. A blind GPT-6 Luna pass scores the whole call separately without the reasons; giving Luna the reasons made no difference on its own and blended worse. The two rankings averaged reached 0.89 rank agreement with the experts, at about $0.007 per call for scores plus written feedback.
With no marked calls yet, the guide says to drop the reasons lines and the "judged the way" clause. The reasons are what lift it further.
How to collect marker reasons
Collect marker reasons by asking your trained assessors to write one sentence explaining each mark on calls they already score.
- Use the scorecard you already teach. Keep each skill's level text in its own words, or your methodology owner's. The guide found rewrites scored worse.
- Have calls marked blind. Assessors mark real calls without ever seeing an AI score. Blind marking is what makes the reasons a human reference.
- Ask for one line per mark. For every skill and level awarded, the assessor writes what they heard that earned it, in plain words, tied to the call. An invented example: "Named the CFO as sign-off and asked for the decision date."
- Store call, skill, level and reason together. That lets you leave out the reasons from the call being scored, so a call never calibrates against its own marks.
- Use up to five reasons per level, from other calls. Put them into each yes/no question for that level.
- Spend marking time on judgment skills. Personas and drivers, negotiation, framing, emotional control and commitment moved most. Checkable behaviors need less.
- Re-test on held-out calls. Choose the setup on calibration calls, then check it against marked calls whose reasons it has never seen.
The reasons go stale as the standard moves, so marking is continuing work. The guide is plain about where the cost sits: rubric design, human marking, testing and maintenance surround every scoring pass, and the model bill of about $0.007 per call is the small part. Assessors must never see the AI score, and that cost comes back every time the standard changes. If your reps follow a methodology, work with whoever owns it: they wrote the standard the scorecard has to express.
Where the reasons go next
The same reasons that teach the scorer make the coaching specific. A rep told "2 on gaining commitment" learns little. A rep told that level 3 on this team means agreeing a dated next step with the person who owns the budget knows what to do on the next call.
Our AI sales experts work to a written standard built with the people who wrote the method: Keenan with ASG for Gap Selling, Val with ValueSelling Associates for the ValueSelling Framework®, and Caty for a team's own playbook and scorecard. The rep talks the live deal through on a call (typing is there for when they cannot talk), and the expert holds them to that standard while working out the next conversation with the buyer.
To try this on your own scorecard, ask your two best assessors to write one sentence on why each level of one skill earns its mark, and give the scorer those sentences before any examples. Related findings: newer AI models mark reps harsher, average error flatters a lazy scorer and AI picked the first call it read 62% of the time. The research is discussed in Episode 01 of Show Your Working.
FAQ
How do you improve AI call scoring accuracy?
Give the scorer the reasons trained assessors wrote for each mark, inside narrow yes/no questions per skill and level. In our call-scoring research that lifted average fit with human scores from 0.62 to 0.72, the biggest single gain. Prompt instructions and scorecard rewrites made scoring worse. Re-test on held-out blind-marked calls.
What are marker reasons in call scoring?
Marker reasons are the one-line notes trained assessors write explaining why a call earned a level on a skill. For strategic negotiation, a scorecard said "explores trade-offs explicitly" and an expert's reason read "Surfaces budget source, CFO gate and decision timeline." They show a model how the team applies its own scorecard.
Do examples in a prompt help AI score sales calls?
Written textbook examples added nothing in our tests. Reasons from real marked calls lifted fit from 0.62 to 0.72. The difference is that a textbook example describes ideal behavior in general, while a marker reason records what earned a specific level on a real call, in the assessor's own words.
Which sales skills are hardest for AI to score?
Skills that need judgment are the hardest to score, and they moved most once marker reasons were added. They were strategic negotiation (0.42 to 0.66), personas and drivers (0.35 to 0.66), framing (0.30 to 0.54), emotional control (0.43 to 0.61) and gaining commitment (0.64 to 0.83). Checkable behaviors, like asking permission before a pitch, need much less calibration.
Do marker reasons work with any AI model?
Marker reasons do not work equally with every model. They lifted the checklist model, which answers narrow yes/no questions per level. Given the same reasons, GPT-6 Sol ranked reps at 0.73, exactly where it started. The question format is what lets a model use the reasons, so method and reasons go together.