A calibration session is a meeting where managers score the same sales calls independently, compare their scores, and agree how the scorecard should be applied.
Running one
Before the session, each manager scores the same small set of recorded calls alone, in whole levels. In the session the group lines up the scores skill by skill, looks at where the scores differ, and asks each scorer for the evidence behind their level. The group then writes a ruling for each disputed boundary ("level 3 needs a date and an owner on the buyer side") and adds it to the scorecard.
- 01Each manager scores aloneThe same calls, in whole levels
- 02Line up the scoresSkill by skill
- 03Look at the biggest gaps
- 04Ask for the evidenceBehind each scorer's level
- 05Write a rulingFor each disputed boundary, added to the scorecard
Even trained experts disagree: in our 2026 scoring research, expert assessors marking the same calls differed on 18% of scores (field guide). The reasons scorers give matter more than the scores. When assessors' written reasons were given to an AI scorer, its average fit with human scores rose from 0.62 to 0.72. When a team uses an AI scorer, the settled scores become the reference it is checked against. The wider topic is the AI call scoring guide.
A session on one boundary
Illustrative. Three sales managers each score five discovery calls before a 45-minute session. On gaining commitment, two give a call level 3 and one gives level 2. The rep had proposed a follow-up and the buyer said "sounds reasonable". The facilitator asks the question every session runs on: "What did the buyer say that puts it at that level?" The group rules that level 3 needs the buyer to confirm a date, writes it into the scorecard as the reason, and re-scores the call at 2. The next AI calibration run uses that ruling, as described in How to calibrate AI call scores.
See also
- Inter-rater agreement: what the session raises
- Coaching scorecard: the document it updates
- How to build a call scoring rubric: writing levels from managers' reasons
FAQ
How often should sales teams hold calibration sessions?
Monthly suits most teams, with an extra session whenever the scorecard changes, a new manager starts scoring, or an AI scorer's model changes. Each session needs only a few calls if they include hard ones.
Who should attend a calibration session?
Every manager who scores calls, plus whoever owns the scorecard in enablement. If an AI scorer is in use, the person who runs it should attend so rulings reach the model's level descriptions.
Is calibration the same as a performance calibration in HR?
No, they are different meetings: HR calibration aligns managers' performance ratings of employees, and call calibration aligns how managers score calls against a scorecard. The mechanics are similar: independent ratings first, then discussion and a written ruling.