Purpose and scope

A practical session format for quality reviewers and team leads

Fictional example: Northstar Support is a made-up contact center used to demonstrate a calibration discussion. The session is about consistent interpretation of service-quality criteria when reviewing interactions—not rating an employee’s overall performance.

Reviewers score the same sample interaction independently, identify where their evidence or interpretation differs, and record a proposed clarification for the quality guidance owner. This page is a workshop example, not an authoritative scoring policy.

Illustrative 45-minute session agenda

Timing is a sample plan; agree a suitable duration and sample interaction for each session.

Time Activity Session output
5 min Set scope and confirm the criteria being discussed Participants know which interaction and criteria are in scope
10 min Listen to the sample interaction Reviewers have heard the same interaction and know which criteria to assess
10 min Score independently and note evidence Each reviewer has an individual score and an evidence note
15 min Compare differences against the criterion and evidence The group identifies agreement, unresolved interpretation, or a proposed clarification
5 min Summarize and hand off guidance questions The facilitator records the question and sends it to the designated quality guidance owner

Prompts for evidence-led discussion

Keep the conversation about the interaction and the stated criterion.

Locate the evidence

Ask each reviewer to identify the exact words or observable action that informed their score.

Separate observation from interpretation

Record what was heard or observed first; then state how the reviewer connected it to the criterion.

Check the criterion

Read the relevant criterion as written and identify any wording that could support more than one interpretation.

Name what remains open

If the evidence and criterion do not settle the difference, record a specific question for the quality guidance owner instead of treating the group’s preference as approved policy.

Worked example: a next step that was implied

A fictional discussion to show how to resolve a difference without guessing.

Sample disagreement

The difference

In a sample Northstar Support interaction, the customer asks what will happen after the call. Reviewer A marks the “clear next step” criterion present because the agent says the case will continue to be reviewed. Reviewer B marks it absent because no action or follow-up point is stated.

  • Reviewer A’s evidence: “We’ll keep looking into this.”
  • Reviewer B’s evidence: no named next action or stated follow-up point.
Route for guidance

The discussion and outcome

The facilitator asks the group to compare the exact wording with the criterion. Reviewers distinguish a general assurance from an explicit next step. The group records the ambiguity as a question for the quality guidance owner; it does not declare a new scoring rule.

  • Shared observation: the agent says the issue will continue to be reviewed.
  • Open question: does the approved criterion require a stated action, a follow-up point, or both?
  • Handoff: the designated owner reviews the question against the authoritative guidance before any clarification is adopted.

Facilitate the calibration

A repeatable sequence for the session facilitator.

  1. 1

    Prepare the session

    Select one interaction appropriate for the agreed review scope. Share the criteria participants will use, and ask reviewers to score independently before the discussion.

  2. 2

    Listen to the same sample

    Confirm that participants are working from the same interaction and the same version of the criteria.

  3. 3

    Compare independent scores

    Invite reviewers to share their scores and evidence notes before discussing why interpretations differ.

  4. 4

    Discuss the criterion and evidence

    Return to the criterion wording and the interaction. Distinguish an observed behavior from an inference, and note when the available wording does not settle the question.

  5. 5

    Record and hand off the open question

    Summarize the evidence and the specific interpretation question. Send it to the designated quality guidance owner for review; do not present a proposed clarification as approved guidance.

Handoff and ownership

The session facilitator records the sample interaction, criterion discussed, evidence cited, areas of agreement, and any unresolved question. The quality guidance owner is responsible for reviewing proposed clarifications against the authoritative criteria and following the applicable approval process before changing guidance.

Use the organization’s approved scoring guidance and any separate formal employee-performance process as their authoritative sources. This example does not establish either process, determine an employee outcome, or define a customer-, assignment-, or location-specific rule. If the group cannot resolve a difference from the approved criterion, keep it open for the designated owner rather than creating a local rule in the session.