A panel interview exists for one reason: several people see a candidate from different angles, and the combination is more reliable than any one of them. Then the debrief starts. Someone senior says what they thought. Others nod, or soften what they were going to say. Twenty minutes later the panel has one opinion, and it is mostly the first one spoken.
This is not a failing of particular people. It is what groups do when they discuss before they commit. The fix is not to stop talking; it is to change the order: write first, independently, against the same criteria, and only then compare.
What goes wrong in an unstructured debrief
Two well-documented effects do most of the damage. Anchoring, described by Amos Tversky and Daniel Kahneman in 1974, is the pull a first number or first impression has on every judgement that follows it. Conformity is the pressure to agree with the group, especially with its most senior member. Irving Janis gave the extreme form its familiar name, groupthink, in the early 1970s.
In an interview debrief they show up in recognisable ways:
- An interviewer who had doubts says "I mostly agree" because the hiring manager was enthusiastic.
- A specific concern ("couldn't explain how they handled the outage") is lost because it does not fit the story the group is telling.
- The panel's view of one strength or weakness spreads to everything else, and a candidate becomes "strong" or "weak" overall.
- Different interviewers turn out to have assessed different things, so their opinions cannot be compared at all.
None of this requires bad intent. It is simply what happens when judgements are formed in conversation rather than recorded beforehand.
Structure first: the same criteria for everyone
Structured interviewing means deciding in advance what the job needs, asking every candidate about those things, and rating the answers against a shared standard. Large reviews of selection research, from Frank Schmidt and John Hunter's 1998 meta-analysis to the 2022 re-analysis by Paul Sackett and colleagues, have found structured interviews to be among the better predictors of job performance, and consistently better than unstructured conversations.
Structure has three parts, and a scorecard is where they meet:
| Part | What it means | What it prevents |
|---|---|---|
| Criteria from the job | Four to six things the role genuinely needs, written before the first interview | Each interviewer judging their own idea of a good candidate |
| A rating scale with anchors | 1 to 4, with a sentence describing what each level looks like for each criterion | "Strong" meaning different things to different people |
| Evidence for every rating | What the candidate said or did that justifies the number | Ratings that are really impressions |
Why 1 to 4
An even number of levels removes the comfortable middle. Each rating has to lean one way or the other, which forces the interviewer to decide what the evidence actually shows.
Independence: write before you hear anyone else
Structure makes opinions comparable. Independence keeps them separate long enough to compare. The rule is simple: each interviewer submits their scorecard before they can see anyone else's, and before the debrief.
In HiringSlate this is enforced rather than requested: the other scorecards on a candidate stay hidden from an interviewer until they submit their own. Nobody has to remember the rule, and nobody can accidentally break it by opening a colleague's notes the night before.
What a good scorecard entry looks like
Communication: 4. Great communicator, really impressive.
Communication: 3. Explained the migration plan clearly and checked I had followed. Rambled on the second question and didn't get to the trade-off.
The second entry is less flattering and far more useful. Another interviewer can see what it is based on, agree or disagree with the evidence, and the hiring manager can weigh it against everything else.
Running it in practice
- Write the criteria from the job before the first interview, and keep them to six or fewer.
- Give each interviewer a focus so the panel covers every criterion between them.
- Ask interviewers to submit within a day, while the conversation is fresh.
- Do not discuss the candidate in chat or the corridor until scorecards are in.
- Open the debrief with the criteria where ratings differ most, not with overall impressions.
- Let the hiring manager speak last.
None of this is new. It is what careful interviewing has recommended for decades. What usually fails is the discipline, especially in a busy week. A tool that holds the order for you is the easiest way to keep it.
Discuss after you commit, not before. The panel's value is in its separate views; protect them until they are written down.
Run your next hire on one slate
Create a slate for the role, invite your panel and add the first resume. Free during early access.
Start a slate