Does Your Hiring Process Have a Black Box?
Aviation figured this out in 1935. Most hiring processes still haven’t.
“Hire fast, fire fast”! I hear the phrase constantly from growth-stage founders and operators. It sounds like discipline. It sounds like iteration — we’re learning, we’re adjusting, we’re not afraid to change direction with decisions that aren’t working.
But when a hire doesn’t work out and the “correction” is firing them, the process that produced the hire often goes unexamined. The intake. The brief. The scorecard. The debrief. The people in the room when the decision got made. All of it stays exactly as it was — ready to produce the same mistake on the next search.
Hiring has no black box
Think about how aviation handles failure.
When a commercial aircraft has a serious incident, two things happen automatically. Investigators pull the black box — not to assign blame, but to reconstruct exactly what occurred. Then the findings flow back into the system. Training adjusts. Checklists get updated. Sometimes regulation shifts. The same failure becomes structurally harder to repeat.
That’s why commercial aviation has gotten roughly 100x safer over the last four decades. Not because pilots and planes got better. Because the system learned and made the necessary adjustments.
Hiring is one of the only high-stakes business functions without a black box.
When a hire doesn’t work out, the default response is to move on. The person leaves, the role reopens, and the process starts again — using the same brief, the same interview loop, and frequently the same unspoken criteria that produced the last miss. The firing becomes the correction. The decision chain that made it necessary goes unexamined.
Systems that learn create better outcomes
Atul Gawande opens The Checklist Manifesto with a story from 1935 that’s worth sitting with.
Boeing had just built the Model 299 — the prototype that would become the B-17. On its demonstration flight, with one of the Army’s most experienced test pilots at the controls, it stalled on takeoff and crashed. Two people died. The press verdict: too much airplane for one man to fly.
The Army almost rejected the design. Boeing nearly went under. But a small group of test pilots pushed back on the framing. They didn’t need a simpler plane, and they didn’t need better pilots. They needed a system that assumed even the best pilots would forget things under pressure.
What they invented was the pilot’s checklist — the first of its kind. The B-17 went on to fly nearly 1.8 million miles without a single accident of the type that killed the prototype.
Any pilot knows the discipline — you run the checklist every flight, even on a clear day to a familiar field.
Hiring is complex work. Dozens of inputs, multiple decision-makers, incomplete information, and a result that won’t be visible for months. And yet when a hire doesn’t work out, the reflexive assumption is almost always the same: we picked the wrong person.
Sometimes that’s true. Often it isn’t. More often, the system that produced the decision was doing exactly what it was set up to do — generating a gut-feel yes from a group of people who weren’t aligned on what they were evaluating, working off a brief that never got tightened, using a scorecard that collected opinions rather than signal.
The hire wasn’t the mistake. The hire was the output of a process that was never designed to be reliable.
What a hiring black box would actually capture
This isn’t about more bureaucracy or another ATS module. Instead it’s about installing scaffolding — the kind that lets you reconstruct what happened when a hire works, and more importantly, when one doesn’t. Four places most hiring processes leak signal:
-
Intake calibration — the brief that never gets tightened
The intake conversation is the single highest-leverage hour in any hiring process — and the one most commonly treated as a formality. Often this is a 30 minute conversation that produces a bulleted list of requirements and a vague sense of “senior.” A black-box version captures the why behind each requirement, the non-negotiables, the trade-offs the hiring manager will accept, and the specific business outcomes the role exists to drive. When a hire doesn’t work out six months later, you can trace the decision back to what was actually agreed — and where the brief started to drift.
-
Debrief quality — adjectives aren’t signal
Most debriefs are directional thumbs-ups and thumbs-downs with adjectives attached. “Strong technically. Maybe not senior enough.” Not because anyone’s being lazy — debriefs get compressed because everyone’s juggling five other things and the hiring manager is the busiest person in the room. But the compression comes at a cost. That’s not signal. It’s the absence of signal wearing signal’s clothes. A black-box debrief forces specificity: what did you see, what question surfaced it, what are you still uncertain about? When you look back at a good or bad hire, you can see which signals were present in the room — and which the group collectively missed.
-
Scorecard drift — evaluating a role no longer in the brief
The criteria you defined in the intake and the criteria the team ends up using in the final round are rarely the same. Someone adds a nice-to-have that becomes a must-have. Someone else prioritizes culture fit in a way the brief didn’t. By the final round, you’re evaluating against a role that no longer matches what was scoped. A black box makes that drift visible while there’s still time to correct it.
-
Post-hire signal at 30/60/90 — the loop almost nobody closes
This is where almost every company drops the thread. The hire happens, the role closes, and nobody goes back to ask: was the scorecard right? Did the person we said was great actually turn out to be? Understandably — by the time 90 days hits, you’ve already opened the next three roles and the last search feels like ancient history. But if you’re not closing that loop, your process isn’t learning — it’s just cycling. And the next search starts with the same assumptions that produced the last miss.
The reframe
Hire fast, fire fast isn’t a feedback loop. It’s a blame system dressed up as iteration.
The companies that hire best don’t fire faster — they investigate earlier, and they investigate process, not people. When a hire doesn’t work, they don’t ask “was this person a fit?” They ask “what in our system allowed someone who wasn’t a fit to make it through?” It’s a different question, and it produces a different kind of answer.
Most hiring processes capture nothing. Which is why most companies hire the same mistake, role after role — and never quite figure out why.
When your last hire didn’t work out, what changed about your process? Or did you just change the person?
We help growth-stage companies build hiring functions that produce signal, not just activity — so each search makes the next one sharper.
Get in Touch- Atul Gawande, The Checklist Manifesto: How to Get Things Right, Metropolitan Books, 2009.
- IATA Safety Report 2023 and Boeing Statistical Summary of Commercial Jet Airplane Accidents — long-run hull-loss accident rate data for commercial aviation.
