Illusion of Validity

A professional interviewer sits with a candidate for an hour and emerges confident: this one will excel. Decades of data say the unstructured interview predicts almost nothing. Yet every interviewer trusts his own hour. The confidence is real. The validity is not.

How does false confidence sustain itself? Two pillars. First, coherence: the mind builds a smooth story from the hour, and smooth stories feel true. Second, no feedback: the rejected candidates' later performance is never seen, so errors never report themselves. Confidence grows in the dark like mushrooms.

Any field with these two features — rich narrative, absent feedback — breeds the same illusion: certain kinds of forecasting, diagnosis, and prophecy flourish there precisely because no one is keeping score.

The defence is unglamorous: keep score. Write your predictions and check them later. The illusion of validity cannot survive a ledger. Confidence earned against recorded outcomes is the only confidence that means anything.