Noise: A Flaw in Human Judgment

Probably not for everyone — a bit too wordy even for me — but important reading regardless for those interested in justice and fairness. Noise hurts us disproportionately, and thus hurts us all. We all need to develop awareness, although not necessarily four hundred pages' worth. This is more a book for policy makers and shapers than for us in the trenches.

I liked the breakdown of noise into categories: level noise (two different immigration judges, one approves 88% of cases, one approves 5%); pattern noise (close to, but not quite, bias); and occasion noise (judges issuing sterner sentences before lunch than after, or if the weather is crappy). I liked the attempts to differentiate noise from bias. I liked the suggestions for evaluating noise and for minimizing it. (I did not care for their using a smooth Gaussian in some examples, but can grudgingly understand why they did so).

“Organizations all over the world see bias as a villain. They are right. They do not see Noise that way. They should. In many areas, the current level of Noise is far too high. It is imposing high costs and producing terrible unfairness. What we have catalogued here is the tip of the iceberg. Laws should do much more to reduce those costs and combat that unfairness.”

Cognitive psychology is a major field of interest to me. If you don't know Daniel Kahneman, he is an absolute titan in the psychology field, after having written Thinking Fast and Slow, a book that has been referenced across and influenced many different fields. This was his follow up book, along with two co-authors: Oliver Sibony and Cass Sunstein.

This book examines the phenomenon of Noise and how it affects our judgment. Sources of noise are essentially anything that can impact a person's judgment- lack of sleep, stomach pain, a breakup, or miniscule things you may not even be aware of like being nutrient deficiency. Noise can even be the simple fact that we may treat people who subtly remind us of a sibling in a manner that's different than we would otherwise. Noise impacts the way we make decisions, but it also causes us to be inconsistent in the way we make decisions. The book focuses a fair amount on judges, as they are commonly studied in cases like this, and judges will frequently hand out different sentences across time that are inconsistent with themselves and each other.

Noise also delves into the wisdom of crowds effect, where the aggregate judgment of a group is most likely to be nearly correct, but that individual judgments are usually more wrong. There's lots of little tidbits like this, and the science is far from settled.

The goal we should strive for in society is a reduction of noise. Police, lawyers, politicians, judges, doctors, and many more are all making judgment calls that are rife with noise and ideally, we should be finding ways to limit this.

This was hard to rate as a lot of the info was not new to me, but was presented in new ways. The book is not dense and does a really good job for people new to the subject!

8/10

Wherever there is judgement, there must be noise.

Judgments are human measurements. We predict if a tumor is benign or not, we decide how long the appropriate prison sentence for a certain crime should be, we choose who the best hire for an open position is. And we want these processes to be consistent and fair. We want the sense of security of receiving the same prognosis from different doctors, we want a crime to receive a similar sentence even if judged by different judges, we want the confirmation that the best candidate always receives the job.

But, obviously, this is not the case. Human judgements are clouded by biases and noise. Biases are easier to pick out, easier to attack, easier to take as the scapegoat. But Kahneman & Co show that noise is equally to blame for large variabilities in human judgement.


System noise ... noise observed in organizations that employ interchangeable professionals to make decisions
Level noise ... when some judges are harsher and others are milder, due to the ambiguity of the judgement scale.
Pattern noise ... variability due to each judge's individual opinions and experiences.
Occasion noise ... if the same person judges different based on time of day, mood, or external influence.

Important to note that this book deals with noise in systems where there should be no noise. There are many other domains of judgements where diversity in opinions is welcome.

Interesting topic, going too much into detail for me though. I would have preferred for it to be a bit more concise. But it's probably eye-opening for someone who deals with reproducible human judgement every day.