» Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart

Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart Reviews

Customer Rating:
Summary: What you can do with large datasets
Comment: The answer is of course: a lot.
And Ian Ayres' book will tell you a little about it.

Supercrunchers are those who use lage datasets
to find patterns in human behaviour, and
predict the future based on these large datasets.

The book informs us that super crunching is on the verge of being
used all over. E.g.
Chess grandmaster Kasparov was no match
for IBMs Deep Blue chess computer,
that stored some 700.000 grandmaster chess games to help find the
winning move.
The IRS could use its data to tell a small business,
if it is spending too much or too little on advertising.
Indeed, the IRS probably has enough data to
make good estimates on whether business, marriages, etc. etc.
will fail - based only on comparison with its existing dataset.

For the paranoid, it is a horror that supermarkets could map your life cycle and predict your next purchases pretty accurately (based on
what other similar customers did).
For the optimist data mining is a good thing and we'll all lead better lives because of it.

Want to write a bestseller about it? Compare your title and some key words with data from a database of books, titlescore.com, containing millions of bestsellers and flops, and you will get your answer.

It all seems pretty straight forward, and the book has some nice examples of what we can expect in the coming years.

-Simon

---

Customer Rating:
Summary: Weak Book, not original material
Comment: This is new? The notion that empirical research is useful has been dealt with in book after book. The book not only recycles stories word for word without quote marks from the New York Times and other publications. There are hundreds of books that show that empirical work can help understand the world. What is new? What is interesting that is new here?

---

Customer Rating:
Summary: comme ci, comme ça
Comment: It comes on the heals of some really great non-fiction analytical books. Unfortunately, this book is all anecdotal and lacks real substance. It is good for non-mathematical, non-analytical people, but not good for people with solid educations in math, statistics, and data analysis.

---

Customer Rating:
Summary: Freakonomics 2: enjoyable survey of interesting research with real-world impacts
Comment: Ayres demonstrates how statistical analysis of large datasets is affecting the way the world works in a broad range of applications: credit card companies, sports teams, wine critics, development economists, medical practitioners,* law enforcement agencies, schools, etc. "Freakonomics didn't talk much about the extent to which quantitative analysis is impacting real-world decisions. In contrast, this book is about just that - the impact of number crunching" (p13).

As an economist, some of the work is familiar (for example, the research Ayres and Steve Levitt did on the value of the vehicle-recovery device LoJack or the Poverty Action Lab), but Ayres gives a good introduction for the uninitiated. And he covers such a broad range of applications that I learned a great deal.

Like other research surveys (Freakonomics, The Tipping Point, Blink, Stumbling on Happiness), I view these books mostly as surveys of interesting research. Each has a central thesis (Ayres' is that traditional intuition and expertise will be - or already has been - replaced by computing power and will have to learn to complement that power rather than compete with it) which may or may not be convincing, but the books tend to be good rides because so much of the surveyed research is interesting. (For example, I'll be studying more about Direct Instruction - a scripted way of teaching reading that may be useful in my own work - based on this book; and the model Ayres expounds of how private firms learn from iterative experimental trials may apply well to some of the agencies I engage.)

As far as Ayres' thesis goes, I find him relatively convincing (computers with lots of data do predict many things better than people**) but despite his many caveats, the tone should probably have been more humble. He doesn't - for example - explore the issues brought by Taleb in The Black Swan: The Impact of the Highly Improbable, how traditional statistics may be worse than useless in financial markets where a single, completely unpredictable bad shock can wipe out years of carefully predicted investments.

This book was lots of fun to listen to, not least (unintentionally) because Ayres loves giving irrelevant but amusing descriptions of his researchers. The examples below are all economists:

"Ashenfelter is a tall man with a bushy mane of white hair and a booming, friendly voice... No milquetoast he" (p2).

"Even now, in his forties, Larry [Katz] still looks more like a wiry teenage than a chaired Harvard professor (which he actually is)" (p65).

"Esther [Duflo] has endless energy. A wiry mountain climber..." (p73).

And of course you know this is the Freakonomics family because of the Levitt-love scattered here and there: "There is a new breed of innovative Super Crunchers - people like Steve Levitt - who toggle between their intuitions and number crunching to see farther than either intuitivists or gearheads ever could before" (p17).

I listened the unabridged audiobook narrated by Michael Kramer (not Michael Kremer - quoted in this book on p74), published by Books on Tape (6 CDs). Kramer does a good job except when he tries an Australian or British accent.

* For an excellently written description of evidence-based medicine and more, read Atul Gawande's Better: A Surgeon's Notes on Performance.

** One of the most striking findings comes from the meta-analysis (1996) of two psychologists, Meehl & Grove, who look at 136 studies comparing human judgment to equation-based judgment. In only 8 of the 136 studies was expert prediction found to be appreciably more accurate than statistical prediction." Overall, experts got the predictions right 66% of the time whereas Super Crunchers got them right 73% of the time. And the 8 in which experts did better weren't concentrated in any particular field. From looking at the paper myself, I found that 64 of the studies favored the Super Crunchers whereas 64 found the two methods roughly equal. Noteworthy. [In the book, p111 and p232.]

---

Customer Rating:
Summary: For the ignorant: shocking, for the initiates: fun. For the cognoscenti: lame
Comment: Super crunchers is a nice little book for those who are statistically challenged, but for those who know about joint probability distributions and copula methods, you miss the "gee whiz!" factor of "can they really figure that out?"

Still, it is a fun read in how large data sets are becoming a norm, and like all residue of science can be used for good or evil depending on the moral skew of those who hold the diagnosis. For the paranoid, it is a horror that Wal-mart can map your life cycle and predict our next purchases pretty accurately. But the major theme of this book is that data mining is a good thing and we'll all lead better lives and corporations will engage in less waste as we refine techniques that are targeted and discard techniques that are blankets (and hence create noise).

The first half of the book is better than the second half, so I suspect this was once really a long Atlantic Monthly article stretched for a mini-book. Still, it is a thinking man's plane ride of a paperback.

---

More Reviews for Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart