100 books like Probability

While studying computer networks, Claude Shannon did something pretty impressive. He reformulated the majority of classical statistics from scratch using the language and concepts of computer science. 

Statistical noise? There’s a new word for that; it’s called entropy. Also, it turns out it is a good thing, not a bad thing because entropy is equal to the information content or a data set. Tired of minimizing the squared error of everything? That’s fine, minimize the log of its likelihood instead. It does the same thing. This book challenges the assumptions of classical statistics in a way that fits neatly in the mind of a computer scientist. As a quant trader, this book will help you understand and measure the information content of data, which is critical to your success.

This book might as well be called Introduction to machine learning, and it is probably one of the only books truly deserving of the title. Did you know neural networks have been used for decades to scan checks at the bank? They are called Boltzman Machine. Have you ever heard of how decision trees were used in old-school data mining? You could only get them from proprietary software packages from the early 2000s.

In quant trading, you will constantly face compute power constraints, so it is invaluable to understand the mathematical foundations of the most old-school machine learning methods out there. Researchers 20 years ago used to do a lot of impressive work with a lot less computing power.

One of my favorite professors, Gretchen Martinet, used this to teach a course called “Mathematical Statistics” when I was at the University of Virginia. It is an extremely profound course full of dense but fundamental mathematical proofs in classical statistics. 

You will learn why the formula for the normal distribution is the way it is, why the sum of squares appears everywhere in statistics, and how to fit a linear regression by hand. In the same way calculus elevates our understanding of rates of changes, the book elevates your understanding of samples, averages, and distributions. Quant trading requires an intuitive sense of how data, models, and aggregates work, making this content essential for your success.

This is one of my favorite underappreciated statistics books of all time. Non-parametric statistics can be otherwise described as statistics without assumptions. The entire goal of this field of study is to prove X is greater than Y without making any assumptions about the underlying distributions of X or Y. The methods are different, and they require more data than other methods, but the learning journey is invaluable.

I personally believe that modern machine learning is simply the modeling section of the school of non-parametric statistics. Working through this book will give you a much deeper understanding of why tools like decision trees are so valuable. It will also to teach you to design rigorous numerical experiments on data sets that are beyond the help of classical statistics.

When people think of financial planning, their first thought is investing. Their second thought is retirement.

Kaplans explain risk succinctly: “Everything is possible, yet only one thing happens.” People understand risk but don’t really understand how to apply it rationally to investing (market risks) or to retirement (longevity risk).

But first, having an understanding of what risk is and isn’t, and where it comes from is important before you can apply it to what fuels your plans – markets and longevity.

This book helped me formulate the basic planning concepts I use in my book since personal finance is all about taking risks – as are any other decisions and actions you take in life.

This is a more comprehensive and mathematically rigorous book than Pierce’s book. For the novice, it should be read-only after first reading Pierce’s more informal text. Due to its vintage, the layout is fairly cramped, but the content is impeccable. At almost 500 pages, it covers a huge amount of material. This was my main reference book on information theory for many years, but it now sits alongside more recent texts, like MacKay’s book (see below). It is also published by Dover, so it is reasonably priced.

My knees tremble and my heart quakes when I think of how much work must have gone into these two companion volumes. Collectively, they are more than four times the length of my book, covering the whole of machine learning.

It is an essential encyclopedic resource that should be on the desk of anyone serious about machine learning.

I found a copy of this book in the sixties. That copy, much loved, was lost in moves and mayhem. Now, I only have a Dover reprint (water-logged during yet another move), but I have never been without and would search high and low if I were.

This is also a history of probability but with a very different focus. Ms. David was a statistician able to explain the calculations intuitively (good to assign to my students). But she was also thoroughly interested in the personalities involved. What was Galileo like? What happened to Pascal at Port-Royal?

I felt personally drawn into the historical narrative that often reads like a novel. 

Sadly, Games, Gods, and Gambling by FN David is out of print. This is the next best thing. Lorraine Daston has the supreme gift of making the complicated idea seem straightforward. This is an account of the frenzy for measuring that happened in the 18th century, and how it made the world we live in today, when the gambler’s eye for odds has become the algorithm of taming chance that guides all our decisions.

US economist Frank Knight is credited with distinguishing uncertainty from risk back in 1921. Yet the two are often conflated.

Kay (an eminent economist) and King (a former Governor of the Bank of England) argue powerfully that the distinction does matter. They range widely across macroeconomics, politics, and consumer choices to show why reducing the future to a set of numbers (probabilities) creates a false – and often disastrous – illusion of power over future outcomes.

They argue that instead we should aim to make decisions that stand a reasonable chance of being robust against unknowable, as well as forecastable, paths that the future might take. That’s very much the ethos of my own books: building in resilience is a key part of successful personal financial planning.