Gun Control and Safety

Another mass shooting happened today. I don’t usually write about this, but it happened in Roseberg, Oregon, close to where I live. Too close. I do understand that it is morally equivalent to a mass shooting in Ethiopia, Indonesia, the Netherlands or Australia, but something about events being close by have a bigger emotional impact.

The question I want to address in this article is whether legislation can help solve this problem. I’ll focus on whether it is effective to implement barriers to gun ownership.

The inspiration for this article is this graphic by Libby Isenstein:

Libby Isenstein's infographic on guns

My methodology is to see if there is a correlation between the number of gun-control-related laws in a state and the number of gun-related deaths. The columns in the above table will each count for up to one point. For example, if a state doesn’t require a permit for a gun, it gets 0, and if it does require a permit, it gets 1. For columns related to the difficulty of getting a gun, easy gets 0 points, moderate gets 0.5 points, and difficult gets 1 point.

These points are then added up, and we look at all the states (plus DC), as pairs (sum of points, gun deaths per 100,000 people)


This has a correlation coefficient of 0.52 (1.0 being a perfect fit), what that means is that there is a moderate trend between the degree of gun control and the number of gun-related deaths. It doesn’t prove that gun control laws cause fewer gun deaths, but it is at least consistent with that claim. And more importantly, it is evidence against the argument that more widespead access to guns make us all safer.

No person should die for no reason. If legislation can help solve this problem (there’s reason to think it can), then it’s our duty to make that come to pass. Unfortunately, certain interpretations of the Second Amendment currently in force have prevented many sane limits on access to guns, this article by John Paul Stevens in the Washington Post explain a little history behind that.

For more detail on how to perform the analysis above, see below (requires familiarity with python, or a similar language).

import pandas as pd
import numpy as np
df = pd.read_csv("guns.csv")
deaths state laws
0 2.6 HI 7.0
1 3.1 MA 5.0
2 4.2 NY 6.0
3 4.4 CT 4.5
4 5.3 RI 4.5
import matplotlib.pyplot as plt
%matplotlib inline
plt.ylabel("gun deaths per 100k")
plt.xlabel("number of gun laws")
plt.scatter(df["laws"], df["deaths"])
<matplotlib.collections.PathCollection at 0x10d678850>


Since it looks like a downward trend, let’s see how well a linear model fits this data.

from sklearn import linear_model
reg = linear_model.LinearRegression() 
X = df["laws"].values.reshape((len(df),1))
y_true = df["deaths"].values.reshape((len(df),1)),y_true)
y_pred = reg.predict(X)
plt.ylabel("gun deaths per 100k")
plt.xlabel("number of gun laws")
<matplotlib.collections.PathCollection at 0x10e980fd0>


It looks like a really good, fit, but to make it precist, we should find the $R2$ score.

from sklearn.metrics import r2_score
r2_score(y_true, y_pred)

Review: The Warmth of Other Suns by Isabel Wilkerson

Isabel Wilkerson’s riveting book about the Great Migration of African-Americans out of the south is the perfect blend of narrative and statisitcs.

The personal stories of Ida Mae Brandon Gladney, George Swanson Starling and Robert Joseph Pershing Foster are the substantive bulk of the book. Around these personal stories are statistics that serve to add context and clarity to a movement that was so large, and took place over a long enough time, that it’s easy to miss it as a distinct historical period.

This book was my introduction to the Great Migration. It was a migration that began in 1915 with World War 1 and ended around 1970. Nearly 6 million African Americans left the South and traveled to cities in the North or the West, such as New York, Chicago, Baltimore, Cleveland, Los Angeles and Oakland. It wasn’t a coordinated effort, it was the result of large economic and social forces, as well as millions of personal decisions to leave a part of the United States that historically had treated black people as property, and hadn’t improved much since the end of the civil war.

Several decades after the end of the civil war, even though slavery had been abolished, the old social order still existed, the same industries existed. Slavery had been replaced with the slightly less bad, but still quite terrible practice of sharecropping, where white planation owners would hire former slaves and children of former slaves as laborers. They usually took advantage of the fact that the black workers weren’t educated, effectively getting free or near-free labor. The de facto segregation that existed quickly became de jure segregation with the passing of Jim Crow laws in most southern states and counties. This began to change in a big way during World War 1, since the men that left for war in the north left a labor vacuum. Recruiters from the north would go south, trying to convince many black residents in the south to move north for jobs. This was just the beginning, since the family and friends of those first migrants kept that route in their minds as a possible option for a better life.

The book proceeds by moving through three parallel timelines, one of Ida Mae, a sharecropper’s wife from Missippi, who left in the 1930s. Then George Starling, a fruit picker from Florida, and finally Robert Foster, from Lousiana. The stories are told in parallel, but they all take place about a decade apart from one another. Wilkerson’s narrative style is so good, I sometimes felt like the stories were happening in the present.

In between the personal stories were discussions relevant statistics, such as how many people had left which state, and incomes of black teachers and white teachers that were similarly qualified. The book does doesn’t just steep the reader in data, it moves briskly back into narrative and continues the unfolding saga of the three brave migrants.

There were two parts of the book that stood out most to me, one having to do with wealth inequality:

In neighboring Mississippi, white teachers and principals were making $630 a year, while the colored ones were paid a third of that-$215 a year, hardly more than field hands. But knowing that didn’t ease the burden of the Foster’s lives, get their children through college, or allow them to build assets to match their status and education. The disparity in pay, reported without apology in the local papers for all to see, would have far-reaching effects. It would mean that the most promising of colored people, having received next to nothing in material assets from their slave foreparents, had to labor with the knowledge that they were now being underpaid by more than half, that they were so behind it would be all but impossible to accumulate the assets their white counterparts could, and that they would, by definition, have less to leave succeeding generations than similar white families.

This was followed by the long term consequences, which influence the world of today (2015):

Multiplied over the generations, it would mean a wealth deficit between the races that would require a miracle windfall or near aceticism on the part of colored families if they were to have any chance of catching up or amassing anything of value. Otherwise, the chasm would continue, as it did for blacks as a group even into the succeeding century. The layers of accumulated assets built up by the better-paid dominant caste, generation after generation, would factor into a wealth disparity of white Americans having an average net worth ten times that of black Americans by the turn of the twenty-first century, dampening the economic prospects of the children and grandchildren of the Great Migration before they were even born.

This explanation clarified the concept of white privilege to me better than anything I’ve seen before, or since. In fact, it’s gotten slightly worse since the turn of the century, a recent Pew Research study found that the average net worth of white families is thirteen times that of the average black family:

The other part that stood out to me was about automation and jobs, the story about technological unemployment usually goes: first a new technology emerges that makes some kind of human labor obsolete, and then those laborers lose their jobs, and get replaced by machines. While there are many examples of this, reality is much more complicated. Here’s the quote from the book related to this topic:

It took World War II and the even bigger outflow of blacks to awaken them to what some agricultural engineers working on a mechanical harvester already knew: “Much of this labor is not returning to the farm,” Harris P. Smith, the chief of agricultural engineering at Texas A&M University, wrote in 1946. “Therefore, the cotton farmer is forced to mechanize.” As for the connection between the Migration and the machine, Smith concluded that “instead of the machines displacing labor, they were used to replace the labor that had left the farm”.

In the case of cotton picking in the south, automation didn’t destroy jobs, it filled in a void of jobs that people willingly left. This aligns with my ideas about what kinds of work should be replaced by machines. Anything repetitive and degrading should be done by a machine. Humans are too important.

The book is well worth your time, it is very informative, eye-opening, and incredibly well written.


[1] Wilkerson, Isabel. The Warmth of Other Suns: The Epic Story of America’s Great Migration. New York, NY: Random House, 2010. Print.

[2] Wealth inequality has widened along racial, ethnic lines since end of Great Recession

Please Steal This Blog Post

I believe that free access to information is a human right. The Internet has dramatically enhanced our ability to exercise this right, but unfortunately most humans cannot access the Internet. Today, over 4.3 billion people cannot connect to the Internet at all and another roughly 1 billion people have their Internet connections censored or monitored. A world where only 20% of humans have truly free access to digital information is unacceptable.

That is why I support the creation of Humanity’s Public Library, an initiative by Outernet. Outernet broadcasts a data signal from satellites that is free to receive anywhere on Earth. While this is not an Internet connection, it is a free stream of critical information. What information is considered “critical?” You decide.

Outernet and Creative Commons are co-hosting the first edit-a-thon for Humanity’s Public Library on July 18-19 2015 to decide what is included in this library. Anyone on Earth is encouraged to participate - details on how to have your voice heard in this process can be found at

I want to encourage our users to submit their own work and to submit content that is licensed for redistribution. One such work is this very blog post. Copy these words and post them on your own blog and let’s all gather together and build a #LibraryFromSpace.

This blog post is licensed under CC0 and is free to be distributed and edited without restriction.

Bayesian Drunk Driving

Driving drunk is illegal for a good reason, it’s way riskier than driving sober. This article isn’t about driving drunk though, it’s more about the sloppy thought processes that can too easily confuse something as obvious as that first sentence. Here’s an example of a bogus argument that appears to support the idea that drunk driving is actually safer:

So the argument is as follows: In 2012, 10,322 people were killed in alcohol-impaired driving crashes, accounting for nearly one-third (31%) of all traffic-related deaths in the United States [1]. That means that approximately one third of traffic-related deaths involve drunk driving, meaning that two thirds of traffic-related deaths don’t involve drunk driving. Therefore, sober drivers are twice as likely to die in a traffic accident.

If you think something is wrong with that argument, you are right, but it’s not just because the conclusion intuitively seems wrong, it’s because it involves a mistake in conditional probability. To see the mistake, it helps to introduce a litle notation, we will define:

  • P(D) to be the probability that a person is drunk
  • P(A) to be the probability that a person will die in a traffic-related accident
  • P(D | A) (pronounced probability of D given A) is the probability that a person is drunk, given that there was a death in a traffic-related accident they were in

So using the 2012 CDC data, we can assign 31%, P(D | A) = 0.31. This is that the probability of a drunk driver being involved given that there was a deadly driving accident.

The first thing to point out is that the statement that ‘sober drivers are twice as likely as drunk drivers to die in an accident’ is really a statement about P(A | D), that is, the probability of a deadly driving accident given that that person is drunk. We don’t know this yet, however, we can figure it out using Bayes’ theorem.

Bayes’ Theorem

Bayes’ Theorem is unusual in that it is extremely useful and easy to prove, but hard to really understand. This is something I learned several times in college, but never really understood it’s importance until much later. To see how easy to prove it is, we go back to the definition of conditional probability:

Where P(X ∩ Y) is the probability of X and Y occurring. Since this is true for any pair of events X and Y, we can reverse them and get

Also, remember that AND is commutative, so that P(X ∩ Y) = P(Y ∩ X), so we can multiply the above two equations by P(Y) and P(X), respectively, to get:

This relates P(X|Y) to P(Y|X), P(X) and P(Y), we can solve the above equation to get:

And that’s it, we took the definition of conditional probability, did a little algebra, and out popped Bayes’ theorem, we can now apply this to the above drunk driving fallacy, and calculate the probability that we are interested in, that is, P(A | D).

Since we know P(D|A), we just need to find P(A) and P(D). Since the CDC data we are using is annual data, we need to take the number of casualties from deadly accidents in the United States for the year of 2012 (33,561) and divide by the number of drivers (211,814,830), that gives an estimate of P(A) = 33,561/211,814,830 = 0.0001584, which is about 1 in 6,313.

Next, we need to find the probability that a driver is drunk P(D), we will use the data from the study referenced in [3], and define ‘drunk’ to be a BAC of ≥ 0.1%. Then P(D) = 0.00387 or about 1 in 258 (more on this calculation in the notes below).

Now that we have:

P(D|A) = 0.31 ( probability of a driver being drunk, given they were involved in an accident where someone died ),

P(A) = 0.0001584 ( probability of a driver being involved in an accident where someone died ), and

P(D) = 0.00387 ( probability of a driver being drunk )

We can figure out P(A|D) ( probability of a drunk driver getting into a deadly accident )

P(A|D) = P(D|A)P(A)/P(D) = (0.31*0.0001584)/0.00387 = 0.0127 (1.27 %)

1.27% is not insignificant, it’s about half the probability of rolling snake eyes in craps. Now, let’s compare that to sober driving, we just need to calculate P(A|Dc). We can use Kolmogorov’s Theorem of total probability, shuffle a few terms to get:

P(A|Dc) = (P(A) - P(A|D)P(D))/P(Dc) = (0.0001584 - 0.0127*0.00387)/(1-0.00387) = .000109, which is about 1 in 9118.


So the probability of getting in a deadly accident, given that you are drunk is 1.27%, and the probability of getting into a deadly accident, given that you are not drunk is .01%, that means that it is 127 times more likely that you will get into a deadly accident while drunk.


[1] Impaired Driving: Get the Facts Centers for Disease Control [link]

[2] Total licensed drivers U.S. Department of Transportation Federal Highway Administration [link]

[3] Probability of arrest while driving under the influence (George A Beitel, Michael C Sharp, William D Glauz) [link]

Notes on [3], we don’t technically have P(D), but we do have P(D|A1), P(A1), and P(A1|D), where A1 is the event that a person is arrested. We can then find P(D) = (P(D|A1)P(A1))/P(A1|D) = (0.06×0.000374)/0.0058 = .00387.

Word Frequencies After Removing Common Words

In taking the Coursera class on Mining Massive Datasets, the problem of computing word frequency for very large documents came up. I wanted some convenient tools for breaking documents into streams of words, and also a tool to remove common words like ‘the’, so I wrote up words and decommonize. The decommonize script is just a big grep -v '(foo|bar|baz)', where the words foo, bar and baz come from the words in a file. I made a script generate_decommonize that reads in a list of common words, and builds the regex for grep -v.

Example usage of words and decommonize

The full source code is available here on github.

After running make install, you should have words and decommonize in your PATH, you can use them to find key words that are characteristic of a document, I chose

  • the U.S. Declaration of Independence:

$ words < declaration_of_independence.txt | decommonize  | sort | uniq -c | sort -n | tail
   4 time
   5 among
   5 most
   5 powers
   6 government
   6 such
   7 right
   8 states
   9 laws
  10 people
  • Sherlock Holmes

$ words < doyle_sherlock_holmes.txt | decommonize  | sort | uniq -c | sort -n | tail
 174 think
 175 more
 177 over
 212 may
 212 should
 269 little
 274 mr
 288 man
 463 holmes
 466 upon
  • Working with Unix Processes (by @jstorimer)

$ words < working_with_unix_processes.txt | decommonize  | sort | uniq -c | sort -n | tail
  73 signal
  82 system
  88 ruby
  90 exit
 100 code
 100 parent
 143 its
 146 child
 184 processes
 444 process

So words breaks up the document into lower-case alphabetic words, then decommonize greps out the common words, and sort and uniq -c are used to count instances of each decommonized word, and then the results are sorted.