So I am connecting to the google api client like this.
googleApiClient = new GoogleApiClient.Builder(context)
.addApi(LocationServices.API)
This a bug in Play services library 8.1.0. Solved it by downgrading the play services library.
change
compile 'com.google.android.gms:play-services:8.1.0'
to
compile 'com.google.android.gms:play-services:7.8.+'
And the error no longer appears. My other code worked (as mentioned in the question because they were using the older version of the library) This will have to do until google releases an update.
Update: I wrote this before the other answer by galex came. That is the correct answer. But if you are unable to find the libraries that have these version conflicts, or if you can't change those library dependencies, the solution is to downgrade your play version to 7.8