My authorization flow in the new Google Drive Android API is as follows:
It sounds like you are relying on the default account selection. In this setup, the user is prompted once to select an account, and this status is remembered.
If you want to provide account switching capabilities in your app, you need to instead launch the account picker from your own app and provide the account name selected when you instantiate the GoogleApiClient.
You could persist the last selected account name in shared preferences so you remember it until the next time the user switches accounts.