Retrieving a Request Token from the Twitter API with Python 3

Question

I am attempting to use Python 3 to interface with the Twitter API to return the link for a page that gives me a pin number to use to request access tokens. As detailed here: https://dev.twitter.com/docs/auth/pin-based-authorization

Twitter's API responds to me telling me that I have not properly authorized my POST request by returning a 401. My best guess as to why is that I'm not encoding the HMAC signature in base64 correctly. Everything other part of the POST request I generate appears correct based on samples of correct POST requests I have looked at.

I have spent several days working on this and I am hoping someone can help nudge me past the final part.

Here are the most relevant parts of the Twitter API docs: https://dev.twitter.com/docs/api/1/post/oauth/request_token

https://dev.twitter.com/docs/auth/authorizing-request

This is the code I am using:

import urllib.parse, urllib.request, json
from hashlib import sha1
import hmac
import binascii
import time
import random
import sys

#Server Links
REQUEST_URL = "https://api.twitter.com/oauth/request_token";
ACCESS_URL = "https://api.twitter.com/oauth/access_token";
AUTHORIZE_URL = "https://api.twitter.com/oauth/authorize";

#Consumer keys
TOKEN = "Omitted"
TOKEN_SECRET = "Omitted"

#Access keys
ACCESS_TOKEN = ""
ACCESS_TOKEN_SECRET = ""

TWEET = ""

count = 1

while len(sys.argv) > count:
    TWEET += sys.argv[count] + " "
    count += 1

TWEET = TWEET[:-1] #Get rid of trailing space

print(TWEET + "\n")

#Build content header for POST to return request tokens

HEADER_TITLE = "Authorization:"

#Consumer key
HEADER = 'OAuth oauth_callback="oob" oauth_consumer_key="' + TOKEN + '", '

#Nonce
HEADER += 'oauth_nonce="'
NONCE = ""
for i in range(32):
    NONCE += chr(random.randint(97, 122))
HEADER += NONCE
HEADER += '", '

#Timestamp
TIMESTAMP = str(int(time.time()))

#Signature
HEADER += 'oauth_signature="'
PARAMETER_STRING = "include_entities=true&oauth_consumer_key=" + TOKEN + "&oauth_nonce=" + NONCE + "&oauth_signature_method=HMAC-SHA1&oauth_timestamp=" + TIMESTAMP + "&oauth_version=1.0"
BASE_STRING = 'POST&' + urllib.parse.quote(REQUEST_URL, '') + '&' + urllib.parse.quote(PARAMETER_STRING, '')
SIGNING_KEY = urllib.parse.quote(TOKEN_SECRET, '') + '&'
print("DEBUG : SIGNING KEY " + SIGNING_KEY + " BASE STRING " + BASE_STRING + "\n")
HEADER += str(binascii.b2a_base64(hmac.new(BASE_STRING.encode(), SIGNING_KEY.encode(), sha1).digest()[:-1]))#Note to self, we may not want to remove the last character...
HEADER += '", '

#Signature Method
HEADER += 'oauth_signature_method="HMAC-SHA1", '

#Timestamp
HEADER += 'oauth_timestamp="' + TIMESTAMP + '", '

#Version
HEADER += 'oauth_version="1.0"'

print(HEADER_TITLE + "\n" + HEADER)

print(urllib.request.urlopen(urllib.request.Request(REQUEST_URL, bytes(HEADER_TITLE+HEADER, 'utf-8'))).read())

Finally, I would like to note that I am aware of the existence of Python OAuth and Twitter modules which aid in development for this. However, as a learning experience, I am choosing not to use them.

Thank you very much in advance for your time and assistance.

Answer 1

An overview of what I changed:

1. I swapped out binascii.b2a_base64 for base64.standard_b64encode
2. I converted the bytes to string using the bytes.decode('ascii') method. str() seemed to be appending b to the string.
3. I fixed the order of the parameters to hmac.new - it's KEY, MESSAGE, not MESSAGE, KEY
4. I removed the reference to include_entities in PARAMETER_STRING - if you don't use include_entities in the request (and I believe it doesn't make sense for the token request) it mustn't be included in the PARAMETER_STRING
5. I added oauth_callback=oob to the begin of the PARAMETER_STRING - all oauth parameters except for the oauth_signature must be included in the base string.
6. I changed the section where the request is made to pre-create the Request object, then add the Authorization header - you were sending the Authorization header as the HTTP body.
7. To match this change, I removed the trailing semicolon from HEADER_TITLE.
8. I added the missing semicolon after oauth_callback="oob".

Enjoy!

Here's a fixed version of your code:

import urllib.parse, urllib.request, json
from hashlib import sha1
import hmac
import base64
import time
import random
import sys

#Server Links
REQUEST_URL = "https://api.twitter.com/oauth/request_token";
ACCESS_URL = "https://api.twitter.com/oauth/access_token";
AUTHORIZE_URL = "https://api.twitter.com/oauth/authorize";

#Consumer keys
TOKEN = "Omitted"
TOKEN_SECRET = "Omitted"

#Access keys
ACCESS_TOKEN = ""
ACCESS_TOKEN_SECRET = ""

TWEET = ""

count = 1

while len(sys.argv) > count:
TWEET += sys.argv[count] + " "
count += 1

TWEET = TWEET[:-1] #Get rid of trailing space

print(TWEET + "\n")

#Build content header for POST to return request tokens

HEADER_TITLE = "Authorization"

#Consumer key
HEADER = 'OAuth oauth_callback="oob", oauth_consumer_key="' + TOKEN + '", '

#Nonce
HEADER += 'oauth_nonce="'
NONCE = ""
for i in range(32):
NONCE += chr(random.randint(97, 122))
HEADER += NONCE
HEADER += '", '

#Timestamp
TIMESTAMP = str(int(time.time()))

#Signature
HEADER += 'oauth_signature="'
PARAMETER_STRING = "oauth_callback=oob&oauth_consumer_key=" + TOKEN + "&oauth_nonce=" + NONCE + "&oauth_signature_method=HMAC-SHA1&oauth_timestamp=" + TIMESTAMP + "&oauth_version=1.0"
BASE_STRING = 'POST&' + urllib.parse.quote(REQUEST_URL, '') + '&' + urllib.parse.quote(PARAMETER_STRING, '')
SIGNING_KEY = urllib.parse.quote(TOKEN_SECRET, '') + '&'
print("DEBUG : SIGNING KEY " + SIGNING_KEY + " BASE STRING " + BASE_STRING + "\n")
HEADER += urllib.parse.quote(base64.standard_b64encode(hmac.new(SIGNING_KEY.encode(), BASE_STRING.encode(), sha1).digest()).decode('ascii'))
HEADER += '", '

#Signature Method
HEADER += 'oauth_signature_method="HMAC-SHA1", '

#Timestamp
HEADER += 'oauth_timestamp="' + TIMESTAMP + '", '

#Version
HEADER += 'oauth_version="1.0"'

print(HEADER_TITLE + ":\n" + HEADER)

HTTP_REQUEST = urllib.request.Request(REQUEST_URL)
HTTP_REQUEST.add_header(HEADER_TITLE, HEADER)
print(urllib.request.urlopen(HTTP_REQUEST, bytes('', 'ascii')).read())

Answer 2

Does not look like your signature base string is correct. Basically, you're not including any parameters in the base string as far as I can see.

To quote the twitter docs on the topic of signature base strings:

1. Convert the HTTP Method to uppercase and set the output string equal to this value.
2. Append the '&' character to the output string.
3. Percent encode the URL and append it to the output string.
4. Append the '&' character to the output string.
5. Percent encode the parameter string and append it to the output string.

The parameter string is in turn made like this: (and I am quoting):

1. Percent encode every key and value that will be signed.
2. Sort the list of parameters alphabetically by encoded key.
3. For each key/value pair:
4. Append the encoded key to the output string.
5. Append the '=' character to the output string.
6. Append the encoded value to the output string.
7. If there are more key/value pairs remaining, append a '&' character to the output string.