Python Requests: Post JSON and file in single request

Question

I need to do a API call to upload a file along with a JSON string with details about the file.

I am trying to use the python requests lib to do this:

import requests

info = {
    'var1' : 'this',
    'var2'  : 'that',
}

data = json.dumps({
    'token' : auth_token,
    'info'  : info,
})

headers = {'Content-type': 'multipart/form-data'}

files = {'document': open('file_name.pdf', 'rb')}

r = requests.post(url, files=files, data=data, headers=headers)

This throws the following error:

    raise ValueError("Data must not be a string.")
 ValueError: Data must not be a string

If I remove the 'files' from the request, it works.
If I remove the 'data' from the request, it works.
If I do not encode data as JSON it works.

For this reason I think the error is to do with sending JSON data and files in the same request.

Any ideas on how to get this working?

Answer 1

Don't encode using json.

import requests

info = {
    'var1' : 'this',
    'var2'  : 'that',
}

data = {
    'token' : auth_token,
    'info'  : info,
}

headers = {'Content-type': 'multipart/form-data'}

files = {'document': open('file_name.pdf', 'rb')}

r = requests.post(url, files=files, data=data, headers=headers)

Note that this may not necessarily be what you want, as it will become another form-data section.

Answer 2

See this thread How to send JSON as part of multipart POST-request

Do not set the Content-type header yourself, leave that to pyrequests to generate

def send_request():
payload = {"param_1": "value_1", "param_2": "value_2"}
files = {
     'json': (None, json.dumps(payload), 'application/json'),
     'file': (os.path.basename(file), open(file, 'rb'), 'application/octet-stream')
}

r = requests.post(url, files=files)
print(r.content)

Answer 3

I'm don't think you can send both data and files in a multipart encoded file, so you need to make your data a "file" too:

files = {
    'data' : data,
    'document': open('file_name.pdf', 'rb')
}

r = requests.post(url, files=files, headers=headers)

Answer 4

What is more:

files = {
    'document': open('file_name.pdf', 'rb')
}

That will only work if your file is at the same directory where your script is.

If you want to append file from different directory you should do:

files = {
    'document': open(os.path.join(dir_path, 'file_name.pdf'), 'rb')
}

Where dir_path is a directory with your 'file_name.pdf' file.

But what if you'd like to send multiple PDFs ?

You can simply make a custom function to return a list of files you need (in your case that can be only those with .pdf extension). That also includes files in subdirectories (search for files recursively):

def prepare_pdfs():
    return sorted([os.path.join(root, filename) for root, dirnames, filenames in os.walk(dir_path) for filename in filenames if filename.endswith('.pdf')])

Then you can call it:

my_data = prepare_pdfs()

And with simple loop:

for file in my_data:

    pdf = open(file, 'rb')

    files = {
        'document': pdf
    }

    r = requests.post(url, files=files, ...)

Answer 5

For sending Facebook Messenger API, I changed all the payload dictionary values to be strings. Then, I can pass the payload as data parameter.

import requests

ACCESS_TOKEN = ''

url = 'https://graph.facebook.com/v2.6/me/messages'
payload = {
        'access_token' : ACCESS_TOKEN,
        'messaging_type' : "UPDATE",
        'recipient' : '{"id":"1111111111111"}',
        'message' : '{"attachment":{"type":"image", "payload":{"is_reusable":true}}}',
}
files = {'filedata': (file, open(file, 'rb'), 'image/png')}
r = requests.post(url, files=files, data=payload)