Python requests_toolbelt MultipartEncoder filename

问题

Using requests_toolbelt to upload large files in a Multipart form, I have constructed a method below which succeeds in uploading the file, however I cannot access the posted filename. How do I access the filename on the server?

# client-side
file = open('/Volumes/Extra/test/my_video.mpg', 'rb')
payload = MultipartEncoder({file.name: file})
r = requests.post(url, data=payload, headers={'Content-Type': 'application/octet-stream'})

# server-side
@view_config(route_name='remote.agent_upload', renderer='json')
def remote_agent_upload(request):
    r = request.response
    fs = request.body_file
    f = open('/Volumes/Extra/tests2/bar.mpg', 'wb')  # wish to use filename here
    f.write(fs.read())
    fs.close()
    f.close()
    return r

回答1:

OK, it looks like you are using the name of the file as the field name. Also, the way that you are doing it, seems like the entire post content is being written to file... Is this the desired outcome? Have you tried to actually play your mpg files after you write them on the server side?

I don't have an HTTP server readily available to test at the moment which automagically gives me a request object, but I am assuming that the request object is a webob.Request object (at least it seems like that is the case, please correct me if I'm wrong)

OK, let me show you my test. (This works on python3.4, not sure what version of Python you are using, but I think it should also work on Python 2.7 - not tested though)

The code in this test is a bit long, but it is heavily commented to help you understand what I did every step of the way. Hopefully, it will give you a better understanding of how HTTP requests and responses work in python with the tools you are using

# My Imports
from requests_toolbelt import MultipartEncoder
from webob import Request
import io

# Create a buffer object that can be read by the MultipartEncoder class
# This works just like an open file object
file = io.BytesIO()

# The file content will be simple for my test.
# But you could just as easily have a multi-megabyte mpg file
# Write the contents to the file
file.write(b'test mpg content')

# Then seek to the beginning of the file so that the
# MultipartEncoder can read it from the beginning
file.seek(0)

# Create the payload
payload = MultipartEncoder(
    {

        # The name of the file upload field... Not the file name
        'uploadedFile': (

            # This would be the name of the file
            'This is my file.mpg',

            # The file handle that is ready to be read from
            file,

            # The content type of the file
            'application/octet-stream'
        )
    }
)

# To send the file, you would use the requests.post method
# But the content type is not application-octet-stream
# The content type is multipart/form-data; with a boundary string
# Without the proper header type, your server would not be able to
# figure out where the file begins and ends and would think the
# entire post content is the file, which it is not. The post content
# might even contain multiple files
# So, to send your file, you would use:
#
# response = requests.post(url, data=payload, headers={'Content-Type': payload.content_type})

# Instead of sending the payload to the server,
# I am just going to grab the output as it would be sent
# This is because I don't have a server, but I can easily
# re-create the object using this output
postData = payload.to_string()

# Create an input buffer object
# This will be read by our server (our webob.Request object)
inputBuffer = io.BytesIO()

# Write the post data to the input buffer so that the webob.Request object can read it
inputBuffer.write(postData)

# And, once again, seek to 0
inputBuffer.seek(0)

# Create an error buffer so that errors can be written to it if there are any
errorBuffer = io.BytesIO()

# Setup our wsgi environment just like the server would give us
environment = {
    'HTTP_HOST': 'localhost:80',
    'PATH_INFO': '/index.py',
    'QUERY_STRING': '',
    'REQUEST_METHOD': 'POST',
    'SCRIPT_NAME': '',
    'SERVER_NAME': 'localhost',
    'SERVER_PORT': '80',
    'SERVER_PROTOCOL': 'HTTP/1.0',
    'CONTENT_TYPE': payload.content_type,
    'wsgi.errors': errorBuffer,
    'wsgi.input': inputBuffer,
    'wsgi.multiprocess': False,
    'wsgi.multithread': False,
    'wsgi.run_once': False,
    'wsgi.url_scheme': 'http',
    'wsgi.version': (1, 0)
}

# Create our request object
# This is the same as your request object and should have all our info for reading
# the file content as well as the file name
request = Request(environment)

# At this point, the request object is the same as what you get on your server
# So, from this point on, you can use the following code to get
# your actual file content as well as your file name from the object

# Our uploaded file is in the POST. And the POST field name is 'uploadedFile'
# Grab our file so that it can be read
uploadedFile = request.POST['uploadedFile']

# To read our content, you can use uploadedFile.file.read()
print(uploadedFile.file.read())

# And to get the file name, you can use uploadedFile.filename
print(uploadedFile.filename)

So, I think this modified code will work for you. (Hopefully) Again, not tested because I don't actually have a server to test with. And also, I don't know what kind of object your "request" object is on the server side.... OK, here goes:

# client-side
import requests
file = open('/Volumes/Extra/test/my_video.mpg', 'rb')
payload = MultipartEncoder({'uploadedFile': (file.name, file, 'application/octet-stream')})
r = requests.post('http://somewhere/somefile.py', data=payload, headers={'Content-Type': payload.content_type})

# server-side
@view_config(route_name='remote.agent_upload', renderer='json')
def remote_agent_upload(request):

    # Write your actual file contents, not the post data which contains multi part boundary
    uploadedFile = request.POST['uploadedFile']
    fs = uploadedFile.file

    # The file name is insecure. What if the file name comes through as '../../../etc/passwd'
    # If you don't secure this, you've just wiped your /etc/passwd file and your server is toast
    # (assuming the web user has write permission to the /etc/passwd file
    # which it shouldn't, but just giving you a worst case scenario)
    fileName = uploadedFile.filename

    # Secure the fileName here... 
    # Make sure it doesn't have any slashes or double dots, or illegal characters, etc.
    # I'll leave that up to you

    # Write the file
    f = open('/Volumes/Extra/tests2/' + fileName, 'wb')
    f.write(fs.read())

回答2:

Probably too late for the original OP but may help someone else. This is how I upload a file with accompanying json in a multipart/form-data upload using the MultipartEncoder. When I require a file to uploaded as binary with a single json string as part of a multipart request (so there are just two parts, the file and the json). Note that in creating my request header (it's a custom header as designated by the receiving server) I get the content_type from the encoded object (it usually comes through as multipart/form-data). I'm using simplejson.dumps but you can just use json.dumps I believe.

m = MultipartEncoder([
    ('json', (None, simplejson.dumps(datapayload), 'text/plain')),
    ('file', (os.path.basename(file_path), open(file_path, 'rb'), 'text/plain'))],
    None, encoding='utf-8')

headers = {'Authorization': 'JwToken' + ' ' + jwt_str, 'content-type': m.content_type}
response = requests.post(uri, headers=headers, data=m, timeout=45, verify=True )

In the file part, the field is called "file", but I use os.path.basename(file_path) to get just the filename from a full file path e.g. c:\temp\mytestfile.txt . It's possible I could just as easily call the file something else (that's not the original name) in this field if I wanted to.

来源：https://stackoverflow.com/questions/30449750/python-requests-toolbelt-multipartencoder-filename

标签

python

python-requests