How to prevent python requests from percent encoding my URLs?

问题

I\'m trying to GET an URL of the following format using requests.get() in python:

http://api.example.com/export/?format=json&key=site:dummy+type:example+group:wheel

#!/usr/local/bin/python

import requests

print(requests.__versiom__)
url = \'http://api.example.com/export/\'
payload = {\'format\': \'json\', \'key\': \'site:dummy+type:example+group:wheel\'}
r = requests.get(url, params=payload)
print(r.url)

However, the URL gets percent encoded and I don\'t get the expected response.

2.2.1
http://api.example.com/export/?key=site%3Adummy%2Btype%3Aexample%2Bgroup%3Awheel&format=json

This works if I pass the URL directly:

url = http://api.example.com/export/?format=json&key=site:dummy+type:example+group:wheel
r = requests.get(url)

Is there some way to pass the the parameters in their original form - without percent encoding?

Thanks!

回答1:

It is not good solution but you can use string:

r = requests.get(url, params='format=json&key=site:dummy+type:example+group:wheel')

BTW:

payload = {'format': 'json', 'key': 'site:dummy+type:example+group:wheel'}

payload_str = "&".join("%s=%s" % (k,v) for k,v in payload.items())
# 'format=json&key=site:dummy+type:example+group:wheel'

r = requests.get(url, params=payload_str)

回答2:

The solution, as designed, is to pass the URL directly.

回答3:

In case someone else comes across this in the future, you can subclass requests.Session, override the send method, and alter the raw url, to fix percent encodings and the like. Corrections to the below are welcome.

import requests, urllib

class NoQuotedCommasSession(requests.Session):
    def send(self, *a, **kw):
        # a[0] is prepared request
        a[0].url = a[0].url.replace(urllib.quote(","), ",")
        return requests.Session.send(self, *a, **kw)

s = NoQuotedCommasSession()
s.get("http://somesite.com/an,url,with,commas,that,won't,be,encoded.")

回答4:

The answers above didn't work for me.

I was trying to do a get request where the parameter contained a pipe, but python requests would also percent encode the pipe. So instead i used urlopen:

# python3
from urllib.request import urlopen

base_url = 'http://www.example.com/search?'
query = 'date_range=2017-01-01|2017-03-01'
url = base_url + query

response = urlopen(url)
data = response.read()
# response data valid

print(response.url)
# output: 'http://www.example.com/search?date_range=2017-01-01|2017-03-01'

回答5:

Please have a look at the 1st option in this github link. You can ignore the urlibpart which means prep.url = url instead of prep.url = url + qry

来源：https://stackoverflow.com/questions/23496750/how-to-prevent-python-requests-from-percent-encoding-my-urls

标签

python

url

python-requests

percent-encoding