Scrap peekyou.com ( having POST METHOD)

问题

Please see the output which I am getting I am trying to scrap peekyou.com which is kind of peoples search engine. They use POST method of php.I am using requests.post method of requests library to scrap the results .

suppose a persons name is "john coasta" then the target url would be : peekyou.com/john_coasta

import requests
import json


payload = { 'formdata' : {'md5': '4a9050a569e0f7d862b771926f7abc57',
             'asynchronous': 'true'}

}
req = requests.post('https://www.peekyou.com/shantanu_sharma',
                 data = payload,
                  headers={ 'X-Requested-With': 'XMLHttpRequest',
                           'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',
                           'referer': 'https://www.peekyou.com/shantanu_sharma',
                            'server':'Apache/2.4.33 (FreeBSD) OpenSSL/1.0.2k-freebsd mod_fastcgi/mod_fastcgi-SNAP-0910052141'


}
                   )
print(req.content)

although I am getting the full result in HTML form , the result which I am seeking for is encoded(I need decoded o/p) in the characters like :\n\t ( inside every HTML tag {surprisingly this is the actual result}).I didn't use POST requests frequently. Please provide me some solution.

Thanks in advance

回答1:

the result which I am seeking for is encoded in the characters like :\n\t

maybe the response is blank becuase you are doing something wrong? when i opened that site, I found that it uses a lot of Cookies,but you are not using any cookies.If you are sure that you are doing everything in correct way ,then use a tool like Chrome dev tools to see what happens after making this post requests (using the browser) ,see if the browser is decoding/encoding/sending cookies/etc. Edit:You are getting Blank response ,as i think :it is not encoded,the server is sending you this because you are doing something wrong in your post request(according to something i faced before!)

来源：https://stackoverflow.com/questions/57239651/scrap-peekyou-com-having-post-method

标签

ajax

python-3.x

python-requests