Using the browser's XHR log to recreate an AJAX request

问题

I want to create a post request fully based on the XHR data that appear in the Chrome inspector(network tab). The goal is to recreate an AJAX request to go to a dynamically shown page 4.

I am programming it as such :

from requests import Session
session = requests.Session()

session.head('http://www.metrocuadrado.com/web/buscarFiltros/bogota-apartamento-venta')

payload = {...}   #copied and pasted literally from the (previously inspected) source code of the XHR request
headersxhr = {...}   #dictionary of all the headers found in the (previously inspected) source code of the XHR

response = session.post(
    url = 'http://www.metrocuadrado.com/web/busqueda/pagina-4',
    data = payload,    
    headers = headersxhr
    )

print response.text

Unfortunately this gives me a 404 error. The pasted payload is very long, with a lot of nested dictionaries. It starts like this :

{"token":"","cantidadResultadosPagina":"16","filtrosJson":"\t\t\n\t\t{\"mnombreinmobiliaria\": {\"nombre\":\"mnombreinmobiliaria\",\"valor\":[\"\"],\"valor2\":null,\"descripcion\":\"Nombre Compañia\",\"tip #.....and so on

Could there be an encoding problem I should be aware of ?
Also, do I have to pass through all the headers ?

Many thanks !!

回答1:

1) filtrosJson is randomly generated code. Which is found in the source code of first page.

We first need to grab that using any method that you like, for now I am using bs4

2) The payload is in json so we ll have to use json.dumps to send the post request. And finally send the post requests indicating the Content-Type as application/json

We can do something like this,

import requests, json
from bs4 import BeautifulSoup as bs

s=requests.Session()
headers={"User-Agent":"Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36"}
s.headers.update(headers)

r=s.get("http://www.metrocuadrado.com/web/buscarFiltros/bogota-apartamento-venta")

soup=bs(r.content)
filtrosJson=soup.find(id="filtrosJson").text
data={"cantidadResultadosPagina": "16","filtroOrdenamiento": "-1","filtrosJson":filtrosJson,"token": ""}


r=s.post("http://www.metrocuadrado.com/web/busqueda/pagina-4",data=json.dumps(data),headers={"X-Requested-With":"XMLHttpRequest","Content-Type":"application/json"})

print r.json()

This works for me on Python2.7, Ubuntu 14.04

Let me know if you face any issues :-)

来源：https://stackoverflow.com/questions/26665388/using-the-browsers-xhr-log-to-recreate-an-ajax-request

标签

ajax

xmlhttprequest

python-requests