python-requests

I cannot autologin to pastebin using requests + BeautifulSoup

与世无争的帅哥 提交于 2021-01-29 20:53:39
问题 I am trying to auto-login to pastebin account using python, but im failing and i don't know why. I copied the request headers exactly and double checked... but still i am greeted with 400 HTTP code. Can somebody help me? This is my code: import requests from bs4 import BeautifulSoup import subprocess import os import sys from requests import Session # the actual program page = requests.get("https://pastebin.com/99qQTecB") parse = BeautifulSoup(page.content, 'html.parser') string = parse.find(

How to properly use proxies in Python requests?

纵饮孤独 提交于 2021-01-29 20:10:47
问题 I have a list of proxies. When I test one of them using curl : curl -proxy https://x.x.x:port https://www.google.com/ I get the expected result, but when I run: proxies = {'http' : 'x.x.x:port', 'https' : 'x.x.x:port'} requests.get('https://www.google.com/', proxies = proxies) It is stuck for a while and then I get this error: requests.exceptions.ProxyError: HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.',

Running out of memory on python product iteration chain

妖精的绣舞 提交于 2021-01-29 19:52:48
问题 I am trying to build a list of possible string combinations to then iterate against it. I am running out of memory executing the below line, which I get because it's several billion lines. data = list(map(''.join,chain.from_iterable(product(string.digits+string.ascii_lowercase+'/',repeat = i) for i in range(0,7)))) So I think, rather than creating this massive iterable list, I create it and execute against it in waves with some kind of "holding string" that I save to memory and can restart

Flask restful output XML only

本小妞迷上赌 提交于 2021-01-29 18:07:50
问题 I am new to experimenting with creating API's and I was interested to make an app that returns data in an XML format only on the API post request. The flask-restful code below is just a glorified version of how to do an XML response from the git repo examples to view. from simplexml import dumps from flask import make_response, Flask from flask_restful import Api, Resource def output_xml(data, code, headers=None): """Makes a Flask response with a XML encoded body""" resp = make_response(dumps

Scrap peekyou.com ( having POST METHOD)

强颜欢笑 提交于 2021-01-29 17:31:25
问题 Please see the output which I am getting I am trying to scrap peekyou.com which is kind of peoples search engine. They use POST method of php.I am using requests.post method of requests library to scrap the results . suppose a persons name is "john coasta" then the target url would be : peekyou.com/john_coasta import requests import json payload = { 'formdata' : {'md5': '4a9050a569e0f7d862b771926f7abc57', 'asynchronous': 'true'} } req = requests.post('https://www.peekyou.com/shantanu_sharma',

Python Requests Library returning 429 on 1 request

空扰寡人 提交于 2021-01-29 17:19:01
问题 I'm trying to access a web page with python requests library. The url is here: https://acs.leagueoflegends.com/v1/stats/game/WMC2TMNT1/210029?gameHash=50ee0d42d77ae4d9 The python is fairly simple, ''' requests.get(url) When I'm not logged in to riot games, this returns the following status code: {"httpStatus":429,"errorCode":"CLIENT_RATE_LIMITED"} So I tried creating a session via the following code: login_url ='https://auth.riotgames.com/login#client_id=rso-web-client-prod&login_hint=na

Why is this Requests and BeautifulSoup login script not working?

喜夏-厌秋 提交于 2021-01-29 15:52:00
问题 The following is a bit of code i produced on the back of the Requests and BeautifulSoup libraries for Python 3. import requests as rq from bs4 import BeautifulSoup as bs def get_data(): return {'email': str(input('Enter your email.')), 'password': str(input('Enter your password.'))} def obtain_data(): login_data=get_data() form_data={'csrf_token': login_data['email'], 'login': '1', 'redirect': 'account/dashboard', 'query': None, 'required': 'email,password', 'email': login_data['email'],

Why is my login not working with Python Requests?

笑着哭i 提交于 2021-01-29 14:07:47
问题 I've been trying to login for the past couple days using Python Requests and have been having issues. Here is my code: import requests LOGINURL = "https://admin.neopets.knetik.com/admin/login" PROTECTEDPAGE = "https://admin.neopets.knetik.com/admin/order_summary/list" payload = { "_target_path": "https://admin.neopets.knetik.com/admin/dashboard", "_username": "***", "_password": "***", "_remember_me": "on", } with requests.session() as session: post = session.post(LOGINURL, data=payload)

Setting mailgun variables in template using Python request

瘦欲@ 提交于 2021-01-29 13:38:33
问题 I want to set the value of a variable using a header attribute (as documented here). My Python code for sending: import requests res = requests.post( "https://api.eu.mailgun.net/v3/mg.domain.de/messages", auth=("api", "xxxxxxxxxxxxxxxxxxxxxxxx-xxxxxxxxx-xxxxxxx"), headers = { "X-Mailgun-Variables": {"{'href_target': 'test'}"} }, data={ "from": "noreply@domain.de", "to": ["target@email.com"], "subject": "Testsubject", "template": "confirm-email-de", }) I'm using the {{href_target}} in my

Scrapping through pages of aspx website

好久不见. 提交于 2021-01-29 10:30:34
问题 for the last month or so, I've been trying to read a few pages from an aspx site. I have no problems finding all the required items on the site but my attempted solution is still not working properly. I read somewhere that all the header details must be present, so I added them. I also read somewhere that the __EVENTTARGET must be set to something to tell aspx which button had been pressed, so I tried a few different things(see below). I also read that a session should be established to deal