python-requests

问题 I am trying to login to a URL & download the content then parse, the URL needs username & password to login. using below gives below errors: import requests url = 'https://test/acx/databaseUsage.jssp?object=all' values = {'username': 'test_user', 'password': 'test_pswd'} headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'} r = requests.post(url, data=values, headers=headers) print r.content Error

问题 I am able to scrap all the data from flipkart website except the images using the code below: jobs = soup.find_all('div',{"class":"IIdQZO _1R0K0g _1SSAGr"}) for job in jobs: product_name = job.find('a',{'class':'_2mylT6'}) product_name = product_name.text if product_name else "N/A" product_offer_price = job.find('div',{'class':'_1vC4OE'}) product_offer_price = product_offer_price.text if product_offer_price else "N/A" product_mrp = job.find('div',{'class':'_3auQ3N'}) product_mrp = product_mrp

问题 I'm new to web scraping and I'm trying to scrape data from this auction website. However, I meet this weird problem when trying to get the text of the anchor tag. Here's the HTML: <div class="mt50"> <div class="head_011"> <a id="item_event_title" href="https://www.storyltd.com/auction/auction.aspx?eid=4158">NO RESERVE AUCTION OF MODERN AND CONTEMPORARY ART (16-17 APRIL 2019)</a> </div> </div> Here's my code: auction_info = LTD_work_soup.find('a', id = 'item_event_title').text print(auction

问题 I'm trying to code a script on a sneakers website that supports adyen checkout. Basically, it encrypts my credit card data in this way and doesn't make me pay. payload_cc = { ... "dwfrm_billing_paymentMethod": "CREDIT_CARD", "dwfrm_billing_creditCardFields_cardType": "Mastercard", "dwfrm_billing_creditCardFields_adyenEncryptedData":"adyenjs_0_1_25$ibmnmdt8wdc5...", "dwfrm_billing_creditCardFields_cardNumber":"************last4digits" "dwfrm_billing_creditCardFields_expirationMonth": "07",

问题 Based on the code from here, I'm able to crawler url for each transation and save them into an excel file which can be downloaded here. Now I would like to go further and click the url link: For each url , I will need to open and save pdf format files: How could I do that in Python? Any help would be greatly appreciated. Code for references: import shutil from bs4 import BeautifulSoup import requests import os from urllib.parse import urlparse url = 'xxx' for page in range(6): r = requests

问题 Based on the code from here, I'm able to crawler url for each transation and save them into an excel file which can be downloaded here. Now I would like to go further and click the url link: For each url , I will need to open and save pdf format files: How could I do that in Python? Any help would be greatly appreciated. Code for references: import shutil from bs4 import BeautifulSoup import requests import os from urllib.parse import urlparse url = 'xxx' for page in range(6): r = requests

问题 I facing a problem. I found a solution I'm explaining below but I want some advice aout to right way to resolve it. Here is the problem: I have a class object called Item . This Item has a method, call make_request which makes a GET request on a server, and save the result. Now, I have implemented 3 Item object which calls make_request . The Item objects gonna call the method each 3 minutes, but these make_requests must be delayed by 1 minutes from the previous object's call. Example : 14:00

问题 I'm trying to login into a website using a post request like this: import requests cookies = { '_SID': 'c1i73k2mg3sj0ugi5ql16c3sp7', 'isCookieAllowed': 'true', } headers = { 'Host': 'service.premiumsim.de', 'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-US,en;q=0.5', 'Referer': 'https://service.premiumsim.de/', 'Content-Type': 'application/x-www

问题 I am attempting to get an auth token from The Trade Desk's (sandbox) api but I get back a 400 response stating: "Error reading Content-Type 'application/json' as JSON: Unexpected character encountered while parsing value: L. Path '', line 0, position 0." Whole response.json() : {u'ErrorDetails': [{u'Reasons': [u"Error reading Content-Type 'application/json' as JSON: Unexpected character encountered while parsing value: L. Path '', line 0, position 0."], u'Property': u'TokenRequest'}], u

问题 For a school project I need get the web addresses of 200 companies (based on a list). My script is working fine, but when I'm around the company 80, I get blocked by google. This is the message that I'm getting. > Our systems have detected unusual traffic from your computer network. > This page checks to see if it's really you sending the requests, and > not a robot. <a href="#" > onclick="document.getElementById('infoDiv').style.display='block' I tried two different ways to get my data: A