python-requests

Python requests 401 error but url opens in browser

拥有回忆 提交于 2021-01-04 03:13:08
问题 I am trying to pull the json from this location - https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY This opens fine in my browser, but using requests in python throws a 401 permission error. I have tried adding headers with different arguments, but to no avail. Interestingly, the json on this page does not open in the browser as well until https://www.nseindia.com is opened separately. I believe it requires some kind of authentication, but surprised it works in the browser

Python Requests Stream Data from API

你离开我真会死。 提交于 2021-01-03 06:48:33
问题 Use Case: I am trying to connect to a streaming API, ingest those events, filter them and save relevant ones. Issue: My code works well until about 1100th response. After this point the code doesn't crash but it seems to stop pulling more data from the stream. I am guessing it is some sort of buffer issue, but honestly streaming is new to me and I have no idea what is causing the issue. Code import requests def stream(): s = requests.Session() r = s.get(url, headers=headers, stream=True) for

Python Requests Stream Data from API

非 Y 不嫁゛ 提交于 2021-01-03 06:48:05
问题 Use Case: I am trying to connect to a streaming API, ingest those events, filter them and save relevant ones. Issue: My code works well until about 1100th response. After this point the code doesn't crash but it seems to stop pulling more data from the stream. I am guessing it is some sort of buffer issue, but honestly streaming is new to me and I have no idea what is causing the issue. Code import requests def stream(): s = requests.Session() r = s.get(url, headers=headers, stream=True) for

How to use .p12 certificate to authenticate rest api

故事扮演 提交于 2021-01-03 06:41:33
问题 I have received a certificate.p12 with username and password. While I am able to use Rest Client for post requests after i install this certificate in my system. How can i use this certificate to authenticate post requests on Rest API using Python requests method ? I am using below code but it is not working. import requests headers = {'Content-Type': 'application/json'} payload = {'folder': '/Trial/trial_dir'} response = requests.post('https://<IP>:8080/siteapi/availabletests', params

Python Requests - Use TLS v.1_3

懵懂的女人 提交于 2021-01-02 20:29:27
问题 I am trying to make a request in Python using TLS 1.3 as it is what the site accepts. I use following code: import requests from requests.adapters import HTTPAdapter from requests.packages.urllib3.poolmanager import PoolManager from requests.packages.urllib3.util import ssl_ CIPHERS = ( 'AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:AES256-SHA' ) class TlsAdapter(HTTPAdapter): def __init__(self, ssl_options=0, **kwargs): self.ssl_options = ssl_options super(TlsAdapter, self).__init__(**kwargs)

Python Requests - Use TLS v.1_3

跟風遠走 提交于 2021-01-02 20:29:13
问题 I am trying to make a request in Python using TLS 1.3 as it is what the site accepts. I use following code: import requests from requests.adapters import HTTPAdapter from requests.packages.urllib3.poolmanager import PoolManager from requests.packages.urllib3.util import ssl_ CIPHERS = ( 'AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:AES256-SHA' ) class TlsAdapter(HTTPAdapter): def __init__(self, ssl_options=0, **kwargs): self.ssl_options = ssl_options super(TlsAdapter, self).__init__(**kwargs)

Streaming download large file with python-requests interrupting

眉间皱痕 提交于 2021-01-02 06:14:21
问题 i have problem with streaming download large file (about 1.5 GB) in python-requests v. 2.0.1 with open("saved.rar",'wb') as file: r = session.get(url,stream=True,timeout=3600) for chunk in r.iter_content(chunk_size=1024): if chunk: file.write(chunk) file.flush() I tested it few times on my vps and sometimes it downloaded 200mb, 500mb or 800mb and saved it without any error. It doesnt reached the timeout, just stopped like finish downloading. Host where im downloading this file is stable

How do connections recycle in a multiprocess pool serving requests from a single requests.Session object in python?

Deadly 提交于 2021-01-01 04:17:25
问题 Below is the complete code simplified for the question. ids_to_check returns a list of ids. For my testing, I used a list of 13 random strings. #!/usr/bin/env python3 import time from multiprocessing.dummy import Pool as ThreadPool, current_process as threadpool_process import requests def ids_to_check(): some_calls() return(id_list) def execute_task(id): url = f"https://myserver.com/todos/{ id }" json_op = s.get(url,verify=False).json() value = json_op['id'] print(str(value) + '-' + str

Missing Host header in HTTP requests from the requests Python library

被刻印的时光 ゝ 提交于 2020-12-31 15:26:52
问题 Where is the HTTP/1.1 mandatory Host header field in HTTP request messages generated by the requests Python library? import requests response = requests.get("https://www.google.com/") print(response.request.headers) Outputs this: {'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'} 回答1: The HOST header is not being added to the request by requests by default. If it is not explicitly added then the decision is delegated to

Missing Host header in HTTP requests from the requests Python library

你离开我真会死。 提交于 2020-12-31 15:23:51
问题 Where is the HTTP/1.1 mandatory Host header field in HTTP request messages generated by the requests Python library? import requests response = requests.get("https://www.google.com/") print(response.request.headers) Outputs this: {'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'} 回答1: The HOST header is not being added to the request by requests by default. If it is not explicitly added then the decision is delegated to