aiohttp

How can i use SOCKS4 and SOCKS5 proxies with aiohttp ClientSession?

懵懂的女人 提交于 2019-12-11 16:12:28
问题 How can i use SOCKS4 and SOCKS5 proxies with aiohttp ClientSession? Need some small example. Thanks! 回答1: Have you tried aiosocks ? import asyncio import aiosocks from aiosocks.connector import SocksConnector conn = SocksConnector(proxy=aiosocks.Socks5Addr(PROXY_ADDRESS, PROXY_PORT), proxy_auth=None, remote_resolve=True) session = aiohttp.ClientSession(connector=conn) async with session.get('http://python.org') as resp: assert resp.status == 200 来源: https://stackoverflow.com/questions

My script encounters an error when it is supposed to run asynchronously

偶尔善良 提交于 2019-12-11 07:53:32
问题 I've written a script in python using asyncio association with aiohttp library to parse the names out of pop up boxes initiated upon clicking on contact info buttons out of diffetent agency information located within a table from this website asynchronously. The webpage displayes the tabular contents across 513 pages. I encountered this error too many file descriptors in select() when I tried with asyncio.get_event_loop() but when I came across this thread I could see that there is a

Script performs very slowly even when it runs asynchronously

假装没事ソ 提交于 2019-12-11 05:31:23
问题 I've written a script in asyncio in association with aiohttp library to parse the content of a website asynchronously. I've tried to apply the logic within the following script the way it is usually applied in scrapy . However, when I execute my script, it acts like how syncronous libraries like requests or urllib.request do. Therefore, it is very slow and doesn't serve the purpose. I know I can get around this by defining all the next page link within the link variable. But, am I not doing

asyncio: why isn't it non-blocking by default

自闭症网瘾萝莉.ら 提交于 2019-12-11 04:38:15
问题 By default, asyncio runs coroutines synchronously. If they contain blocking IO code, they still wait for it to return. A way around this is loop.run_in_executor(), which converts the code into threads. If a thread blocks on IO, another thread can start executing. So you don't waste time waiting for IO calls. If you use asyncio without executors, you loose those speedups. So I was wondering, why do you have to use executors explicitly. Why not enable them by default ? (In the following, I'll

How to connect to .onion sites using python aiohttp?

泄露秘密 提交于 2019-12-10 23:38:15
问题 I am trying to connect to a .onion site using python. I have tor running on port 9050 and I am getting the following error: Traceback (most recent call last): File "/Users/jane/code/test/test.py", line 15, in main res = await fetch(session, id) File "/Users/jane/code/test/test.py", line 9, in fetch async with session.get(url) as res: File "/usr/local/lib/python3.7/site-packages/aiohttp/client.py", line 1005, in __aenter__ self._resp = await self._coro File "/usr/local/lib/python3.7/site

aiohttp how to log access log?

自闭症网瘾萝莉.ら 提交于 2019-12-10 22:37:19
问题 I am trying to get a basic logger for aiohttp working, but there are simply no log messages being logged. Note_ logging custom messages works as expected. async def main_page(request: web.Request): return "hello world" def setup_routes(app): app.router.add_get('/', main_page) async def init(loop): # load config from yaml file in current dir conf = load_config(str(pathlib.Path('.') / 'async_config.yml')) # setup application and extensions app = web.Application(loop=loop) # setup views and

How to run an aiohttp server in a thread?

|▌冷眼眸甩不掉的悲伤 提交于 2019-12-10 15:52:36
问题 This example of aiohttp server in a thread fails with an RuntimeError: There is no current event loop in thread 'Thread-1'. error: import threading from aiohttp import web def aiohttp_server(): def say_hello(request): return web.Response(text='Hello, world') app = web.Application(debug=True) app.add_routes([web.get('/', say_hello)]) web.run_app(app) t = threading.Thread(target=aiohttp_server) t.start() How to run a aiohttp server in thread? 回答1: Create handler in main thread and manually

Shared state with aiohttp web server

匆匆过客 提交于 2019-12-10 09:22:45
问题 My aiohttp webserver uses a global variable that changes over time: from aiohttp import web shared_item = 'bla' async def handle(request): if items['test'] == 'val': shared_item = 'doeda' print(shared_item) app = web.Application() app.router.add_get('/', handle) web.run_app(app, host='somewhere.com', port=8181) Results in: UnboundLocalError: local variable 'shared_item' referenced before assignment How would I properly use a shared variable shared_item ? 回答1: Push your shared variable into

Does aiohttp have ORM?

给你一囗甜甜゛ 提交于 2019-12-10 03:01:04
问题 There is relatively new Python 3 aiohttp library that contains client/server. Does it contain ORM? If not - is it possible to use it with 3-rd party ORM? If not possible - for which purpose it could be used? I don't mean that app could not be written without ORM, but major waste of Python Frameworks support it and developers are used to such style of programming. 回答1: Short answer -- aiohttp has no ORM yet. You can use SQLAlchemy-like queries for aiopg driver, see example The same is

Integrate Autobahn|Python with aiohttp

半世苍凉 提交于 2019-12-08 02:49:37
问题 I'm trying to integrate an aiohttp web server into a Crossbar+Autobahn system architecture. More in detail, when the aiohttp server receive a certain API call, it has to publish a message to a Crossbar router. I've seen this example on the official repos but i've no clue on how to integrate it on my application. Ideally, i would like to be able to do this # class SampleTaskController(object): async def handle_get_request(self, request: web.Request) -> web.Response: self.publisher.publish('com