问题
I'm writing an application which reads text data and acts on it. The text data could come from a TCP port, or from a text file (which contains data earlier read from the TCP port and archived). I'm writing it in Python 3, and using asyncio seems like the obvious tool to use.
It's straightforward to use the Streams API open_connection() to open the TCP port and read from it. The asyncio architecture has the concept a Transport and a Protocol for the lower and upper layers of input-output. So, it seems I should implement a Transport for reading text from a file, and pass it to the Protocol. That will let me keep the rest of my application decoupled from whether the text data came from the TCP port or the file.
But I'm having a hard time figuring out how to tell asyncio to use my preferred Transport.
- the Streams API open_connection() has a parameter list which is all about the TCP port Transport, with no way to specify a different Transport, much less parameters like file path.
open_connection()
turns around and calls loop.create_connection(). This is just as specialised for the TCP Port Transport. Still now way to provide a different Transport.- The implementation of
loop.create_connection()
gets its Transport object from eitherself._make_ssl_transport()
orself._make_socket_transport()
. These have alternative implementations inasyncio.selector_events.BaseSelectorEventLoop
andasyncio.proactor_events.BaseProactorEventLoop
, so we are clearly past the point where a File Transport ought to have been selected.
Am I missing some place where asyncio lets me tell it what Transport to use? Or is asyncio really coded down to its roots to use its own TCP port and UDP datagram Transports, and nothing else?
If I want to allow the possibility of using my own Transport with asyncio, it looks like I have to extend the event loop, or write more a flexible alternative create_connection()
that is coded to a particular event loop implementation. That seems like a lot of work, and vulnerable to changes in the implementation.
Or, is it a foolish errand to handle file input with a Transport? Should I instead structure my code to say:
if (using_tcp_port):
await asyncio.open_connection(....)
else:
completely_different_file_implementation(....)
回答1:
According to the documentation of API create_connection()
, it takes a protocol and creates a streaming transport, which is a TCP connection. So it is not supposed to be an API for custom transports.
However, the idea to reuse the same protocol for either TCP transports or custom file transports is valid. It won't be a "completely different implementation", but at least not using create_connection()
. Let's assume it is read_file()
:
def my_protocol_factory():
return your_protocol
if using_tcp_port:
transport, protocol = await loop.create_connection(my_protocol_factory, host, port)
else:
transport, protocol = await read_file(loop, my_protocol_factory, path_to_file)
Then you would have something like this:
from asyncio import transports
import aiofiles # https://github.com/Tinche/aiofiles
def read_file(loop, protocol_factory, path):
protocol = protocol_factory()
transport = FileTransport(path, loop)
transport.set_protocol(protocol)
return transport, protocol
class FileTransport(transports.ReadTransport):
def __init__(self, path, loop):
super().__init__()
self._path = path
self._loop = loop
self._closing = False
def is_closing(self):
return self._closing
def close(self):
self._closing = True
def set_protocol(self, protocol):
self._protocol = protocol
self._loop.create_task(self._do_read())
def get_protocol(self):
return self._protocol
async def _do_read(self):
try:
async with aiofiles.open(self._path) as f:
self._loop.call_soon(self._protocol.connection_made, self)
async for line in f:
self._loop.call_soon(self._protocol.data_received, line)
if self._closing:
break
self._loop.call_soon(self._protocol.eof_received)
except Exception as ex:
self._loop.call_soon(self._protocol.connection_lost, ex)
else:
self._loop.call_soon(self._protocol.connection_lost, None)
来源:https://stackoverflow.com/questions/51110406/how-to-add-a-filetransport-cleanly-to-asyncio