Line buffered serial input

Question

I have a serial device that I\'m trying to read input from. I sent it a string \"ID\\r\", and it returns \"ID XX\\r\" (where \\r is an ASCII carriage return, hex 0x0d).

Answer 1

Caveat: I am using Python 3.4 on a Mac so your mileage may vary, though I believe with TextIOWrapper backported to Python 2.7, the situation in Python 2.7 (and other OSs) will essentially be the same as what I describe below.

The main issue is that io.TextIOWrapper itself uses a buffering mechanism, controlled by the undocumented _CHUNK_SIZE attribute. This is pretty unpleasant. So you have two choices:

1. Use a timeout as you tried out. This is what is hinted on in the documentation of readline on the pyserial documentation page. However, if you use a large value (as you did), when there is not enough data to fill the buffer of TextIOWrapper, your code will block until the timeout is reached. This is what you are experiencing essentially (I did not go after why you have to wait double the timeout value, but I think that this could be sorted out by looking at the implementation of TextIOWrapper and ultimately is irrelevant to your question).

2. The second choice is to change _CHUNK_SIZE to 1. In your case, simply add the line

   sio._CHUNK_SIZE = 1
   

   to your code right after you initialized sio. This has the perhaps unpleasant effect that the buffering within TextIOWrapper itself will be turned off (this is used for the incremental decoding of the input). If performance is not an issue, this is the simplest solution. If performance is an issue, you can set a low value of timeout, not toucing _CHUNK_SIZE. However, in this case be prepared to get an empty string from readline() (if the device sends you an empty line, that will come through as '\n', so it can be distinguished from the empty string that you will get when a read runs out of the alloted time).

There is another problem with your code: When sio will be removed, the close method of ser will be called twice, which will result in an exception when your program will be about to finish (at least this is what happens if I try your code on my computer). You should create (it seems) to instances of serial.Serial and pass those to BufferedRWPair.

I have also created a wrapper class based on TextIOWrapper, which I could also post, if there is interest, just I did not want to litter response with some extra code which, strictly speaking, is not needed.

PS: In the meanwhile, I have experimented with the code on Ubuntu. While on my Mac, I did not see a need for setting the buffer size of io.BufferedRWPair to 1, on Ubuntu I had to do this, too, in addition to setting _CHUNK_SIZE to 1.