Reading serial data in realtime in Python

Question

I am using a script in Python to collect data from a PIC microcontroller via serial port at 2Mbps.

The PIC works with perfect timing at 2Mbps, also the FTDI usb-seri

Answer 1

From the manual:

Possible values for the parameter timeout: … x set timeout to x seconds

and

readlines(sizehint=None, eol='\n') Read a list of lines, until timeout. sizehint is ignored and only present for API compatibility with built-in File objects.

Note that this function only returns on a timeout.

So your readlines will return at most every 2 seconds. Use read() as Tim suggested.

Answer 2

A very good solution to this can be found here:

Here's a class that serves as a wrapper to a pyserial object. It allows you to read lines without 100% CPU. It does not contain any timeout logic. If a timeout occurs, self.s.read(i) returns an empty string and you might want to throw an exception to indicate the timeout.

It is also supposed to be fast according to the author:

The code below gives me 790 kB/sec while replacing the code with pyserial's readline method gives me just 170kB/sec.

class ReadLine:
    def __init__(self, s):
        self.buf = bytearray()
        self.s = s

    def readline(self):
        i = self.buf.find(b"\n")
        if i >= 0:
            r = self.buf[:i+1]
            self.buf = self.buf[i+1:]
            return r
        while True:
            i = max(1, min(2048, self.s.in_waiting))
            data = self.s.read(i)
            i = data.find(b"\n")
            if i >= 0:
                r = self.buf + data[:i+1]
                self.buf[0:] = data[i+1:]
                return r
            else:
                self.buf.extend(data)

ser = serial.Serial('COM7', 9600)
rl = ReadLine(ser)

while True:

    print(rl.readline())

Answer 3

You need to set the timeout to "None" when you open the serial port:

ser = serial.Serial(**bco_port**, timeout=None, baudrate=115000, xonxoff=False, rtscts=False, dsrdtr=False) 

This is a blocking command, so you are waiting until you receive data that has newline (\n or \r\n) at the end: line = ser.readline()

Once you have the data, it will return ASAP.

Answer 4

You can use inWaiting() to get the amount of bytes available at the input queue.

Then you can use read() to read the bytes, something like that:

While True:
    bytesToRead = ser.inWaiting()
    ser.read(bytesToRead)

Why not to use readline() at this case from Docs:

Read a line which is terminated with end-of-line (eol) character (\n by default) or until timeout.

You are waiting for the timeout at each reading since it waits for eol. the serial input Q remains the same it just a lot of time to get to the "end" of the buffer, To understand it better: you are writing to the input Q like a race car, and reading like an old car :)