PySerial non-blocking read loop

Question

I am reading serial data like this:

connected = False
port = 'COM4'
baud = 9600

ser = serial.Serial(port, baud, timeout=0)

while not connected:
    #serin = ser.read()
    connected = True

    while True:
        print("test")
        reading = ser.readline().decode()

The problem is that it prevents anything else from executing including bottle py web framework. Adding sleep() won't help.

Changing "while True"" to "while ser.readline():" doesn't print "test", which is strange since it worked in Python 2.7. Any ideas what could be wrong?

Ideally I should be able to read serial data only when it's available. Data is being sent every 1,000 ms.

Answer 1

Put it in a separate thread, for example:

import threading
import serial

connected = False
port = 'COM4'
baud = 9600

serial_port = serial.Serial(port, baud, timeout=0)

def handle_data(data):
    print(data)

def read_from_port(ser):
    while not connected:
        #serin = ser.read()
        connected = True

        while True:
           print("test")
           reading = ser.readline().decode()
           handle_data(reading)

thread = threading.Thread(target=read_from_port, args=(serial_port,))
thread.start()

http://docs.python.org/3/library/threading

Answer 2

Using a separate thread is totally unnecessary. Just do this for your infinite while loop instead (Tested in Python 3.2.3):

import serial
import time # Optional (if using time.sleep() below)

while (True):
    # NB: for PySerial v3.0 or later, use property `in_waiting` instead of function `inWaiting()` below!
    if (ser.inWaiting()>0): #if incoming bytes are waiting to be read from the serial input buffer
        data_str = ser.read(ser.inWaiting()).decode('ascii') #read the bytes and convert from binary array to ASCII
        print(data_str, end='') #print the incoming string without putting a new-line ('\n') automatically after every print()
    #Put the rest of your code you want here
    time.sleep(0.01) # Optional: sleep 10 ms (0.01 sec) once per loop to let other threads on your PC run during this time. 

This way you only read and print if something is there. You said, "Ideally I should be able to read serial data only when it's available." This is exactly what the code above does. If nothing is available to read, it skips on to the rest of your code in the while loop. Totally non-blocking.

(This answer originally posted & debugged here: Python 3 non-blocking read with pySerial (Cannot get pySerial's "in_waiting" property to work))

pySerial documentation: http://pyserial.readthedocs.io/en/latest/pyserial_api.html

UPDATE:

• 27 Dec. 2018: added comment about in_waiting vs inWaiting(). Thanks to @FurkanTürkal for pointing that out in the comments below. See documentation here: https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.Serial.in_waiting.
• 27 Oct. 2018: Add sleep to let other threads run.
  • Documentation: https://docs.python.org/3/library/time.html#time.sleep
  • Thanks to @RufusV2 for bringing this point up in the comments.

Note on multi-threading:

Even though reading serial data, as shown above, does not require using multiple threads, reading keyboard input in a non-blocking manner does. Therefore, to accomplish non-blocking keyboard input reading, I've written this answer: How to read keyboard-input?.

Answer 3

Use a timer driven event to test and read the serial port. Untested example:

import threading
class serialreading():
    def __init__(self):
        self.active = True
        self.test() 
    def test(self):
        n_in =comport.in_waiting()
        if n_in> 0:
            self.data = self.data + comport.read(size=n_in)
    if len(self.data) > 0: 
        print(self.data)
        self.data=""
    if self.active:
        threading.Timer(1, test).start()  # start new timer of 1 second
    def stop(self):
        self.active = False

Answer 4

I would warn against using blocking IO in a thread. Remember Python has a GIL and at one time only one thread can execute. Now please note that pyserial module is a wrapper over an OS implementation of accessing the serial port. That means it calls code external to the Python. If that code blocks, then the interpreter also get blocked and nothing will execute in the Python program, even the main thread.

This can even happen when using non-blocking IO or timeout based polling if the underlying device driver does not implement timeout well.

A more robust approach is to use multiprocessing module with a queue. Run serial read code in a separate process. This will make sure main and other threads don't block and the program can exit in clean way.