How make a twisted python client with readline functionality

Question

I'm trying to write a client for simple TCP server using Python Twisted. Of course I pretty new to Python and just started looking at Twisted so I could be doing it all wrong.

The server is simple and you're intended to use use nc or telnet. There is no authentication. You just connect and get a simple console. I'd like to write a client that adds some readline functionality (history and emacs like ctrl-a/ctrl-e are what I'm after)

Below is code I've written that works just as good as using netcat from the command line like this nc localhost 4118

from twisted.internet import reactor, protocol, stdio
from twisted.protocols import basic
from sys import stdout

host='localhost'
port=4118
console_delimiter='\n'

class MyConsoleClient(protocol.Protocol):
    def dataReceived(self, data):
        stdout.write(data)
        stdout.flush()

    def sendData(self,data):
        self.transport.write(data+console_delimiter)

class MyConsoleClientFactory(protocol.ClientFactory):
    def startedConnecting(self,connector):
        print 'Starting connection to console.'

    def buildProtocol(self, addr):
        print 'Connected to console!'
        self.client = MyConsoleClient()
        self.client.name = 'console'
        return self.client

    def clientConnectionFailed(self, connector, reason):
        print 'Connection failed with reason:', reason

class Console(basic.LineReceiver):
    factory = None
    delimiter = console_delimiter

    def __init__(self,factory):
        self.factory = factory

    def lineReceived(self,line):
        if line == 'quit':
            self.quit()
        else:
            self.factory.client.sendData(line)

    def quit(self):
        reactor.stop()

def main():
    factory = MyConsoleClientFactory()
    stdio.StandardIO(Console(factory))
    reactor.connectTCP(host,port,factory)
    reactor.run()

if __name__ == '__main__':
    main()

The output:

$ python ./console-console-client.py 
Starting connection to console.
Connected to console!
console> version
d305dfcd8fc23dc6674a1d18567a3b4e8383d70e
console> number-events
338
console> quit

I've looked at

Python Twisted integration with Cmd module

This really didn't work out for me. The example code works great but when I introduced networking I seemed to have race conditions with stdio. This older link seems to advocate a similar approach (running readline in a seperate thread) but I didn't get far with it.

I've also looked into twisted conch insults but I haven't had any luck getting anything to work other than the demo examples.

What's the best way to make a terminal based client that would provide readline support?

http://twistedmatrix.com/documents/current/api/twisted.conch.stdio.html

looks promising but I'm confused how to use it.

http://twistedmatrix.com/documents/current/api/twisted.conch.recvline.HistoricRecvLine.html

also seems to provide support for handling up and down arrow for instance but I couldn't get switching Console to inherit from HistoricRecVLine instead of LineReceiver to function.

Maybe twisted is the wrong framework to be using or I should be using all conch classes. I just liked the event driven style of it. Is there a better/easier approach to having readline or readline like support in a twisted client?

Answer 1

I landed up solving this by not using the Twisted framework. It's a great framework but I think it was the wrong tool for the job. Instead I used the telnetlib, cmd and readline modules.

My server is asynchronous but that didn't mean my client needed to be so I used telnetlib for my communication to the server. This made it easy to create a ConsoleClient class which subclasses cmd.Cmd and get history and emacs-like shortcuts.

#! /usr/bin/env python

import telnetlib
import readline
import os
import sys
import atexit
import cmd
import string

HOST='127.0.0.1'
PORT='4118'

CONSOLE_PROMPT='console> '

class ConsoleClient(cmd.Cmd):
    """Simple Console Client in Python.  This allows for readline functionality."""

    def connect_to_console(self):
        """Can throw an IOError if telnet connection fails."""
        self.console = telnetlib.Telnet(HOST,PORT)
        sys.stdout.write(self.read_from_console())
        sys.stdout.flush()

    def read_from_console(self):
        """Read from console until prompt is found (no more data to read)
        Will throw EOFError if the console is closed.
        """
        read_data = self.console.read_until(CONSOLE_PROMPT)
        return self.strip_console_prompt(read_data)

    def strip_console_prompt(self,data_received):
        """Strip out the console prompt if present"""
        if data_received.startswith(CONSOLE_PROMPT):
            return data_received.partition(CONSOLE_PROMPT)[2]
        else:
            #The banner case when you first connect
            if data_received.endswith(CONSOLE_PROMPT):
                return data_received.partition(CONSOLE_PROMPT)[0]
            else:
                return data_received

    def run_console_command(self,line):
        self.write_to_console(line + '\n')
        data_recved = self.read_from_console()        
        sys.stdout.write(self.strip_console_prompt(data_recved))        
        sys.stdout.flush()

    def write_to_console(self,line):
        """Write data to the console"""
        self.console.write(line)
        sys.stdout.flush()

    def do_EOF(self, line): 
        try:
            self.console.write("quit\n")
            self.console.close()
        except IOError:
            pass
        return True

    def do_help(self,line):
        """The server already has it's own help command.  Use that"""
        self.run_console_command("help\n")

    def do_quit(self, line):        
        return self.do_EOF(line)

    def default(self, line):
        """Allow a command to be sent to the console."""
        self.run_console_command(line)

    def emptyline(self):
        """Don't send anything to console on empty line."""
        pass


def main():
    histfile = os.path.join(os.environ['HOME'], '.consolehistory') 
    try:
        readline.read_history_file(histfile) 
    except IOError:
        pass
    atexit.register(readline.write_history_file, histfile) 

    try:
        console_client = ConsoleClient()
        console_client.prompt = CONSOLE_PROMPT
        console_client.connect_to_console()
        doQuit = False;
        while doQuit != True:
            try:
                console_client.cmdloop()
                doQuit = True;
            except KeyboardInterrupt:
                #Allow for ^C (Ctrl-c)
                sys.stdout.write('\n')
    except IOError as e:
        print "I/O error({0}): {1}".format(e.errno, e.strerror)
    except EOFError:
        pass

if __name__ == '__main__':
    main()

One change I did was remove the prompt returned from the server and use Cmd.prompt to display to the user. I reason was to support Ctrl-c acting more like a shell.