Working with curses in IPython. How can I improve this?

Question

I've found a way to interactively work with curses while still enjoying most of the benefits of IPython. It works, with some limitations, but not as well as I'd like.

The original problem, of course, is that I'd like to be able to work with my interactive Python session while having it control a terminal screen, using the curses (ncurses) module (or urwid, for example). One solution is to write a simple TCP server with a simple event loop that evaluates each string it reads from the socket and send back serialized strings representing and return results. As described here: SO: Is there a way to interactively program a Python curses Application).

Here's a somewhat simpler trick (assuming you have IPython installed).

    #!/usr/bin/python 
    #!/usr/bin/env python 
    from IPython import embed_kernel
    import curses

    def interact_with_curses(screen):
        '''set global stdscr variable and run embedded IPython kernel
           suitable to be called by curses.wrapper()
        '''
        global stdscr
        stdscr = screen
        embed_kernel()

    if __name__ == '__main__':
        curses.wrapper(interact_with_curses)

(slightly munged to get SO's syntax highlighting happy).

Running this will result in output roughly like:

 [IPKernelApp] To connect another client to this kernel, use:
            [IPKernelApp] --existing kernel-2869.json

And switching to another window or screen session you can run:

ipython console --existing kernel-2869.json

To connect to that process and work with it.

This is nice enough. You can then call things like stdscr.refresh(). Work with your curses/window and pad objects, call dir() on them to explore their functionality and generally work with the code as if you were in a normal IPython session which just happens to be updating a different terminal's screen and reading from it as well (through the curses input functions).

Problems with this approach, and questions:

• To exit it seems that I have to run quit() from the IPython console, and this doesn't exit the interpreter in the normal means. It doesn't seem to allow curses.wrapper() to reset the terminal and various attempts to called .endwin(), .resetty() (after having performed a .savetty() of course), .reset_shell_mode() (and .reset_prog_mode()) and so on have all failed. I've tried calling them in main after the call to curses.wrapper() and I've tried registering them with atexit
  • How do I cleanly quit from such a session?
• [Tab] completion doesn't work
  • How do I get IPython's [Tab] completely working through an IPython console session to one of these embeded kernels?
• Calling the IPython embed_kernel() function prints the socket information to the curses screen, which is already initialized by the curses.wrapper() by that time. This is ugly; also if want to do more interesting work, in curses and before calling the embed_kernel() function then I can't see the text which was printed to stdout or stderr by that function.
  • How do I make embed_kernel() silent and force it to register the connection details through some other mechanism? Can I give it my own socket name/path to use?

I'm sure I'll think of other questions, but I hope others will find this trick useful and will discover some other tricks I can use when I want to dabble with Python curses coding.

Answer 1

It turns out that we can now just use IPython in a fairly natural manner for interactively working with curses.

From one terminal simply type:

ipython kernel

This will print a line something like:

[IPKernelApp] To connect another client to this kernel, use:
[IPKernelApp] --existing kernel-14321.json

From another terminal/window type:

ipython console --existing kernel-14321.json

... and you'll be in a seemingly perfectly normal IPython session. The only difference will be that you're actually accessing the "remote" IPython kernel session in the other window. From there you'll be able to use curses functions, see changes in the other window, type inputs thereto, use [Tab]-completion, and so on.

Note that [Ctrl]-[D] will offer to exit the IPython console (client) while quit() will close the IPython kernel (remote window --- server).

But, overall this model is cleaner and easier then what I described in my question last year. I don't know if it's the newer version of IPython (0.13.1) or if was simple ignorance that made my previous attempts somewhat clunkier.

Answer 2

I have found an answer to the tab completion, you can acquire the IPython shell object using get_ipython(), and then call init_completer() on it:

get_ipython().init_completer()

It should be possible to acquire the kernel and shut it down, to quit the terminal application., too but crawling around the documentation has so far not uncovered how.