Editing programs “while they are running”? Why?

Question

I\'ve been getting more into Lisp and Lispy languages lately, and I\'m finding them quite powerful.

One thing I\'ve been reading all over the net is that a benefit o

Answer 1

You see real data. That is a big advantage. You then don't have to speculate.

Answer 2

It's mostly for development, where it's just a time saver.

But time savers are staggeringly important.

Once you're used to it going back to the old way feels like going from flying to swimming in tar.

Answer 3

Another good thing apart from modifying the program on the fly without having to restart everything (having done it for decades doesn't mean it is the best thing, right?), is that you get to inspect your program in its current state and being able to figure out what's going on.

Answer 4

I remember somebody from NASA described his experience. His team implemented the soft used in a spaceship back in the 70s. And they effectively modified their soft remotely on the fly when some bugs were found.

Or imagine you have a long process taking days to execute and at the end it cannot write results because of permissions or other small problem.

Yet another example. You are in the integration phase and you have to make a lot of small changes. And again a lot of them. I dream about such a possibility in Java because currently it takes me 30-40 min to rebuild and reinstall my application (to rebuild it again in 10 min).

Answer 5

There are some extremely cool use cases. One example is in GUI programming - I saw this while developing a GUI app in real time as it was running beside my Emacs: I added code for a new button and hit "C-c C-c" to compile that single function, and the button just appeared in the window! Didn't have to close and reopen the app. Then I began tweaking widgets and manipulating the layout, and the open window would instantly rearrange itself - buttons would move around, new text fields would just pop into being, etc. as soon as I executed each little change I'd made.

Another example is an excellent screencast about the Clojure OpenGL library "Penumbra" where the programmer creates a 3D tetris game in real time. He starts with an empty OpenGL window next to his emacs. He defines a cube object - C-M-x - and it's on the screen. Runs a command to rotate it, immediately it starts spinning. Runs a loop defining 5 more cubes in different locations, pop-pop-pop-pop-pop they appear. It's all immediately responsive, the full OpenGL toolkit right there to play with. Add a new surface texture to your cube and see it appear right away. It becomes a malleable 3d world - the code dynamically modifies the existing world instead of closing and reopening the 3d canvas with every change.

Penumbra Livecoding Screencast - download HD version for best experience.

There is also a great presentation/screencast about the audio library "Overtone" for Clojure. The library is a synthesizer toolkit where you have a set of synth functions to manipulate the soundwave. During the presentation, the developer writes a bit of code that starts a tone playing. He then spends ten seconds writing a loop that plays that sound 10 times but makes the frequency higher each time, and again C-M-x and you hear it, notes ascending higher. Over the space of 20 minutes in real time he gets a song going. It looks like a ton of fun.

Overtone Presentation Link

Other uses would be, for example: Web crawling/data mining - develop and refine algorithms for extracting information in real time, seeing the data returned at each step; Robotics programming - send commands to a robot while it's live; Facial/image recognition - with a library like OpenCV watch your changes instantly update what the library recognizes in an image/video as you're developing the code; Mathematics work (Clojure has "Incanter" for statistics); and any environment where you want to immediately see what effect your changes have had on the data you're working with.

So that's the most fun aspect of having a REPL in front of you. Things that weren't tangible, malleable, interactive, start to be. GUI design, 3D graphics, programmatic sound production, extracting and transforming data, these things normally have been done at arm's length. But with Clojure (and to some extent with other dynamic languages too) it's made to be really tangible and immediate; you see each change as soon as you write the code, and if something doesn't work or you don't get back the result you expected, you just change what you missed and re-execute it immediately.

Clojure is very aligned towards doing this. The wild thing is you can use Java libraries in real-time the same way - despite the fact that Java itself can't! So Overtone is using a Java synth library in realtime despite the fact you never could in Java, Penumbra is using the Java OpenGL bindings, etc. This is because Rich Hickey designed Clojure so it could compile to JVM bytecode on the fly. It's an amazing language - Clojure has made a huge contribution to how incredibly fun and productive programming can be.

Answer 6

In the real world this is mainly used in development and like many features its only worth drooling over in the right context.

1. personal programmer enlightenment bliss*
2. true continuous deployment.
3. zero planned downtime Service Level Agreements.
4. debug production servers.

*not a guarantee.

---

for me, and i suspect some others here the real benefit of this REPL driven development is that it can be indescribably fun. addictive even. It can sometimes really give a sense of crafting-code. give it a try... come on man try it out, first REPL's always free :)

---

one big draw these days is continual deployment.

currently the idea for continual deployment is that you change one thing, build everything (or package it rather) then deploy. with the lisp model its actually possible to edit a deployed (usually a box that is recieving a mirror of real customer sessions) box while it is in deployment.

just a pedantic note. you dont actually edit running classes. you compile a new copy of the class and leave it in a known location (a var) then the next time it is used the new copy is found and used. its not really editing while running and more like new code takes effect immediately this reduces the scope of the devlopment process from programs to expressions (typically functions).

---

another drooling point is the idea of getting the bennefit of security fixes with out having to declare any downtime. you can do an upgrade with out it costing your SLA any of you precious "scheduled downtime". If you have to schedule planned downtime six months in advance, and you only get two hours of it then (for these poor souls) it could really make them drool.

---

If you have repl access to your running application as it is deployed (potentially (with permission) at a customer site) you can connect to the app while it is running and run tests on the existing code in the existing context with out having to stop and connect a debugger. you also wont get any speed loss from a debugger. It's possible to do this with out a REPL, though when you get the repl in there you can then create new code easily (some will say that injecting dynamic class loaders through the debugger is easy) and then fix things. So you could connect to a running server. discover that a function failed to re-connect to a database after a short outage and then reconnect it right then and there.

---

as with all programming constructs there never will be a silver bullet and this continual deployment/development has an interesting downside: you program can be correct in memory and wrong on the disk. if you compile a function then break it and save then the only working copy of the code is the one running. I't useful to be aware of this and re-eval files just after you save them.

---

This may sound fanciful so go checkout how to Embed a Clojure REPL in your production application