Use of # a.k.a. read-macro

Question

Reading book "Let Over Lambda" by Doug Hoyte, I found the following description of #. sign, a.k.a. read-macro:

A basic read macro that comes built in with COMMON LISP is the #. read-time eval macro. This read macro lets you embed objects into the forms you read that can't be serialised, but can be created with a bit of lisp code.

It's from Chapter 4, most part of the book can be found here: http://letoverlambda.com/index.cl/toc

This is example from the book that shows how the same expression may be read differently every time:

* '(football-game
     (game-started-at
       #.(get-internal-real-time))
     (coin-flip
       #.(if (zerop (random 2)) 'heads 'tails)))

(FOOTBALL-GAME
  (GAME-STARTED-AT 187)
  (COIN-FLIP HEADS))

* '(football-game
     (game-started-at
       #.(get-internal-real-time))
     (coin-flip
   #.(if (zerop (random 2)) 'heads 'tails)))

(FOOTBALL-GAME
  (GAME-STARTED-AT 309)
  (COIN-FLIP TAILS))

Next, author demonstrates some hardcore tricks, creating variations with # macro.

So, it turns out that #' is also some kind of reading macro, and it's usually used before symbols that represent names of functions. But is it necessary, and what's exactly his job there?

I can put symbol for higher-order functions with #' or without it:

CL-USER> (defun test nil t)
TEST
CL-USER> (funcall #'test)
T
CL-USER> (funcall 'test)
T

with the same success.

Answer 1

You can call global definitions of functions both ways:

CL-USER> (defun test nil t)
TEST
CL-USER> (funcall #'test)
T
CL-USER> (funcall 'test)
T

But see this:

CL-USER 10 > (defun foo () 42)
FOO

CL-USER 11 > (flet ((foo () 82))
               (print (foo))
               (print (funcall 'foo))
               (print (funcall #'foo)))

82   ; the local definition
42   ; the symbol references the global definition
82   ; (function foo) / #'foo  references the local definition

(funcall 'foo) looks up the function from the symbol.

(funcall #'foo) calls the function from the lexical environment. If there is none the global definition is used.

#'foo is a shorthand notation for (function foo).

CL-USER 19 > '#'foo
(FUNCTION FOO)

Answer 2

Unlike most languages Common Lisp doesn't really have a parser. It has a lexer known as the reader. The reader consumes single characters and looks them up in a table and calls the function found there[1]. The role played by a parser in other languages is, in Lisp, fulfilled by macros.

[1] http://www.lispworks.com/documentation/lw51/CLHS/Body/02_b.htm

For example semicolon's reader consumes the rest of the line and discards it as a comment. So, for example, the reader for open paren. calls a function that recursively reading the elements of a list. So, for example single-quote recursively reads a single form and then wraps it in quote. Thus '(1 2 3) is read as (quote (1 2 3)). There are a handfull of these key complex token readers.

[2] http://www.lispworks.com/documentation/lw51/CLHS/Body/02_d.htm

The character #\# provides a place for to put a slew of extra reader behaviors. The reader for hash repeats the design for the main reader. It consumes another character and look that up in a table and calls the function found there. There are a lots[3] of these.

[3] http://www.lispworks.com/documentation/lw51/CLHS/Body/02_dh.htm

So, for example, we have a reader analogous to the one for lists that reads vectors instead, e.g. #(1 2 3). So, for example, we have a reader for single characters such that you can enter a single semicolon, double quote, or period as #\;, #\", and #\. respectively.

To answer your specific question: the hash reader for quote, e.g. #'foo, is analogous to the one for regular quote. It reads the following token and wraps it in function. #'foo is read as (function foo).

It is possible to modify the reader's tables to customize the language. The entries in the table are known as reader macros. A name that tends to confuse people somewhat because they are quite distinct from the macros defined by defmacro. Together they provide what has been called the ability to "grow" the language[4].

[4] http://www.catonmat.net/blog/growing-a-language-by-guy-steele/

Answer 3

Another difference between using #'foo and 'foo as function designators is that #'foo evaluates to a function object, but 'foo evaluates to a symbol. So using 'foo transfers the work of finding the function object to a later time. That can be a noticeable performance hit if you do it in every cycle of a loop instead of just once.

CL-USER> (defun foo () 42)
FOO
CL-USER> (read-from-string "'foo")
=> (QUOTE FOO), 4

CL-USER> (eval *)
FOO
CL-USER> (read-from-string "#'foo")
=> (FUNCTION FOO), 5

CL-USER> (eval *)
=> #<FUNCTION FOO>

Answer 4

The experience shown me that in a big system composed of many parts, the "'" vs. the "#'" idiom makes patching easier. The reason is that the function object associated with the symbol is looked for every time it is encountered and this in every environment. Once and as soon as you've loaded (interactively, of course) a new definition (most likely, a patch) it is used the next time it is encountered. The performance cost is really small, but the advantage of the flexibility is huge. An it is really nice to imagine your customer's face when trying again the application saying "Wow! it works now!" :-)