How useful are Lisp macros?

Question

Common Lisp allows you to write macros that do whatever source transformation you want.

Scheme gives you a hygienic pattern-matching system that lets you perform transformations as well. How useful are macros in practice? Paul Graham said in Beating the Averages that:

The source code of the Viaweb editor was probably about 20-25% macros.

What sorts of things do people actually end up doing with macros?

Answer 1

Take a look at this posting by Matthias Felleisen to the LL1 discuss list in 2002. He suggests three main uses for macros:

1. Data sublanguages: I can write simple-looking expressions and create complex nested lists/arrays/tables with quote, unquote, etc. neatly dressed up with macros.
2. Binding constructs: I can introduce new binding constructs with macros. That helps me get rid of lambdas and place things closer together that belong together.
3. Evaluation reordering: I can introduce constructs that delay/postpone the evaluation of expressions as needed. Think of loops, new conditionals, delay/force, etc. [Note: In Haskell or any lazy language, this one is unnecessary.]

Answer 2

I mostly use macros for adding time-saving new language constructs, that would otherwise require a bunch of boilerplate code.

For example, I recently found myself wanting an imperative for-loop similar to C++/Java. However, being a functional language, Clojure didn't come with one out of the box. So I just implemented it as a macro:

(defmacro for-loop [[sym init check change :as params] & steps]
  `(loop [~sym ~init value# nil]
     (if ~check
       (let [new-value# (do ~@steps)]
         (recur ~change new-value#))
       value#)))

And now I can do:

 (for-loop [i 0 , (< i 10) , (inc i)] 
   (println i))

And there you have it - a new general-purpose compile-time language construct in six lines of code.

Answer 3

What sorts of things do people actually end up doing with macros?

Writing language extensions or DSL's.

To get a feel for this in Lisp-like languages, study Racket, which has several language variants: Typed Racket, R6RS, and Datalog.

See also the Boo language, which gives you access to the compiler pipeline for the specific purpose of creating Domain-Specific Languages through macros.

Answer 4

Macros are useful to embedded some patterns.

For example, Common Lisp doesn't define the while loop but has do which can be used to define it.

Here is an example from On Lisp.

(defmacro while (test &body body)
  `(do ()
       ((not ,test))
     ,@body))

(let ((a 0))
  (while (< a 10)
    (princ (incf a))))

This will print "12345678910", and if you try to see what happens with macroexpand-1:

(macroexpand-1 '(while (< a 10) (princ (incf a))))

This will return:

(DO () ((NOT (< A 10))) (PRINC (INCF A)))

This is a simple macro, but as said before, they're usually used to define new languages or DSLs, but from this simple example you can already try to imagine what you can do with them.

The loop macro is a good example of what macros can do.

(loop for i from 0 to 10
      if (and (= (mod i 2) 0) i)
        collect it)
=> (0 2 4 6 8 10)
(loop for i downfrom 10 to 0
      with a = 2
      collect (* a i))
=> (20 18 16 14 12 10 8 6 4 2 0)               

Common Lisp has another kind of macros called reader macro which can be used to modify how the reader interprets the code, i.e. you can use them to use #{ and #} has delimiters like #( and #).

Answer 5

Here are some examples:

Scheme:

• define for function definitions. Basically it makes a shorter way to define a function.
• let for creating lexically scoped variables.

Clojure:

• defn, according to its docs:

  Same as (def name (fn [params* ] exprs*)) or (def name (fn ([params* ] exprs*)+)) with any doc-string or attrs added to the var metadata

• for: list comprehensions
• defmacro: ironic?
• defmethod, defmulti: working with multi-methods
• ns

A lot of these macros make it much easier to write code at a more abstract level. I think of macros as being similar, in many ways, to syntax in non-Lisps.

The plotting library Incanter provides macros for some complex data manipulations.

Answer 6

Here's one I use for debugging (in Clojure):

user=> (defmacro print-var [varname] `(println ~(name varname) "=" ~varname))
#'user/print-var
=> (def x (reduce * [1 2 3 4 5]))
#'user/x
=> (print-var x)
x = 120
nil

I had to deal with a hand-rolled hash-table in C++, where the get method took a non-const string reference as an argument, meaning I can't call it with a literal. To make that easier to deal with, I wrote something like the following:

#define LET(name, value, body)  \
    do {                        \
        string name(value);     \
        body;                   \
        assert(name == value);  \
    } while (false)

While something like this problem is unlikely to come up in lisp, I find it particularly nice that you can have macros which don't evaluate their arguments twice, for instance by introducing a real let-binding. (Admitted, here I could have gotten around it).

I also resort to the awfully ugly hack of wrapping stuff in a do ... while (false) such that you can use it in the then-part of an if and still have the else-part work as expected. You don't need this in lisp, which is a function of macros operating on syntax trees rather than strings (or token sequences, I think, in the case of C and C++) which then undergo parsing.

There are a few built-in threading macros which can be used to reorganize your code such that it reads more cleanly ('threading' as in 'sowing your code together', not parallelism). For example:

(->> (range 6) (filter even?) (map inc) (reduce *))

It takes the first form, (range 6), and makes it the last argument of the next form, (filter even?), which in turn is made the last argument of the next form and so on, such that the above gets rewritten into

(reduce * (map inc (filter even? (range 6))))

I think the first reads much more clearly: "take these data, do this to it, then do that, then do the other and we're done", but that's subjective; something that's objectively true is that you read the operations in the sequence they're performed (ignoring laziness).

There is also a variant which inserts the previous form as the first (rather than last) argument. One use case is arithmetic:

(-> 17 (- 2) (/ 3))

Reads as "take 17, subtract 2 and divide by 3".

Speaking of arithmetic, you can write a macro which does infix notation parsing, so that you could say e.g. (infix (17 - 2) / 3) and it would spit out (/ (- 17 2) 3) which has the disadvantage of being less readable and the advantage of being a valid lisp expression. That's the DSL/data sublanguage part.