Why are Lisp programs a sequence of S-expressions and not a single list?

Question

This is quite basic question that started puzzling me recently while studying Lisp (and derivatives). I have read some Lisp books and web sites and this subject is somewhat obscured by other Lisp properties that seem to be having more meaning to the authors so they give them more attention. So I cannot find this subject in books and I have no one to ask to help me out figure it out, because I do not know anyone who 'speak' Lisp. So this forum seem to be appropriate place to ask.

Here's a question (explanation first):

Lisp program can be generally described as a list of s-expressions executed in sequence (in a file, including other source files loading). As far as I understand Lisp program generally looks like this:

(op data, data...)
(op data, data...)
(op data, data...)
(op data, data...)
(op)
.... ; presumably endless stream of s-expressions (like in 'repl')

whereby above s-expressions may take number of different forms (function definitions, variable declarations, branching and flow control etc...)

So, how come Lisp program is not defined as:

(program
   (op data, data...)
   (op data, data...)
   (op data, data...)
   (op data, data...)
   (op)
   ... ; presumably append-able list of s-expressions
)

?

It would seem logical and natural that program is a list of s-expressions to execute (as it is already implicitly understood just looks like missing explicit list declaration).

Is there particular (design) reason why first form is used and not the second? ( Or reasons are completely historical? Or ?)

Thank you!

Answer 1

One of the reasons is that a single s-expression is slighty less practical in actual programming in many cases.

Remember, the evaluator in Lisp works like this: it reads a form and then evaluates it. The file compiler in Lisp works like this: it reads a form, then compiles it and creates the compiled code.

Now if we have a single form in a file, then it might have some drawbacks:

• the form may be very large and the reader would return a very large data structure for the program

• if there is a s-expression syntax error, just reading that large form creates an error. Thus we would not work with smaller expressions and have eventually an error, instead we would always get the error from the s-expression reader for the whole program up-front. That can be undesirable.

• a program may want to manipulate the state of Lisp, such that the reader would read the following expressions differently. But that's not possible if all code is in an already read expression. Common Lisp for example has packages, which are symbol namespaces. Changing a namespace might for example import a symbol and one might later use that symbol as it was imported. These things would not be possible that way if we have a single s-expression for the code.

• all your code would be already indented by some characters, by the usual Lisp code formatting conventions, which wastes horizontal space

Of course it is possible to write code in one expression, but as indicated above it is usually seen as less flexible and more problematic to edit in conventional text editors.

CL-USER 16 > (defmacro program (&body body) `(progn ,@body))
PROGRAM

CL-USER 17 > (program
               (defun foo (bar) (1+ bar))
               (defun baz (bar) (* pi (foo bar)))
               (+ (foo 10) (baz 20)))
76.97344572538566D0

Common Lisp was designed in such a way that:

• there are top-level defining macros like defun, defmacro, defvar and defclass for individual definitions.

• there are special macros and functions influencing the execution and compilation, which are not nesting: in-package, proclaim, ...

• there are some operators which are influencing execution and compilation, which are nesting: progn, eval-when, ... But these are usually not used to group large amounts of code - but sometimes in code which is generated by Lisp.

Answer 2

It would seem logical and natural that program is a list of s-expressions to execute (as it is already implicitly understood just looks like missing explicit list declaration).

Only if you think of programs as lists of orders to execute like a recipe.

That is not how lisp was designed, and not the mental model for functional programming at all. There, it is logical and natural that a program is a function - very often comprised of progressively smaller functions. You give it some input, it gives you some output. Very mathematical, and very reasonable for the people of the time that knew of computers as things that compute.

Though lisp the language wasn't really designed so much as McCarthy took Lambda Calculus and implemented it. Lambda Calculus in turn wasn't designed for computers at all (since it predated electronic computers by a few decades). It was trying to express the simplest sort of foundation for computation in general. It's intuitive for us to think about lists as lay-people, but they don't really exist in mathematics.

Answer 3

I think a big part of it is that there isn't an overall sequence to top-level code in most cases. Sure, maybe the inside of a function is sequential, but the code as a whole is not, and the order of functions usually isn't significant.