Are Lisp source code files themselves lists?

Asked 2012-05-09T16:47:39.610

No matter the Lisp dialect, it looks like every source code file containing Lisp functions isn't itself a list (the first time I was "surprised" by this was when working on my Emacs .el files).

I've got a few questions but they're all related to the same "issue" and it's probably just me misunderstanding a few things.

Is there a reason why source code files for the various Lisp dialects seems to be a bunch of "disorganized" functions like this:

(function1 ...)
(function2 ...)
(function3 ...)

Instead of a "Lisp list" of functions, maybe like this:

(
  '(function1 ...)
  '(function2 ...)
  '(function3 ...)
)

I'm a bit surprised in this whole "code is data, data is code" thing to see that source code file themselves apparently aren't neat lists... Or are they!?

Are the source code files something you're supposed to "manipulate" or not?

What if I wanted to, say, convert one of my .clj (Clojure) source file to some CSS+HTML webpage, isn't it a "problem" that the source code file apparently isn't itself a list?

I'm beginning with Lisp so I don't know if my question makes sense or not and any explanation would be welcome.

Edit Report

2 Answers

In Common Lisp a source file contains lisp forms and comments. Lisp forms are either data or Lisp code. Typical operations on a source file are done by the functions LOAD and COMPILE-FILE.

LOAD would read forms from a file and execute them one by one.

COMPILE-FILE is much more complex. It typically reads forms and compiles them to some other representation (machine code, byte code, C code, ...). It does not execute the code.

What would it help you if the file contain one list of forms instead of just multiple forms below each other?

you would have one level of added parentheses
you would have to read the whole list before you can do anything with it (or alternatively you need a different reader mechanism)
adding forms to the end of a file by a program would be a pain
you can't add something into the file which changes the reader interpretation of the rest of the file
files can't be infinitively long for LOAD

Now for a example a compiler would read lisp forms from a file stream and compile them piece by piece.

If you want all forms you can do

CL-USER 170 > (defun read-forms (file)
               (with-open-file (stream file)
                 (loop for form = (read stream nil nil)
                       while form
                       collect form)))
READ-FORMS

CL-USER 171 > (read-forms (capi:prompt-for-file "source file"))
((DEFPARAMETER *UNITS-TO-SHOW* 4.1)
 (DEFPARAMETER *TEXT-WIDTH-IN-PICAS* 28.0)
 (DEFPARAMETER *DEVICE-PIXELS-PER-INCH* 300)
 (DEFPARAMETER *PIXELS-PER-UNIT* (* (/ (/ *TEXT-WIDTH-IN-PICAS* 6)
                                       (* *UNITS-TO-SHOW* 2))
                                    *DEVICE-PIXELS-PER-INCH*))
...

If you want to put parentheses around everything use PROGN:

 (progn
   'form-1
   (defun function-d


                        
                        
                            answered 2012-05-09T19:34:30.087


            
            
                
                    
                        
                            
                            
                        
                        6
                        
                            
                            
                        
                        
                    
                    
                        
                        To be thorough, all the source files are text, not lisp data structures. To evaluate or compile the code, the lisp must first READ the file, which means to transform the text to lisp data structures. Recall the acronym REPL, for which the first two letters stand for READ, and EVAL. READ takes a string representation of the code, and returns a data structure representing the code. EVAL takes the returned data structure, and interprets (or compiles and runs) the data structure as code. Thus, its important to remember that there are intermediate steps involved.

A good question is, what happens when multiple s-expressions are passed to READ, and they are not in a list, as you mentioned?

If you look at the code, you'll usually find multiple versions of READ, clojure's read-string only reads and returns the first s-expression, ignoring the rest. But, the reader used in clojure's load-file, will take the whole string, and "effectively" (implementations may differ) wrap an implicit do (or progn in common lisp) around all of the forms, and then pass that to eval. This behavior contrasts to what happens in the REPL, forms are read, evaluated, and printed sequentially.

In both cases though, this "behind the scene" behavior is a trade-off made for concision. We can assume when we load a file of text of s-expressions, we want them all to be evaluated, and at most return the value of the last s-expression.

                        
                        
                            answered 2012-05-09T18:47:08.717



        
        
            Your Answer