Using Lisp as a CGI Processor

Using Lisp as a CGI Processor on cubist.cs.washington.edu

This page explains how to use Gnu Common Lisp (gcl) to implement online conversational agents

There are several things you need to know about:

Accessing your account on our web server: In order to use the CGI facilities described here you will need a computer account on a web server that has GCL (Gnu Common Lisp) installed. One such web server is cubist.cs.washington.edu. If you have an account on Cubist, log in using your CSE kerberos password.
Installing an html page and testing it. Try creating an ordinary web page with a filename such as index.html and put it in your www directory. Use a web browser on some other machine and see if you can access the page. If your user name is joey, then the URL would be

http://cubist.cs.washington.edu/~joey/index.html

Installing a CGI script and testing it. The next thing to try is installing a simple Lisp program that begins with the line:

#! /usr/local/bin/gcl.exe -f

and that prints out the correct HTTP header info before anything else. It's the description of the type of content, followed by two newline characters. To keep life simple, just use the following line of Lisp:

(format t "Content-type: text/html~%~%")

Then put in some more FORMAT lines to create some text in the page. For example, to create a page with just a heading, you could use

(format t "<html><body><h1>This is my Lisp CGI script's")

(format t " output!</h1></body></html>~%")

From the Linux command line, type something like
chmod +x mycgi.lsp
(or whatever your filename is, as long as it ends with .lsp) to make the program executable. Try accessing that script with your browser with a URL such as
http://cubist.cs.washington.edu/~joey/mycgi.lsp

How to process POST requests from the user. The key here is to read a string from standard input and parse it to obtain a collection of parameter-value pairs. These pairs are the info sent by the browser. The example program shows how to do this. You can run the sample program and you can examine the source code. It uses my package of Lisp CGI and HTML routines .
Another example is the knock.lsp program, which you can try out here. A version of the Knock-Knock program that uses cookies on the browser is knock-with-cookies.lsp. It seems to do the same thing from the user's point of view, except that it maintains session continuity by storing a session identification string on the client computer in the form of a browser cookie.
How to use a "session" parameter in a form to help maintain a continuous session over a series of HTTP interactions. The web-shrink.lsp program illustrates how this is done. When the user first accesses the program via the URL, there is no form data coming in, and so the program does not find any session parameter. So it makes up a unique session identifier, creates a session file on the server, and prompts the user with an HTML form for the first textual input from the user. When the user submits this form, the session identifier, represented as the value of a hidden parameter in the form, is passed back to the server to identify what session this input is coming from. The Lisp program on the server then uses the session identifier to access the corresponding session file and reconstruct the state of the dialog at the end of the previous interaction.
One advantage of this scheme is that any number of users can be having independent "simultaneous" conversations with the Lisp CGI program. Also, this scheme does not rely on "cookies" on the browser, like the next method does.
How to set a cookie on the user's browser. This will allow the next POST or GET to let the server know who the info is coming from, without relying on the value of a hidden parameter in a form. (See the knock-with-cookies.lsp example program).
How to read a cookie on the user's browser. Once we have set the cookie, we need to read it in subsequent requests from the browser. (See the same example program).
How to create a session file. The way to maintain session continuity from one user request to the next is to keep a file on the server for each session in progress. The name of the file should be based on the user's unique information, so that it doesn't get mixed up with session files of other users. The web-cgi package provides a function (create-session) that will create a session file with a unique file name and return that name. (See either the web-shrink.lsp or the knock.lsp example program).
How to debug your script. "What do I do when I get `Internal Server Error'?". One thing you can do is look at the Web server's error messages. Here are today's error messages with your username. If the internal server error is of the type "Premature end of script headers", this could mean any of several things: (a) Your Lisp program file does not have execute permissions turned on, (b) The first line of the file, which must contain an appropriately formatted path to the gcl executable, may not be right, (c) Your program does not print the "header" as the first output, or (d) there is some other error that prevents the header from ever getting printed.

If you run a Linux system and want to get your own copy of GCL, you can download it from the GNU software site.

Each student should be aware that CGI script writing involves system security risks. We all need to cooperate to see to it that we can all have a successful learning experience and avoid security lapses. Please read the WWW Security FAQ to get familiar with basic security issues.