fukamachi tech notes

How to build a web app with Clack/Lack (1)

Eitaro Fukamachi's photo
Eitaro Fukamachi
·

7 min read

Hi, all Common Lispers.

Sorry for the delay in publishing this. It was hard to wrap up this important topic, web development.

As I said in the previous post, today's topic is Clack and Lack.

What's Clack and Lack?

"Clack" and "Lack" are the set of web libraries to define protocols for web applications and servers. They are intended to make web applications modular and reusable.

Clack is an abstraction layer for web servers. It acts as a web application adaptor. It makes web applications loosely coupled from the web server and makes migration between servers easy.

Lack is a rule that proposes implementing web applications as a set of reusable components and increasing the portion that is not dependent on a particular framework.

Clack, an HTTP abstraction layer

Let me begin with the introduction of Clack.

With Clack and without Clack

In the old days, web applications were tightly coupled to the web server on which they were run. A typical example is Hunchentoot, a web server that provides some web framework-like facilities, like routing and session management.

In such a system, it's hard to run an application on multiple kinds of servers. As I mentioned in the Woo article, Hunchentoot has a performance problem though it's easy to use in development. In addition, the dependence of web frameworks on web servers divides the web development community and makes it difficult for them to share their efforts.

Usage of Clack

Nothing difficult. It provides a function clack:clackup to start a web server and clack:stop to stop it.

;; Lack application
;; (Just a function, it will be mentioned later)
CL-USER> (defvar *app*
          (lambda (env)
           '(200 (:content-type "text/plain") ("Hello, Clack!"))))
;=> *APP*

;; Start Hunchentoot server at port=5000 (Default)
CL-USER> (clack:clackup
           *app*
           :server :hunchentoot)
;-> Hunchentoot server is started.
;   Listening on 127.0.0.1:5000.
;=> #S(CLACK.HANDLER::HANDLER
;      :SERVER :HUNCHENTOOT
;      :SWANK-PORT NIL
;      :ACCEPTOR #<SB-THREAD:THREAD "clack-handler-hunchentoot" RUNNING
;                   {10089A1493}>)

;; Start Woo server at port=5050
CL-USER> (clack:clackup
           *app*
           :server :woo
           :port 5050)
;-> Woo server is started.
;   Listening on 127.0.0.1:5050
;=> #S(CLACK.HANDLER::HANDLER
;      :SERVER :WOO
;      :SWANK-PORT NIL
;      :ACCEPTOR #<SB-THREAD:THREAD "clack-handler-woo" RUNNING {1009DBCD63}>)

clack:clackup takes :server to which server to start. The following servers are supported.

  • Hunchentoot: :hunchentoot
  • Woo: :woo
  • Wookie: :wookie
  • Toot: :toot
  • FCGI (cl-fcgi): :fcgi

If you'd like to add a new web server, look into src/handler/ directory of the Clack repository.

clackup command

A function clack:clackup can be invoked from the command-line interface via clackup command.

While in development, people usually use the REPL interface, but in the production or Docker container, this kind of shell interface should be helpful.

Create a file app.lisp containing a Lack application something like:

;; app.lisp
(lambda (env)
  (declare (ignore env))
  '(200 (:content-type "text/plain") ("Hello, Clack!")))

And specify the file path to clackup command:

;; Start Hunchentoot with a swank server
$ clackup app.lisp --server hunchentoot --swank-port 4005

;; Start Woo, without CL debugger
$ clackup app.lisp --server woo --debug nil

Those options should be understandable since they are pretty straightforward.

Lack, the application side

Then, let's look into the application side that runs on Clack.

Lack is a web application/framework builder library. So if you're a web application engineer or a web framework developer, I want you to know about it.

As I wrote at the top of this article, Lack aims to make web applications modular and reusable by separating them into small components which follow a specific rule.

Typical HTTP applications receive a request and return a response as HTML. By considering this simple, web applications are a function that takes an input and outputs something.

HTTP application

Literally, in Lack, applications are functions. The following example is an app that always returns "Hello, World!".

(lambda (env)
  (declare (ignore env))
  '(200 (:content-type "text/plain") ("Hello, World!")))

Request

The environment env is a property list given by a web server containing the following keys:

  • :request-method (Required, Keyword)
    • The HTTP request method: :GET, :HEAD, :OPTIONS, :PUT, :POST, or :DELETE.
  • :script-name (Required, String)
    • The initial portion of the request URI path corresponds to the Clack application. The value of this key may be an empty string when the client is accessing the application represented by the server's root URI. Otherwise, it is a non-empty string starting with a forward slash (/).
  • :path-info (Required, String)
    • The remainder of the request URI path. The value of this key may be an empty string when you access the application represented by the server's root URI with no trailing slash.
  • :query-string (Optional, String)
    • The portion of the request URI that follows the ?, if any.
  • :url-scheme (Required, String)
    • "http" or "https", depending on the request URI.
  • :server-name (Required, String)
    • The resolved server name or the server IP address.
  • :server-port (Required, Integer)
    • The port on which the request is being handled.
  • :server-protocol (Required, Keyword)
    • The version of the protocol the client used to send the request: typically :HTTP/1.0 or :HTTP/1.1.
  • :request-uri (Required, String)
    • The request URI. Always starts with "/".
  • :raw-body (Optional, Stream)
    • The new body of the request.
  • :remote-addr (Required, String)
    • The remote address.
  • :remote-port (Required, Integer)
    • The remote port.
  • :content-type (Optional, String)
    • The header value of Content-Type.
  • :content-length (Optional, Integer)
    • The header value of Content-Length.
  • :headers (Required, Hash-Table)
    • A hash table of headers.

Note that any keys can be added to this env between a web server and an app, like web servers, Lack middlewares, or other related libraries.

Here's an example when requesting to http://localhost:5000/path?q=aiueo:

(:REQUEST-METHOD :GET
 :SCRIPT-NAME ""
 :PATH-INFO "/path"
 :SERVER-NAME "localhost"
 :SERVER-PORT 5000
 :SERVER-PROTOCOL :HTTP/1.1
 :REQUEST-URI "/path?q=aiueo"
 :URL-SCHEME "http"
 :REMOTE-ADDR "127.0.0.1"
 :REMOTE-PORT 45668
 :QUERY-STRING "q=aiueo"
 :RAW-BODY #<FLEXI-STREAMS:FLEXI-IO-STREAM {1003118383}>
 :CONTENT-LENGTH NIL
 :CONTENT-TYPE NIL
 :CLACK.STREAMING T
 :CLACK.IO #<CLACK.HANDLER.HUNCHENTOOT::CLIENT {100320A083}>
 :HEADERS #<HASH-TABLE :TEST EQUAL :COUNT 3 {100320A103}>)

Response

The response can be 2 kinds.

Normal response

The normal response is a list of 3 elements, which respectively expresses an HTTP status code, headers, and response body data.

  • The status code (Integer >=100)
    • An integer greater than or equal to 100
    • ex. 200 404
  • The headers (Property List)
    • Keys are keywords
    • ex. (:content-type "text/html")
  • The response body (List of strings / Byte vector / Pathname)
    • ex. ("Hello, World!") #(72 101 108 108 111 44 32 87 111 114 108 100) #P”index.html”

Delayed response

Another type of response is delayed response. If you'd like to stream something to the client, this will be the one.

Such an application must return a function, which takes a function.

(lambda (env)
  (lambda (responder)
    ;; Return the HTTP status code and the headers first.
    ;; It returns another function to write a chunk.
    (let ((writer (funcall responder '(200 (:content-type "text/plain")))))
      ;; Use writer multiple times
      ;; (funcall writer <body chunk>)
      ;; The body chunk must be one of string, byte vector or nil

      ;; Ends with :close t
      (funcall writer nil :close t))))

If you prefer an output stream instead of a function for some reason, lack.util.writer-stream will help.

(ql:quickload :lack-util-writer-stream)
(import 'lack.util.writer-stream:make-writer-stream)

(lambda (env)
  (lambda (responder)
    (let* ((writer (funcall responder '(200 (:content-type "text/plain"))))
           (stream (make-writer-stream writer)))
      ;; (format stream <something>)

      ;; Ends with cl:finish-output
      (finish-output stream))))

There is a slight performance tradeoff since it's implemented with trivial-gray-streams.

Here is an example that can actually run. It returns the string "Hello, World!" one character at a time at one-second intervals.

(lambda (env)
  (declare (ignore env))
  (lambda (responder)
    (let ((writer (funcall responder '(200 (:content-type "text/plain")))))
      (loop for chunk across "Hello, World!"
            do (funcall writer (princ-to-string chunk))
               (sleep 1)
            while chunk
            finally (funcall writer nil :close t)))))

After running the app with clackup, send a request with curl:

# Add -N option to disable buffering
$ curl -N http://localhost:5000

It shows "Hello, World!" in the terminal slowly.

Last words

Still, many things need to talk about Lack, but it's already long enough to finish.

In the next article, I'll introduce Lack's Middleware and how to test Lack apps.