Day 3: Roswell: Common Lisp scripting

ยท

5 min read

Hi, all Common Lispers.
๐ŸŽŠ And, happy new year in 2022! ๐ŸŽ

I'm happy that you came to this blog again this year ๐Ÿ˜†

I explained how to install Lisp implementations, libraries, and applications with Roswell until the previous article.

Today, I can finally introduce a different aspect of Roswell -- as a scripting supporter.

Scripting in Common Lisp world

Let's think about writing a script in Common Lisp.

It has to accept input as a shell command, execute some code, and quit.

How to achieve it in the REPL first language? In SBCL, the --script option is just for it.

$ sbcl --script hello.lisp

Using this in shebang, it works as a shell command.

#!/usr/local/bin/sbcl --script

(write-line "Hello, World!")

Save this code as a "hello.lisp" file and give it execute permission.

$ ./hello.lisp
Hello, World!

Good work. But, this requires SBCL to be installed at "/usr/local/bin/sbcl". If it's installed at the other location, the shell raises an error:

$ ./hello.lisp
-bash: ./hello.lisp: /usr/local/bin/sbcl: bad interpreter: No such file or directory

This problem is not limited to Common Lisp but also in Python, Ruby, and other languages. A standard solution is to rewrite shebang using /usr/bin/env.

#!/usr/bin/env -S sbcl --script

(write-line "Hello, World!")

The -S option is specified to make /usr/bin/env accept options. If you don't specify it, an error will occur trying to find the program named "sbcl --script".

Command-line arguments can be accessed via sb-ext:*posix-argv*. Looks good, huh?

Is it portable enough?

Of course, this only works with SBCL, but it's not so bad for UNIX-like OSes where SBCL is installed. Not often, but I use this method in Docker containers.

However, what if you want to write a Common Lisp application with a command-line interface that is supposed to run in various environments. Is the SBCL script portable enough?

First of all, it's not easy to load libraries via Quicklisp. Because sbcl --script doesn't read .sbclrc, and don't load Quicklisp even if it's installed.

Besides, it also becomes more difficult if you are aiming for a general-purpose script, such as if you want to run it on other implementations or if you want to support Windows.

Roswell Script

"Roswell script" is a good option in that case.

Start writing a Roswell script that outputs "Hello, World!". ros init is a command to create a template for a Roswell script.

$ ros init hello-world
Successfully generated: hello-world.ros

The output file ending with .ros will look like the following:

#!/bin/sh
#|-*- mode:lisp -*-|#
#|
exec ros -Q -- $0 "$@"
|#
(progn ;;init forms
  (ros:ensure-asdf)
  #+quicklisp(ql:quickload '() :silent t)
  )

(defpackage :ros.script.hello-world.3848003839
  (:use :cl))
(in-package :ros.script.hello-world.3848003839)

(defun main (&rest argv)
  (declare (ignorable argv)))
;;; vim: set ft=lisp lisp:

It will be bewildering at first, so let's take a look at what it does, part by part.

Line 1-5: Hack to make the startup portable

The first 5 lines are for hacking to make the startup process portable. You don't need to know how these lines work, but I'm writing this just for curious people.

#!/bin/sh
#|-*- mode:lisp -*-|#
#|
exec ros -Q -- $0 "$@"
|#

The terminal reads the first line as its shebang and launches the program with /bin/sh. # is a comment in the shell, so the 2-3 lines are skipped. Next, the shell comes to the exec ros line and finally invokes ros.

The second launch is with ros. ros will skip the first shebang. Also, inside of #| to|# will be ignored since it's a multi-line comment in Common Lisp. Then, execute the code below as Common Lisp.

Line 6-9: Initial forms

Lines 6 to 9 are the place to write the code for initialization.

(progn ;;init forms
  (ros:ensure-asdf)
  #+quicklisp(ql:quickload '() :silent t)
  )

Mainly, this is for loading external libraries.

I asked the Roswell author, @snmsts, something like, "Is there any significance to write external dependencies here?" He said, "I think it would be treated special when it's built to the binary, but I don't remember much. That is for fast startup, maybe?"

After some investigation, he found it didn't work as intended. Nowadays, it seems to remain merely a good manner to list all dependencies in one place.

After Line 10: Main part

The rest of the file is a normal Common Lisp program.

(defpackage :ros.script.hello-world.3848003839
  (:use :cl))
(in-package :ros.script.hello-world.3848003839)

(defun main (&rest argv)
  (declare (ignorable argv)))

The main function is always required. It will be the entry function when the Roswell script is executed. It takes command-line arguments as a list of strings.

There are no restrictions on defining other functions and so on. This part can be written like a regular program.

Benefits of Roswell script

Why should you write a script in a Roswell manner? There are 3 benefits to writing Roswell scripts.

First of all, the Roswell script is independent of Lisp implementations. sbcl --script obviously doesn't work other than SBCL; however, the Roswell script works every Lisp supported by Roswell.

The second benefit is automatic script installation with ros install.

I introduced ros install in "Day 2", which is for installing Common Lisp applications/libraries. Not only placing files, but also it treats Roswell scripts special. When the application has a directory named roswell, copy all scripts under it into ~/.roswell/bin. This is the reason why you can use the qlot command right after running ros install fukamachi/qlot.

The last one is binary-build.

Roswell scripts can be built as a binary by running ros build. It can be a boon as it skips reading a file, compilations, and loading external libraries.

To summarize,

  • Lisp implementation portable
  • Automatic installation with ros install
  • Allow building a binary with ros build

These benefits make it the perfect way to distribute a general-purpose application and provide its command-line interface.

Examples

At last, I introduce the real examples of Roswell scripts. See inside ~/.roswell directory of those projects.

  • Qlot
    • A project-local library installer
    • Provides qlot command to install/update project dependencies and utilize them.
  • Clack
    • Web server abstraction layer
    • Provides clackup command to start a web application.
  • lem
    • Common Lisp editor/IDE with high expansibility
    • Provides lem command to start an editor.
ย