Day 4: Roswell: How to make Roswell scripts faster
5 min read
Hi, all Common Lispers.
In the previous post, I introduced "Roswell script", the powerful scripting integration of Roswell. Not only does it allow to provide a command-line interface, but it makes it easy to install via Roswell. In this point of view, Roswell can be regarded as a distribution system for Common Lisp applications.
Today's topic is related — the speed of scripts.
Why my simple Roswell script is so slow?
Let's begin with the following simple Roswell script. It's a pretty simple one that just prints "Hello" and quits.
#!/bin/sh #|-*- mode:lisp -*-|# #| exec ros -Q -- $0 "$@" |# (progn ;;init forms (ros:ensure-asdf)) (defpackage :ros.script.hello.3850273748 (:use :cl)) (in-package :ros.script.hello.3850273748) (defun main (&rest argv) (declare (ignorable argv)) (write-line "Hello")) ;;; vim: set ft=lisp lisp:
$ ./hello.ros Hello
Though people expect this simple script to end instantly, it takes longer.
$ time ./hello.ros Hello real 0m0.432s user 0m0.315s sys 0m0.117s
0.4 seconds to print "Hello". It feels like an early scripting language.
An equivalent script with
sbcl --script is much faster for the record.
$ time ./hello.lisp Hello real 0m0.006s user 0m0.006s sys 0m0.000s
Fortunately, there're several solutions to this problem.
Hacks to speedup
I have to admit that the Roswell script can't be faster than
sbcl --script since it does many things, but it's possible to make it closer.
Stop loading Quicklisp
The first bottleneck is "Quicklisp".
Quicklisp is a de fact standard and available anywhere today, so we may not realize the cost of loading it. But, it can't be ignored in scripting.
Fortunately, it's easy to disable Quicklisp in the Roswell script. Just replace
+Q in the
#!/bin/sh #|-*- mode:lisp -*-|# #| -exec ros -Q -- $0 "$@" +exec ros +Q -- $0 "$@" |# (progn ;;init forms (ros:ensure-asdf))
Let's see the difference.
# No Quicklisp version $ time ./hello.ros Hello real 0m0.142s user 0m0.119s sys 0m0.020s
It's approximately 0.3 seconds faster. Conversely, it takes this long to load Quicklisp. This is not little time for a program that starts many times, like scripts.
ros:ensure-asdf since ASDF is unnecessary in this script.
exec ros +Q -- $0 "$@" |# (progn ;;init forms - (ros:ensure-asdf)) + ) (defpackage :ros.script.hello.3850273748 (:use :cl))
$ time ./hello.ros Hello real 0m0.072s user 0m0.052s sys 0m0.020s
ASDF seems to require to load approximately 0.07 sec. Now, it's 6 times faster.
These changes are effective for small scripts which don't require Quicklisp or ASDF.
Even in the case of scripts that use Quicklisp and ASDF, this method can be applied partially by loading them conditionally.
For example, a script has several subcommands like
run requires Quicklisp to load the main application, and
If you use
-Q option, Quicklisp will make
help command slow though it doesn't require Quicklisp.
In this case, it is better to use the
+Q option and load Quicklisp if necessary.
ros:quicklisp is a function to load Quicklisp manually, even when the
ros started with
+Q. By calling this function right before Quicklisp is needed, it's possible to make the other part faster.
Dump core with
What about in case of fairly complicated applications which must require Quicklisp to load external dependencies.
Building a binary is a prevailing solution.
I suppose it won't surprise you. It's a common technique even in no Roswell world. Also, I've mentioned
ros build in the previous article, which makes a binary executable from a Roswell script.
However, we can't assume people always run
ros build to speed up your application after installation.
Roswell takes care of it. Roswell has a feature to build the script dump implicitly to speed up its execution.
-m option to the
exec ros line. Then, enable Quicklisp and ASDF to see how this feature is practical.
-exec ros +Q -- $0 "$@" +exec ros -Q -m hello -- $0 "$@" |# (progn ;;init forms - ) + (ros:ensure-asdf)) (defpackage :ros.script.hello.3850273748 (:use :cl))#!/bin/sh #|-*- mode:lisp -*-|# #|
And install the script.
$ ros install hello.ros /home/fukamachi/.roswell/bin/hello
Let's try it. It'll take a little time for Roswell to dump a core named
hello.core for the first time.
$ hello Making core for Roswell... building dump:/home/fukamachi/.roswell/impls/arm64/linux/sbcl-bin/2.2.0/dump/hello.core WARNING: :SB-EVAL is no longer present in *FEATURES* Hello
The second time, it's way faster.
$ time hello Hello real 0m0.032s user 0m0.009s sys 0m0.024s
It's approximately 13 times faster than the initial version. Of course, it includes the load time of Quicklisp and ASDF.
Remember that this requires the script to be installed at
A living example is "lem", a text editor written in Common Lisp.
In the case of "lem", it requires lots of dependencies to run, and people expect a text editor to launch instantly. The dumping core works nicely for it.
# Installation $ ros install lem-project/lem # Takes a little time for the first time $ lem Making core for Roswell... building dump:/home/fukamachi/.roswell/impls/arm64/linux/sbcl-bin/2.2.0/dump/lem-ncurses.core WARNING: :SB-EVAL is no longer present in *FEATURES* # === lem is opened in fullscreen === # Type C-x C-c to quit
It takes a little time to boot up the first time, but the second time is quicker. Also, it'll be dumped again when Roswell detects some file changes.
Note that the name of the core needs to be unique. If there is a conflict, Roswell will load a different core.
Actually, this behavior doesn't go with Qlot well. If there's an application installed in user local and another installed in project local, Roswell can't distinguish between their cores. So then, even if you think you have fixed the version of the library, it will be using a core with a different version loaded.
This is not a problem for independent software like lem, but you should be careful with applications that load other software while running. A bad example of this problem is "Lake".
In this article, I introduced a technique to speed up the startup of Roswell scripts.
- To startup faster
+Qto disable loading QuicklispF
- Dump cores with
Both have pros and cons.
The nice thing about the
-m option is that the end-user doesn't need to be aware of it, which is a good part of Roswell as a distribution system for Common Lisp applications.