Hi, all Common Lispers.
In the previous post, I introduced "Roswell script", the powerful scripting integration of Roswell. Not only does it allow to provide a command-line interface, but it makes it easy to install via Roswell. In this point of view, Roswell can be regarded as a distribution system for Common Lisp applications.
Today's topic is related — the speed of scripts.
Why my simple Roswell script is so slow?
Let's begin with the following simple Roswell script. It's a pretty simple one that just prints "Hello" and quits.
#!/bin/sh
#|-*- mode:lisp -*-|#
#|
exec ros -Q -- $0 "$@"
|#
(progn ;;init forms
(ros:ensure-asdf))
(defpackage :ros.script.hello.3850273748
(:use :cl))
(in-package :ros.script.hello.3850273748)
(defun main (&rest argv)
(declare (ignorable argv))
(write-line "Hello"))
;;; vim: set ft=lisp lisp:
$ ./hello.ros
Hello
Though people expect this simple script to end instantly, it takes longer.
$ time ./hello.ros
Hello
real 0m0.432s
user 0m0.315s
sys 0m0.117s
0.4 seconds to print "Hello". It feels like an early scripting language.
An equivalent script with sbcl --script
is much faster for the record.
$ time ./hello.lisp
Hello
real 0m0.006s
user 0m0.006s
sys 0m0.000s
Fortunately, there're several solutions to this problem.
Hacks to speedup
I have to admit that the Roswell script can't be faster than sbcl --script
since it does many things, but it's possible to make it closer.
Stop loading Quicklisp
The first bottleneck is "Quicklisp".
Quicklisp is a de fact standard and available anywhere today, so we may not realize the cost of loading it. But, it can't be ignored in scripting.
Fortunately, it's easy to disable Quicklisp in the Roswell script. Just replace -Q
with +Q
in the exec
line.
#!/bin/sh
#|-*- mode:lisp -*-|#
#|
-exec ros -Q -- $0 "$@"
+exec ros +Q -- $0 "$@"
|#
(progn ;;init forms
(ros:ensure-asdf))
Let's see the difference.
# No Quicklisp version
$ time ./hello.ros
Hello
real 0m0.142s
user 0m0.119s
sys 0m0.020s
It's approximately 0.3 seconds faster. Conversely, it takes this long to load Quicklisp. This is not little time for a program that starts many times, like scripts.
Additionally, omit ros:ensure-asdf
since ASDF is unnecessary in this script.
exec ros +Q -- $0 "$@"
|#
(progn ;;init forms
- (ros:ensure-asdf))
+ )
(defpackage :ros.script.hello.3850273748
(:use :cl))
$ time ./hello.ros
Hello
real 0m0.072s
user 0m0.052s
sys 0m0.020s
ASDF seems to require to load approximately 0.07 sec. Now, it's 6 times faster.
These changes are effective for small scripts which don't require Quicklisp or ASDF.
Even in the case of scripts that use Quicklisp and ASDF, this method can be applied partially by loading them conditionally.
For example, a script has several subcommands like run
or help
.
Let's suppose run
requires Quicklisp to load the main application, and help
doesn't.
If you use -Q
option, Quicklisp will make help
command slow though it doesn't require Quicklisp.
In this case, it is better to use the +Q
option and load Quicklisp if necessary.
ros:quicklisp
is a function to load Quicklisp manually, even when the ros
started with +Q
. By calling this function right before Quicklisp is needed, it's possible to make the other part faster.
Dump core with -m
What about in case of fairly complicated applications which must require Quicklisp to load external dependencies.
Building a binary is a prevailing solution.
I suppose it won't surprise you. It's a common technique even in no Roswell world. Also, I've mentioned ros build
in the previous article, which makes a binary executable from a Roswell script.
However, we can't assume people always run ros build
to speed up your application after installation.
Roswell takes care of it. Roswell has a feature to build the script dump implicitly to speed up its execution.
Add -m
option to the exec ros
line. Then, enable Quicklisp and ASDF to see how this feature is practical.
@@ -1,10 +1,10 @@
#!/bin/sh
#|-*- mode:lisp -*-|#
#|
-exec ros +Q -- $0 "$@"
+exec ros -Q -m hello -- $0 "$@"
|#
(progn ;;init forms
- )
+ (ros:ensure-asdf))
(defpackage :ros.script.hello.3850273748
(:use :cl))
And install the script.
$ ros install hello.ros
/home/fukamachi/.roswell/bin/hello
Let's try it. It'll take a little time for Roswell to dump a core named hello.core
for the first time.
$ hello
Making core for Roswell...
building dump:/home/fukamachi/.roswell/impls/arm64/linux/sbcl-bin/2.2.0/dump/hello.core
WARNING: :SB-EVAL is no longer present in *FEATURES*
Hello
The second time, it's way faster.
$ time hello
Hello
real 0m0.032s
user 0m0.009s
sys 0m0.024s
It's approximately 13 times faster than the initial version. Of course, it includes the load time of Quicklisp and ASDF.
Remember that this requires the script to be installed at ~/.roswell/bin
via ros install
.
A living example is "lem", a text editor written in Common Lisp.
In the case of "lem", it requires lots of dependencies to run, and people expect a text editor to launch instantly. The dumping core works nicely for it.
# Installation
$ ros install lem-project/lem
# Takes a little time for the first time
$ lem
Making core for Roswell...
building dump:/home/fukamachi/.roswell/impls/arm64/linux/sbcl-bin/2.2.0/dump/lem-ncurses.core
WARNING: :SB-EVAL is no longer present in *FEATURES*
# === lem is opened in fullscreen ===
# Type C-x C-c to quit
It takes a little time to boot up the first time, but the second time is quicker. Also, it'll be dumped again when Roswell detects some file changes.
Note that the name of the core needs to be unique. If there is a conflict, Roswell will load a different core.
Actually, this behavior doesn't go with Qlot well. If there's an application installed in user local and another installed in project local, Roswell can't distinguish between their cores. So then, even if you think you have fixed the version of the library, it will be using a core with a different version loaded.
This is not a problem for independent software like lem, but you should be careful with applications that load other software while running. A bad example of this problem is "Lake".
Conclusion
In this article, I introduced a technique to speed up the startup of Roswell scripts.
- To startup faster
- Add
+Q
to disable loading QuicklispF - Dump cores with
-m
option
- Add
Both have pros and cons.
The nice thing about the -m
option is that the end-user doesn't need to be aware of it, which is a good part of Roswell as a distribution system for Common Lisp applications.