Hi, all Common Lispers.
I've been writing about Roswell through 5 articles. I saw people trying out Roswell after reading my blog a couple of times, so I guess that helped to get interested.
I think it's time to move. Let's take Qlot for the next topic.
From the perspective of developing and operating applications over the long term, it is irreplaceable and crucial. But I consider that Qlot is still not widely accepted yet among the Common Lisp community. I'm not certain why, but it may be because of the lack of resources to learn it.
Qlot 1.0.0 was out on March 12th. I believe that this tool is now stable enough to be used in production. I hope you will take this opportunity to give it a try.
What's Qlot?
Qlot is a tool to fix versions of dependencies for each project. By fixing the versions of dependencies, it can be avoided breaking the app by version-up of dependencies unintentionally. Qlot ensures that all the same versions will be installed 5 years later.
It's important when running in other environments. Consider a project like a web application whose development environment is different from the environment it actually runs. Without Qlot, it will be cumbersome to use the same dependencies in all environments, regardless of when they are deployed. It's the same about CI/CD environments and other developers' machines.
It is not only useful for applications that connect to the Internet.
After Qlot v0.12.0 (released in November 2021), it got bundle command which allows to download all files of dependencies and make them loadable without Qlot/Quicklisp. It should be useful even for standalone applications.
Setup the project-local Quicklisp with Docker
I won't repeat the same explanations in Qlot's README, like how to use and the tutorial. Instead, I'm going to introduce how to use it with Docker.
As a starting point, create a new file "qlfile" at the project root. It is a file to write project dependencies. It's okay to be empty for now.
# Create a new empty file
$ touch qlfile
Let's install dependencies with it. The following command is equivalent to qlot install except it uses Docker:
# Equivalent to "qlot install"
$ docker run --rm -it -v $PWD:/app fukamachi/qlot install
It creates a new directory named .qlot. It is a project-local Quicklisp directory that contains dependencies written in qlfile. As a default, only a "quicklisp" dist is placed.
$ tree .qlot/dists
.qlot/dists
└── quicklisp
├── distinfo.txt
├── enabled.txt
├── preference.txt
├── releases.txt
└── systems.txt
1 directory, 5 files
There's another file named qlfile.lock at the same directory as qlfile. This file is generated by qlot install to keep track of versions at the time.
On my laptop, the content is like this:
$ cat qlfile.lock
("quicklisp".
(:class qlot/source/dist:source-dist
:initargs (:distribution "http://beta.quicklisp.org/dist/quicklisp.txt" :%version :latest)
:version "2022-02-20"))
Some of the information is unnecessary for humans because it contains internal information for Qlot to use, but it is written here that the quicklisp dist version 2022-02-20 is used for this project.
While qlfile.lock exists, qlot install downloads quicklisp 2022-02-20 even when the newer version is released.
When you use VCS, like git, you don't want to version the .qlot directory since it contains a large number of source files of dependencies. The directory can be reproducible from qlfile.lock anytime by running qlot install.
$ echo .qlot/ >> .gitignore
$ git add qlfile qlfile.lock
$ git commit -m 'Start using Qlot.'
Using the project-local Quicklisp
There're several ways to use the project-local Quicklisp. REPL can be launched with Docker image by the following command:
# Equivalent to 'qlot exec ros run'
$ docker run --rm -it -v $PWD:/app fukamachi/qlot exec ros run
* ql:*quicklisp-home*
#P"/app/.qlot/
* (ql-dist:dist "quicklisp")
#<QL-DIST:DIST quicklisp 2022-02-20>
However, it would be inconvenient since it's inside a separated Docker container. Actually, it's possible to be loaded without Qlot, like these:
# With Roswell
$ QUICKLISP_HOME=.qlot/ ros run
# With sbcl command
# The point is loading .qlot/setup.lisp
$ sbcl --no-userinit --load .qlot/setup.lisp
It also can be applied to other implementations as long as it loads .qlot/setup.lisp on startup.
Let's see where to load the Quicklisp on the launched REPL:
* ql:*quicklisp-home*
#P"/Users/fukamachi/myproject/.qlot/"
* (ql-dist:dist "quicklisp")
#<QL-DIST:DIST quicklisp 2022-02-20>
It seems fine.
Adding a new dependency
Since here, Qlot only keeps track of the version of the Quicklisp dist.
Let's add another dependency, "clack" from GitHub. Add a line to qlfile and run qlot install.
# Run 'qlot add'
$ docker run --rm -it -v $PWD:/app fukamachi/qlot add github clack fukamachi/clack
Add 'github clack fukamachi/clack' to 'qlfile'.
Reading '/app/qlfile'...
Already have dist "quicklisp" version "2022-02-20".
Installing dist "clack" version "github-6fd0279424f7ba5fd4f92d69a1970846b0b11222".
Successfully installed.
# Same as the above
$ echo 'github clack fukamachi/clack' >> qlfile
$ docker run --rm -it -v $PWD:/app fukamachi/qlot install
After running qlot install, it applies the changes to qlfile.lock and .qlot/ directory. Now Clack of the latest GitHub version can be discovered in REPL:
$ QUICKLISP_HOME=.qlot/ ros run
* (ql:where-is-system :clack)
#P"/Users/fukamachi/myproject/.qlot/dists/clack/software/clack-6fd0279424f7ba5fd4f92d69a1970846b0b11222/"
If the added project provides Roswell scripts, Qlot adds scripts with the same names under .qlot/bin/. They are the same as the original scripts, except that they always use the version fixed in Qlot.
It refers to the default branch of GitHub (typically master or main), but it also can specify a specific branch or tag. See the README "qlfile syntax" section for detail.
Updating the version of dependencies
When you want to update a fixed version to the latest version, use qlot update.
For instance, when you find some changes of Clack on GitHub and want to use the newest version, run qlot update --project clack:
# Update a specific project (ex. Clack)
$ docker run --rm -it -v $PWD:/app fukamachi/qlot update --project clack
# Update all
$ docker run --rm -it -v $PWD:/app fukamachi/qlot update
qlot update works something like that ignores qlfile.lock, runs qlot install again, and updates the existing qlfile.lock.
Bundling dependencies
To dump all dependencies to the project root, qlot bundle is available.
Considering a project ASDF system like this:
(defsystem "myproject"
:depends-on ("clack"
"lack"))
qlot bundle extracts "clack" and "lack" including their dependencies into ".bundle-libs" directory.
$ docker run --rm -it -v $PWD:/app fukamachi/qlot bundle
To use it, just load .bundle-libs/bundle.lisp. It should work without Qlot or Quicklisp.
Alright, I explained through all daily operations with Qlot briefly.
In the next article, I will explain how to create your own Docker image based on this Docker container.