Following on from our work implementing support for compiling multiple units at once in GHC, we have now been extending the ecosystem to take advantage of this new support. This work has once again been made possible by Hasura. This work continues our productive and long-running collaboration on important and difficult tooling tasks which will ultimately benefit the entire ecosystem.
This post focuses on updates to the
cabal repl command, allowing multiple components to be loaded at once into an interactive session. The work is being reviewed in Cabal MR #8726, and should be available in a future release of
Multiple Component Repl
cabal, most commands take a “target” which specifies which units you want to operate on. A command such as
cabal build <target> will resolve all the units that the target
<target> resolves to, and build all of them. The behaviour of the
cabal repl command is different: you must specify a single unit to build.
Here are some common targets which you can specify when using
all: Build all the locally defined components.
exe:haskell-language-server: Build the executable called
lib:pkg-a lib:pkg-b: Build the local libraries pkg-a and pkg-b.
src/Main.hs: Build the unit which
After enabling multi-repl, passing a target specification to
cabal repl which resolves to multiple units will load all those units into a single repl session. For example:
cabal repl --enable-multi-repl lib:pkg-a lib:pkg-b
When the modules are compiled, the unit which they came from is listed next to the module name. The
interactive herald in the build plan indicates that the library will be loaded into GHCi rather than being built like a normal package.
In order, the following will be built (use -v for more details): - pkg-a-0 (interactive) (lib) (first run) - pkg-b-0 (interactive) (lib) (dependency rebuilt) Preprocessing library for pkg-a-0.. Preprocessing library for pkg-b-0.. GHCi, version 9.4.3: https://www.haskell.org/ghc/ :? for help [1 of 2] Compiling Foo[pkg-a-0-inplace] [2 of 2] Compiling Bar[pkg-b-0-inplace] Ok, two modules loaded.
You will need to use at least
ghc-9.4.1 in order to use multiple unit support. It’s advised to use
9.6.1, in order to benefit from bug fixes.
There are three ways to enable the multi-repl depending on how much you like it:
- Globally: Add
multi-repl: Trueto your
- Project-wide: Add
multi-repl: Trueto your cabal.project file.
- Per-invocation: Pass
A future cabal version is likely to enable multi-repl by default. For the time being, and due to the experimental nature of the command and lack of support in ghci for some features, the multi-repl feature is opt-in.
Closure Property for Multiple Home Units
For tools or libraries using the GHC API there is one very important closure property which must be adhered to:
Any dependency which is not a home unit must not (transitively) depend on a home unit.
For example, if you have three units
p depends on
q which depends on
r, then it is illegal to load both
r as home units but not
q is a dependency of the home unit
p which depends on another home unit
cabal will automatically enable loading of all units which are needed by the closure property (including non-local) packages. Given the previous example, if you specify on the command line
cabal repl lib:p lib:q then
lib:r will also be loaded into the same session as it is needed for the closure property.
Configuring and Promised Dependencies
The lowest-level interface which the
Cabal library provides in order to build a package is the
Setup.hs script. This consists of a normal Haskell file which depends on the
Cabal library and can be executed in order to build the package. This is done, after compiling
Setup.hs, via the following invocations:
./Setup configure ./Setup build
configure phase checks to make sure that everything is in order so that when the build phase is run we know that all the environmental dependencies have already been provisioned by the user.
In the very old days, people would compile and run
Setup.hs themselves in order to build a package, but these days, all the interactions with
Setup.hs are managed by a higher-level build tool such as
nix. All of these tools ultimately call
The main technical change to enable the multi-repl was to modify the
Setup.hs scripts to allow you to configure a package before all its dependencies are built. Now you can promise to
Setup.hs that a certain dependency will be built by the time we attempt to build the unit. Since all units in a project are going to be built at the same time with one GHC invocation, they all need to be configured before anything else is built. So we just promise that all local packages will be built.
./Setup configure --promised-dependency=pkg-a
In addition to the
Setup.hs also provides a
repl command which starts
GHCi and loads a single component.
This design is quite awkward because the
Setup.hs scripts operate on a per-component basis. The interface is not aware of the existence of complicated multi-component projects, that is solely the domain of higher-level tools like
cabal-install. Therefore, instead of starting the repl from the
Setup.hs script, we need to start a multi-repl from
cabal-install. However, the
Setup.hs script is still responsible for computing the arguments we need to pass to GHC in order to compile that component. The solution is to allow the
repl command to write its arguments into a file so that they can be collected later by
cabal-install to correctly start a multi-component session.
./Setup repl --repl-multi-file=multi-args # Arguments can be found in the `multi-args` directory.
This allows all the units in your project to be configured before any of them are built. After a project is configured, the
Setup executable can be consulted to find out what options GHC would use to build the unit, and because we have promised to make sure things are built in the right order, we can supply these options to GHC in order to start a multi unit GHCi session.
HLS support for multiple home units
Zubin has already updated HLS to use native multiple home unit support for GHC-9.4.
The missing piece has been a mechanism to set up a multi component session which satisfies the closure property. Without such a mechanism, HLS would construct a multiple component session incrementally by adding units to a session as they are opened by the user. For a complicated project structure, users would have to very carefully load their files in the right order to get a session which worked correctly. Even worse, this doesn’t even work when a non-local package is needed to satisfy the closure property.
HLS consults cabal in order to set up a session: it invokes
cabal repl and intercepts the final call to
ghc which would start the repl. That command is then used as the options which are needed for the session in order to compile that unit.
cabal repl supports creating a command line which specifies the options for multiple components at once, it makes sense to augment the HLS session loading logic to also understand these command lines in order to set up a whole multi-component session at once.
HLS now can understand and parse the kind of command line produced by a multiple component session. As a result:
- The correct session is initialised up-front. Loading any component in your local project will work seamlessly and quickly.
- The time taken to initialise a session is reduced, because no local dependencies are built before the session is started. All local components are configured before anything is built.
- Cabal ensures the closure property holds, even for non-local packages.
I have been testing this support when working on
ghc, both projects with many local dependencies and the experience is much improved. In particular for
cabal, the non-local
hackage-security package is needed for the closure property but could never be loaded before. This made using HLS on
cabal very error-prone because if you opened a file from the
Cabal library and
cabal-install library, you would break the session without a way to recover it. For
ghc, it is a lifeline to be able to edit packages like
template-haskell and see the changes ripple upwards through all the boot libraries and compiler.
Now that there is a way to easily create and invoke a multi-repl session, users are probably going to run into limitations of the multi-repl.
Many features are not yet implemented because there is not a good way to change what the “active unit” of the repl session is. Some more careful thinking needs to be done to modify the GHCi interface in order to work nicely with multiple components in all situations.
At this time, the multi-repl is best used for interactive development situations where you want to use the repl to obtain fast-feedback about your project. We have made sure that the multi-repl works with
ghcid for example.
cabal repl support for multiple home units allows developers to easily interact with multiple home unit support in GHC. There are still limitations to the repl supported in multiple unit sessions, but as more users start using and wanting this feature we hope to expand the repl to work properly with multiple home units as well.
Well-Typed is able to work on GHC, HLS, Cabal and other core Haskell infrastructure thanks to funding from various sponsors. If your company might be able to contribute to this work, sponsor maintenance efforts, or fund the implementation of other features, please read about how you can help or get in touch.