This is the twenty-fourth edition of our GHC activities report, which describes the work Well-Typed are doing on GHC, Cabal, HLS and other parts of the core Haskell toolchain. The current edition covers roughly the months of June to August 2024. You can find the previous editions collected under the ghc-activities-report tag.
Sponsorship
We are delighted to offer new Haskell Ecosystem Support Packages to provide commercial users with access to Well-Typed’s experts while investing in the Haskell community and its technical ecosystem. If your company is using Haskell, read more about our offer, or get in touch with us today, so we can help you get the most out of the toolchain. We need more funding to continue our essential maintenance work!
Many thanks to our existing sponsors who make this work possible: Anduril and Juspay. In addition, we are grateful to Mercury for funding specific work on improved performance for developer tools on large codebases.
Team
The GHC team at Well-Typed currently consists of Ben Gamari, Andreas Klebinger, Matthew Pickering, Zubin Duggal, Sam Derbyshire and Rodrigo Mesquita. Cabal maintenance is undertaken by Mikolaj Konarski. In addition, many others within Well-Typed are contributing to GHC more occasionally.
Releases
- Zubin prepared and released GHC 9.6.6.
- Mikolaj assisted Artem Pelenitsyn with the release of cabal-install 3.12.1.0.
Profiled dynamic support
Matthew has added support for building executables with the combination of dynamic linking and profiling, building on top of work by Ian-Woo Kim.
On the GHC side, we need to build and distribute libraries built in the profiled dynamic way. This required a few small changes in Hadrian (!12595).
The bulk of the work was adding support in Cabal (#9900).
With Cabal >= 3.14 and GHC >= 9.12, libraries can be compiled in the
profiled dynamic way by passing --profiling-shared. Passing --enable-profiling
together with --enable-executable-dynamic will then allow one to build
executables with profiling which are dynamically linked against the appropriate
p_dyn archives.
Object determinism
Matthew and Rodrigo have been working on making it so that the object code that GHC produces is deterministic (#12935). One main benefit of determinism is improved caching of build products.
Matthew and Rodrigo have identified several sources of non-determinism (not just in object files):
Unique identifiers in Cmm are generated non-deterministically. Making these deterministic is the main challenge; it can be achieved by a combination of:
Threading through a unique supply to ensure that the uniques are generated in a deterministic order,
Where that isn’t possible (e.g. due to performance implications), recover determinism in a subsequent renaming pass.
This is the work of the main MR !12680, which is still in progress at the time of writing.
Re-exports of entire modules can cause non-determinism in interface files (#13093). This is fixed by using a stable sort of the exports (!13093).
Files added by
addDependentFilecan be added in a non-deterministic order (#25131); this is fixed by using a deterministic module environment in GHC (!13099), and avoiding temporary file names which use randomly generated names inside Template Haskell.The existence of rules can impact inlining decisions, and a bug in GHC (#19725) means that sometimes rules that are not in the dependency closure of a given module can impact inlining decisions in that module (#25170); as these rules may or may not be loaded (depending e.g. on parallel compilation), this causes non-determinism.
Build paths could leak into object files, e.g. a preprocesor could introduce a
LINEpragma that referred to a build path, due to a regression in Cabal (#10242). Matthew addressed a plethora of issues surrounding the treatment of working directories in Cabal in #10256, putting this problem to rest.
SIMD
SIMD (“Single Instruction, Multiple Data”) refers to CPU support for instructions
that operate on short vectors of data in parallel. For example, on X86-64 with
SSE instructions, one can use a single mulps
instruction to multiply two vectors containing 4 32-bit floating point values,
element by element. Making good use of these instructions can often be critical
in high-performance applications, in particular those involving array processing.
In Haskell, SIMD is exposed through GHC.Exts, with e.g. FloatX4#
representing a 4-element vector of Float#s, and timesFloatX4# the corresponding elementwise multiplication operation
that lowers to mulps on X86-64.
Historically, these types and operations were only supported in GHC’s LLVM backend
(i.e. one had to use -fllvm), and not GHC’s native code generator (NCG).
However, Sam has been working to add SIMD support to GHC’s X86 native code generator.
The first challenge was fixing register allocation, which takes in Cmm code and
assigns variables to registers. GHC didn’t previously keep track of how registers
were used, but SIMD instructions can use registers such as xmm0 to store many
different things, such as f64 (Double#) and v4f32 (FloatX4#).
The register allocator needs to know precisely what is stored in the register,
so that if we need to spill the register to the stack we know how much to spill
(and conversely how much to load when reloading from the stack).
Tracking the usage of registers was made possible in Cmm with previous work (!9167);
it was then a case of propagating this information through the register allocator
and emitting the correct store and load instructions.
However, work in this area revealed further issues with the existing SIMD support in GHC, thus affecting both the NCG and LLVM backends:
- The generated Cmm code for unknown function calls involving SIMD vectors was plain broken (#25062).
- GHC is liable to drop the upper part of vector registers around hand-written Cmm code in the RTS (#25169).
There were a few other tricky aspects to the implementation, in particular to account for the fact that the Windows X86_64 C calling convention dictates that vector values must be passed by reference and cannot be passed directly in registers.
Haddock
The Haddock tool for generating Haskell documentation has been merged into the GHC tree. This greatly simplifies the contribution workflow, especially as GHC patches that required Haddock changes used to require a delicate dance with git submodules.
Zubin finalised the merge, restoring missing commits that were scattered across various branches (!12720). He then integrated the building and testing of Haddock with GHC’s build system, Hadrian (!12743).
Zubin then fixed many other outstanding issues with Haddock, including:
Handling of non-hs files, so that haddock can generate documentation for modules with foreign imports (!13008)
Several issues with incorrect cross-package references (!12980)
Bindist testing
Historically, the GHC project did not substantively test the deployment of
produced GHC binary distributions. After several releases in which the CI
of the ghcup project revealed issues, Matt implemented
a more robust suite of tests for GHC bindists, testing both ghcup usage and
manual installation.
This culminated in the creation of the
ghcup-ci repository,
which will be used henceforth as part of the GHC release process.
Semaphores for faster parallel builds
We previously introduced a new feature to GHC and Cabal allowing them to share compute resources more effectively using a semaphore: see our previous blog post on Reducing Haskell parallel build times using semaphores.
An issue with this feature was reported on the Cabal issue tracker (#9993).
Zubin investigated and discovered that this was due to cabal-install and
ghc being compiled and linked against inconsistent implementations of libc.
For example, if cabal-install is built against musl, it will create
a POSIX semaphore named /dev/shm/cabal_semaphore_k, but ghc built against glibc
would instead attempt to read /dev/shm/sem.cabal_semaphore_k.
This is not simply a naming issue, as the semaphore implementations can be
genuinely incompatible.
To address this, Zubin has been rewriting the implementation used on POSIX-like systems to use sockets instead of semaphores, which don’t suffer from this problem, and has proposed to amend the GHC Proposal accordingly.
Other work
Frontend
Sam allowed the renamer to take COMPLETE pragmas into consideration when deciding whether a
doblock should requireMonadFail(!10565), fixing a long standing feature request (#15681).Sam fixed an issue where incomplete pattern match suggestions included out-of-scope patterns (#25115, !13089).
Sam updated the GHC diagnostic code infrastructure to allow it to be extensible for other tools. This enabled another GHC contributor, Jade, to re-use that infrastructure for GHCi error messages (#23338).
Matthew fixed an issue where
-Wmissing-home-modulesgave incorrect warnings when multiple units have the same module name (!13085).Matthew extended the
-reexported-moduleflag to support module renaming (!13117), unblocking Cabal issue #10181.Zubin fixed the pretty printing of ticked prefix constructors (#24237, !13115).
Code generator
Andreas fixed a bug in the graph-colouring register allocator on AArch64 (#24941, !12990).
Rodrigo helped land work by contributor Alex Mason that improves the lowering of the byte-swap primitive in the AArch64 NCG using the
REVinstruction (!12814).
Compiler modularity
Rodrigo introduced a ‘one-shot’ strict state monad and used it to replace boilerplate monad instances in the GHC code base (!12978).
At Zurihac, Rodrigo oversaw a collaboration on the trees-that-grow infrastructure in GHC, aiming to make the Haskell AST separate from GHC in the hopes of splitting it up into a separate library. This culminated in a collaborative MR !12830 with a wide range of contributions from several new contributors.
GHCi
Andreas fixed tagging issues causing segfaults in GHCi (!12773).
Sam fixed an issue with the GHCi debugger’s treatment of record fields (#25109, !13091).
Build system and CI
Andreas investigated and fixed issues revealed by the
test-primopssuite.Matt migrated the CI away from Debian 10 and onto Debian 12 (!13033), and addressed subsequent issues with
ghcupmetadata generation (!13044, !13078).Sam updated the testsuite driver to use
py-cpuinfoto query for available CPU features, fixing feature detection on Windows (!12971).Matt added
file-ioas a boot package, to allow upgrading the version ofdirectory(!13122).