GHC activities report: September–November 2024

Adam Gundry, Andreas Klebinger, Ben Gamari, Hannes Siebenhandl, Matthew Pickering, Mikolaj Konarski, Rodrigo Mesquita, Sam Derbyshire, Zubin Duggal – Friday, 13 December 2024

all community ghc ghc-activities-report well-typed  

This is the twenty-fifth edition of our GHC activities report, which describes the work Well-Typed are doing on GHC, Cabal, HLS and other parts of the core Haskell toolchain. The current edition covers roughly the months of September to November 2024. You can find the previous editions collected under the ghc-activities-report tag.

Sponsorship

We are delighted to offer Haskell Ecosystem Support Packages to provide commercial users with access to Well-Typed’s experts, while investing in the Haskell community and its technical ecosystem. Clients will both fund the work described in this report and support the Haskell Foundation. If your company is using Haskell, read more about our offer, or get in touch with us today, so we can help you get the most out of the toolchain. We need more funding to continue our essential maintenance work!

Many thanks to our existing sponsors who make this work possible: Anduril and Juspay. In addition, we are grateful to Mercury for funding specific work on improved performance for developer tools on large codebases.

Team

The GHC team at Well-Typed currently consists of Andreas Klebinger, Ben Gamari, Matthew Pickering, Rodrigo Mesquita, Sam Derbyshire and Zubin Duggal. Adam Gundry acts as Secretary to the GHC Steering Committee. Cabal maintenance is undertaken by Mikolaj Konarski, and HLS maintenance by Hannes Siebenhandl and Zubin Duggal. In addition, many others within Well-Typed are contributing to GHC more occasionally.

GHC Releases

Cabal

Cabal-3.14.0.0 was released in September, adding initial support for the new hooks build-type we implemented to replace custom setup scripts, as part of our work for the Sovereign Tech Fund on Cabal long-term maintainability. Corresponding new versions of cabal-install and the release of the Cabal-hooks library are due soon, at which point it will become easier for users to explore replacing their custom setup scripts with the hooks feature. Related to this effort:

  • Sam amended the Cabal-hooks version to match the Cabal library version (#10579).

  • Rodrigo made progress on Sam’s work to simplify the way cabal-install uses the Cabal library to build packages. This will make the code easier to work with in the future, and should improve build performance (#9871).

  • Rodrigo made cabal-install invoke git clone concurrently when downloading from git repositories for source-repository-package stanzas or cabal get, and switched it to use shallow clones by default (#10254). This speeds up the cloning step significantly.

  • Rodrigo’s work on private dependencies (#9743) is slowly making progress, thanks to recent work by Kristen Kozak.

  • Rodrigo and Matthew fixed various minor Cabal bugs, improved documentation and future-proofed against future core libraries changes (#10311, #10404, #10415 and #10433).

HLS

  • Hannes finished off and merged support for a new “jump to instance definition” feature in HLS (#4392), which will make it easier for users to understand which typeclass instance is in use in a particular expression.

GHC

Exception backtraces

Rodrigo worked on improving several facets of the exception backtraces story:

  • Improved the rendering of uncaught exceptions so the default output is much clearer and easier to understand (CLC proposal #285, !13301). This included reformatting the output, reducing duplication, avoiding exposing internal implementation details in the call stacks, and reducing duplication.

  • Changed functions such as catch to propagate the original cause if another exception is subsequently thrown (CLC proposal #202.

  • Landed a patch by Ben so that the HasCallStack-based backtrace for an error call is more informative (!12620, #24807).

Overall, exception backtraces will be much more useful in GHC 9.12 and later, making it easier to debug Haskell applications.

Frontend

  • Andreas added new primops, is[Mutable]ByteArrayWeaklyPinned#, which allow checking whether a bytearray can be moved by the RTS, as per CLC proposal #283.

  • Sam fixed a GHC panic involving out-of scope pattern synonyms (#25056, !13092).

  • Sam augmented the -fdiagnostics-as-json output to include the reason an error or warning was emitted (!13577, #25403).

  • Matthew and Rodrigo deprecated the unused -Wcompat-unqualified-imports warning (!12755, #24904, !13349, #25330).

  • Ben improved the parsing and parser errors for sizes in GHC RTS flags (!12384, #20201).

  • Matthew fixed a bug in the interaction of -working-dir and foreign files (!13196, #25150).

  • Zubin bumped the Haddock binary interface version to 46, to improve errors when there are mismatched interface files (!13342).

SIMD in the NCG backend

  • Sam and Andreas teamed up to finish the mega-MR adding SIMD support to GHC’s X86 native code generator backend (!12860, and see our previous report for more background). This also fixed critical correctness bugs that affected SIMD support in the existing LLVM backend, such as #25062 and #25169.

  • Sam followed this up with user’s guide documentation for the feature (!13880) and a couple of additional fixes for bugs that have been reported since (!13561, !13612).

LLVM backend

  • Sam implemented several fixes relating to the LLVM backend, in collaboration with GHC contributor @aratamizuki:

    • fix bugs involving fltused to ensure that GHC can use the LLVM backend on Windows once more (#22487, !13183),
    • use +sse4.2 instead of +sse42 (#25019),
    • make SSE4.2 imply +popcnt (#25353).

  • Matthew bumped the LLVM upper bound to allow GHC to use LLVM 19 (!13311, #25295).

RISC-V backend

  • Andreas added support for floating-point min/max operations in the RISC-V NCG backend (!13325)

  • Matthew fixed some issues to do with the fact that the RISC-V backend does not yet support SIMD vectors (!13327, #25314, #13327).

Object code determinism

  • Rodrigo merged !12680, which goes 95% of the way towards ensuring GHC produces fully deterministic object code (#12935).

  • Rodrigo made the unique generation used by the LLVM backend deterministic (!13307, #25274), thus making GHC 96% object-code deterministic.

  • Rodrigo ensured that re-exports did not spoil determinism of interface files in !13316 (#25304).

Compiler performance

  • Matthew, Rodrigo and Adam published a proposal for Explicit Level Imports. The proposed language feature will allow users of Template Haskell to communicate more precise dependencies for quotes and splices, which can unlock significant compile-time performance improvements and is a step towards better cross-compilation support.

  • Rodrigo improved the performance of module reachability queries (!13593), which can significantly reduce compile times and memory usage for projects with very large numbers of modules.

  • Andreas introduced a new flag, -fmax-forced-spec-args, to control the maximum size of specialised functions introduced when using the SPEC keyword (!13184, #25197). This avoids a potential compile-time performance cliff caused by specialisations with excessively large numbers of arguments.

  • Matthew greatly reduced the memory footprint of linking with the Javascript backend by making some parts of the compiler more lazy (!13346).

  • Zubin’s proposal to address libc compatibility issues when using semaphores for build parallelism has been making progress through the proposal process.

Runtime system

  • Ben fixed the encoding of breakpoint instructions in the RTS to account for a recent addition of support for inlining breakpoints (!13423, #25374).

  • Ben tightened up the documentation and invariants in the bytecode interpreter (!13565).

  • Ben allowed GNU-style non-executable stack notes to be used on FreeBSD (!13587, #25475).

  • Ben fixed an incorrect EINTR check in the timerfd ticker (!13588, #25477).

  • Zubin ensured that any new cost centres added by freshly-loaded objects are correctly included in the eventlog (!13114, #24148).

  • Ben increased the gen_workspace alignment in the RTS from 64 bytes to 128 bytes in order to prevent false sharing on Apple’s ARMv8 implementation, which uses a cache-line size of 128 bytes (!13594, #25459).

  • Ben removed some incorrect platform-dependent pointer casts in the RTS (!13597).

  • Zubin fixed a segfault when using the non-moving GC with profiling (!13271, #25232).

  • Ben fixed a stack overrun error with i386 adjustors (!13599, #25485).

  • Ben introduced a convenience printIPE debugging function for printing info-provenance table entries (!13614).

  • Andreas fixed a crash that could happen if an exception and the compacting GC aligned in specific ways (#24791, !13640).

Documentation

  • Ben fleshed out missing documentation of the eventlog format in the user’s guide (!13398, #25296).

  • Ben documented the :where GHCi command (!13399, #24509).

  • Andreas clarified the documentation of fexpose-overloaded-unfoldings (!13286, #24844).

  • Ben documented that GHC coalesces adjacent sub-word size fields of data constructors (!13397).

  • Ben improved the documentation of equality constraints to mention which language extensions are required to use them (!12395, #24127).

Codebase improvements

  • Ben fixed a few warnings in the runtime system, including some FreeBSD specific warnings (!13586).

  • Ben fixed (!13394, #25362) some incomplete pattern matches in GHC.Internal.IO.Windows.Handle that came to light in !13308, a refactor of the desugarer. Andreas also chipped in, squashing some warnings in ghc-heap (!13510).

  • Matthew refactored the partitionByWorkerSize function to avoid spurious pattern-match warning bugs when compiling with -g3 (!13359, #25338).

  • Matthew removed the hs-boot file for Language.Haskell.Syntax.ImpExp and introduced one for GHC.Hs.Doc, which better reflects the intended modular hierarchy of the modules (!13406).

GHC API

  • We are pleased to see that the Haskell Foundation and Tweag are resuming efforts aimed at defining a stable API for GHC.

  • Ben added lookupTHName, a more convenient way to look up a name in a GHC plugin that re-uses Template Haskell lookup functions (!12432, #24741).

  • Zubin made sure that the driverPlugin is correctly run for static plugins (!13199, #25217).

Libraries

  • Andreas allowed unknown FD device types in the setNonBlockingMode function (!13204, #25199), as per CLC proposal #282. This fixes a regression in the hinotify package on GHC 9.10.

  • Andreas made sure that all primops are re-exported in the ghc-experimental package via the module GHC.PrimOps (!13245), and changed the versioning scheme of ghc-experimental to follow GHC versions (!13344, #25289).

  • Ben fixed a performance regression in throw by judicious insertion of noinline (!13275, #25066), as discussed in CLC proposal #290.

  • Matthew unwired the base package to pave the way for a reinstallable base package (!13200).

  • Matthew upgraded GHC’s Unicode support to Unicode 16 (!13514).

  • Rodrigo removed BCO primops from the GHC.Exts re-export, as per CLC proposal #212 (!13211, #25110).

Profiling

  • Matthew enabled late cost-centres by default when the libraries distributed with GHC are built for profiling (!10930, #21732). This greatly improves the resolution of cost centre stacks when profiling.

  • Andreas fixed a bug in which profiling could affect program behaviour by allowing profiling ticks to move past unsafeCoerce# in Core (!13413, #25212).

Build system

  • Ben fixed the configure script incorrectly reporting that subsections-via-symbols is enabled on AArch64/Darwin (!12834, #24962).

  • The first alpha pre-release of 9.12 incorrectly had a version number of 9.12.20241014 instead of 9.12.0.20241014, which broke the expected lexicographic ordering of GHC releases. Ben added a check for the validity of a release GHC version number to prevent such issues in the future (!13456).

  • Matthew allowed GHC to build with happy-2.0.2 (!13318, #25276), and Ben with happy-2.1.2 (!13532, #25438).

  • Andreas made the Hadrian progress messages including the working directory to allow quickly distinguishing builds when multiple builds are in progress (!13353, #25335).

  • Ben allowed Haddock options to be passed as Hadrian key-value settings (!11006).

Testsuite

  • Ben improved the reporting of certain errors in the testsuite (!13332).

  • Ben ensured performance metrics are collected even when there are untracked files (!13579, #25471).

  • Ben ensured performance metrics are properly pushed for WebAssembly jobs (!13312).

  • Zubin made several improvements to the testsuite, in particular regarding normalisation of tests and fixing a Haddock bug involving files with the same modification time (!13418, !13522).

CI

  • Matthew added a i386 validation job that can be triggered by adding the i386 label on an MR (!13352).

  • Matthew added a mechanism for only triggering certain individual jobs in CI by using the ONLY_JOBS variable (!13350).

  • Matthew fixed some issues of variable inheritance in the ghcup-metadata testing job (!13306).

  • Matthew added Ubuntu 22.04 jobs to CI (!13335).