GHC activities report: December 2022–January 2023

Ben Gamari, Andreas Klebinger, Matthew Pickering, Zubin Duggal, Sam Derbyshire – Tuesday, 28 February 2023

all community ghc ghc-activities-report well-typed  

We’re happy to support the Haskell Foundation in planning a pre-ZuriHac workshop for new GHC contributors on 7th-9th June 2023 in Zurich (and online). Find out more and express interest here.

This is the sixteenth edition of our GHC activities report, which describes the work on GHC and related projects that we are doing at Well-Typed. The current edition covers roughly the months of December 2022 and January 2023. You can find the previous editions collected under the ghc-activities-report tag.

We are delighted that Juspay are now sponsoring this work. Many thanks to them and to our other sponsors, including GitHub via the Haskell Foundation, and IOG. We are also working with Hasura and Mercury on specific GHC-related improvements, such as interface files with core definitions.

However, some of our sponsorship agreements are coming to an end and we need more sponsorship to sustain the team! If your company might be able to contribute funding to ensure that we can continue this valuable work, please read about how you can help or get in touch.

Of course, GHC is a large community effort, and Well-Typed’s contributions are just a small part of this. This report does not aim to give an exhaustive picture of all GHC work that is ongoing, and there are many fantastic features currently being worked on that are omitted here simply because none of us are currently involved in them in any way. Furthermore, the aspects we do mention are still the work of many people. In many cases, we have just been helping with the last few steps of integration. We are immensely grateful to everyone contributing to GHC!

Team

The existing GHC team consists of Ben Gamari, Andreas Klebinger, Matthew Pickering, Zubin Duggal and Sam Derbyshire. Many others within Well-Typed are contributing to GHC more occasionally.

We are very pleased to welcome Rodrigo Mesquita to the team for an internship over the next six months. Rodrigo will be working on enabling the safe reinstallation of the ghc and template-haskell libraries (#20742). This is a crucial step which will unlock better cross-compilation, more robust tooling support, and potential improvements in library compatibility across GHC versions.

Releases

  • Ben and the team have been busy working on the 9.6 release (#22562). Several alphas have been released and a release candidate is expected soon. Sam worked on the 9.6 migration guide.

  • Matt and Ben released version 9.4.4 in December, fixing various bugs in the 9.4 series.

  • Zubin prepared version 9.2.6 for release in February.

Driver

  • Matt finished off and landed a patch from a contributor reworking plugin initialisation points, so that plugins can install hooks (#21279, !9824).

  • Matt fixed a variety of bugs in multiple home unit support which were found when testing new Cabal support for starting multi-unit sessions (!9618).

Compiler performance

  • Matt and Zubin investigated GHCi memory usage on partial reloads. They found and squashed quite a few leaks, resulting in substantial reductions in memory usage after a :reload (!9787).

  • Matt fixed a bug when using interface files with core definitions (!9869).

Runtime performance

  • Ben fixed an unexpected regression in MutVar# performance due to a memory fence (#22764, !9718).

  • Andreas investigated many runtime performance issues relating to inlining, specialisation, INLINEABLE and NOINLINE pragmas (#22321, #22609, #22689, #22781, !9723). He has work in progress allowing NOINLINE and INLINEABLE to be combined, enabling specialization of NOINLINE functions (!9621).

  • Andreas opened a GHC proposal with some ideas for transitive specialization.

  • Andreas investigated performance regressions in aeson (#22115) which were caused by three independent issues:

    • A code generation regression which was then quickly fixed by Ben (!9718).
    • A change to the implementation of Bits instances which caused regressions due to partial register stalls. This is a known issue (#20405) where the fix had not been applied to all branches at the time.
    • A regression due to changes to the constructed product result optimisation (#22983).

  • Andreas diagnosed a runtime performance regression in Data.OldList.{maximumBy,minimumBy} in 9.2.5 (#22609, !9565).

Frontend

  • Andreas diagnosed an issue where an OPAQUE pragma triggered a compiler panic (#22502).

  • Sam proposed a patch adding Template Haskell support for explicit forall quantifiers in instances (!9840).

  • Matt fixed the module locations of some definitions in GHC.Types for Typeable (#22510, !9459).

Error messages

  • Matt improved the error messages of the exhaustiveness checker when checking statements that have been moved around with ApplicativeDo (#22483, !9460).

Runtime system

  • Ben investigated a recurring runtime crash on MacOS Ventura (#22497), which turned out to be an upstream bug in the GMP library, necessitating backports to the 9.2, 9.4 and 9.6 branches.

  • Ben began looking at improving the linkage story for RTS symbols, which is a long-standing headache for distribution packagers (#17157).

  • Ben fixed warning-cleanliness in the runtime system’s C sources and fixed Hadrian to ensure that they remain clean (!9579)

Garbage collector

  • Andreas fixed an issue with GCing of sparks (#22528) that sometimes caused sparks to be evaluated sequentially instead of in parallel.

  • Andreas investigated and fixed a crash when GCing a CAF that is evaluated from TH (#22417). This fixes some long standing spurious segfaults involving TH on static build targets such as Windows.

  • Ben made significant improvements to the non-moving garbage collector:

    • Fixed the handling of empty arrays in the non-moving write barrier (#22931).
    • Greatly improved handling of Weak# objects, which previously may not have been promptly finalized (#22327).
    • Fixed a few potentially unsound interactions with setNumCapabilities (#22926, #22927).
    • Identified and fixed a missing write barrier in the GC’s selector optimisation implementation (#22930).
    • Significantly improved pause times for some programs by bounding marking work in the sync pause (#22929).
    • Continued work in implementing parallel marking support.
    • Improved accuracy of residency reporting.

  • While working on the GC, Ben also identified and fixed a number of data races in the RTS thanks to ThreadSanitizer instrumentation (!6232, !9371, !9372, !9482).

Code generation

  • Sam improved how GHC keeps track of the types at which Cmm registers are used. In particular, this helps ensure that SIMD vector registers, which can be used to store different types of data, are used in a consistent manner (!9167).

  • Ben diagnosed and fixed an AArch64 backend issue resulting in compile-time crashes when the graph-colouring register allocator (-fregs-graph) is in use (#22798).

  • Andreas fixed a runtime crash caused by a bug in the tag inference analysis due to a difference between Cmm and Bytecode backends (#22840).

  • Andreas avoided unboxing large constructors containing unboxed tuples into other constructors by mistake (#22309, !9724). They can still be explicitly unboxed if desired.

  • Ben fixed -fsplit-sections on Windows (#22834).

  • Ben fixed a linking-related compilation failure on MacOS (#21972).

Testsuite

  • Andreas updated the nofib benchmark suite to ensure it is usable with the upcoming 9.6 release, along with some other minor tweaks.

  • Ben improved the reporting output of the nofib-compare utility.

  • Matt added tests for issues milestoned for 9.4.4 but already fixed.

  • Ben found that the testsuite was failing with -threaded and sanity-checking enabled, and started fixing these issues.

Packaging

  • Matt fixed an issue with manpages not being installed.

  • Matt made various fixes to the build system and packaging including enabling split-sections for release builds.

  • Matt added a flag to omit self-recompilation info from interface files, for situations where you are never going to attempt to recompile a package. This avoids a source of non-determinism in builds.

  • Ben bumped the Windows toolchain shipped with GHC (#21293, !9683).

  • Matt added support for LLVM 15 and while doing so removed support for LLVM 10. GHC will henceforth support LLVM 11 through 15.

  • Zubin fixed an issue with the binary distribution configure script where it didn’t check for the find command being available (!9777).

  • Matt added scripts for automatically generating metadata for ghcup. When a new release is made, the metadata is sent to the ghcup maintainers who can easily use it to add support for new releases to ghcup. The metadata is generated on nightly and release pipelines and is now used to test binary distributions on many different platforms.

CI

  • Matt upgraded CI runners to use GHC 9.4.3 for CI.

  • Matt fixed a variety of issues on Darwin CI runners, so they could be re-enabled, increasing CI capacity.

  • Matt added Ubuntu 18.04 nightly and release jobs.

  • Ben added a Rocky Linux CI job, filling the hole left by the retirement of Centos 7.

  • Ben prototyped infrastructure for collecting hardware performance counter metrics in CI to catch code generation regressions more reliably (!7414).

head.hackage

ghc-debug

  • Matt added support for tracing of static reference tables (SRTs) to ghc-debug.

Supporting users/contributors

  • Matt looked into a reported issue with a runtime loop, and diagnosed it as expected behaviour involving an interaction with overlapping instances specific to the project.

  • Ben reproduced and characterised a linking issue affecting a user (#22803).

  • Sam reviewed and assisted with an MR by Georgi Lyubenov that allows record wildcards to be used with nullary constructors (!9004).

  • Sam reviewed several MRs by Andrew Lelechenko which eliminate uses of head and tail within GHC’s codebase.

  • Ben fixed many other tickets (#22527, #22538, #22561, #22631, #22714, #22322, #22777, #22640, #22595, #22566).

Stability

  • Ben wrote and shepherded a Core Libraries Committee proposal adding the new Foreign.C.ConstPtr.ConstPtr type, facilitating resolution of #22043.

  • Ben began assessing the stability of the many interfaces provided by base’s GHC.* module namespace as a first step towards making a clearer separation between the public and GHC-internal APIs of base.

  • Ben introduced CI support for tracking the interfaces exposed by base to help ensure that core libraries changes receive the review that they deserve (!9816).

  • Sam made a CLC proposal concerning the reporting of exceptions arising during weak pointer finalization. Until agreement on this issue is reached, GHC’s error reporting has been reverted to the status quo, with users free to change it as they see fit.