GHC activities report: August–September 2023

Adam Gundry, Andreas Klebinger, Ben Gamari, Finley McIlwaine, Jaro Reinders, Matthew Pickering, Sam Derbyshire, Zubin Duggal – Thursday, 12 October 2023

all community ghc ghc-activities-report well-typed  

This is the twentieth edition of our GHC activities report, which describes the work on GHC and related projects that we are doing at Well-Typed. The current edition covers roughly the months of August and September 2023. You can find the previous editions collected under the ghc-activities-report tag.

Many thanks to our sponsors who make this work possible: Anduril, Hasura and Juspay. In addition, we are grateful to Mercury for funding specific work on improved performance for GHC, HLS and related projects.

However, we need more sponsorship to sustain the team! If your company might be able to contribute funding to sustain this work, please read about how you can help or get in touch.

Of course, GHC is a large community effort, and Well-Typed’s contributions are just a small part of this. This report does not aim to give an exhaustive picture of all GHC work that is ongoing, and there are many fantastic features currently being worked on that are omitted here simply because none of us are currently involved in them. Furthermore, the aspects we do mention are still the work of many people. In many cases, we have just been helping with the last few steps of integration. We are immensely grateful to everyone contributing to GHC!

Team

The GHC team at Well-Typed consists of Ben Gamari, Andreas Klebinger, Matthew Pickering, Zubin Duggal, Sam Derbyshire and Finley McIlwaine. In addition, Jaro Reinders recently completed an internship with us. Many others within Well-Typed are contributing to GHC more occasionally.

Releases

Frontend

Typechecker & renamer

  • Sam fixed incorrect scoping behaviour with ImplicitParams that was the result of a refactor of the constraint solver (!11013).

  • Sam removed the pattern synonym error recovery mechanism which allowed GHC to limp along after a pattern synonym failed to typecheck: we simply fail eagerly instead, as this avoids unpredictable knock-on consequences (#23467, !11114).

  • Sam fixed a long-standing bug in the processing of bundled items in export lists (#23570, !11107).

  • Sam reviewed, debugged and finished !10860 by GHC contributor Antoine Leblanc. This work prevents inconsistencies between .hs and .hs-boot files from causing compiler panics.

  • Sam ensured that we always use unsatisfiable to fill-in missing class methods of a class instance with an Unsatisfiable context, even when there are default methods available, to avoid accidental runtime loops under -fdefer-type-errors (!11075).

Error messages and warnings

  • Sam improved the logic and the error messages surrounding unused type variables in type and data family instance declarations, fixing several issues (#23768, #23778, #23734, #23784).

  • Matthew added the -Winconsistent-flags warning flag to control the warning that triggers when inconsistent command line flags are passed to GHC. This allows users to silence the warning when desired (!11213).

  • Sam added functionality for querying all the diagnostic codes that a given GHC version emits by way of new Hadrian targets codes and codes:outdated (!11079).

Optimisations

  • Ben ensured that functions with a SPEC argument are always specialised regardless of the sc_max_args limit (#14003, !11225).

  • Andreas identified an issue whereby using the magic noinline function would hamper strictness analysis (#23911).

Driver

  • Matthew ensured that we check the transitive closure of the dependencies when deciding whether to re-link (#23724, !11178).

  • Matthew reverted a long-incorrect patch which made -optP unusable as the options were always passed to a C compiler (#16737).

Backend

Runtime system

  • Ben ensured that the IO manager shuts down reliably on Windows (!10905).

  • Ben refactored concurrent marking in the non-moving garbage collector, avoiding creating a new thread with every major GC cycle and thereby improving memory usage due to stacks (!11048).

  • Ben fixed missing atomic operation symbols errors on AArch64 (!11172).

  • Ben investigated an issue reporting a possible memory leak in GHC 9.2 (#23949).

Primops

  • Ben added missing acquire barriers to array read instructions which are necessary to ensure soundness on platforms with weak memory ordering (#23541, !10953).

  • Ben simplified the implementation of atomicModifyMutVar2#, eliminating a redundant load in the non-threaded RTS (!11005).

Code generation

  • Andreas ensured the AArch64 code generator doesn’t emit overly large conditional jump instructions (#23746, !11254).

  • Ben reworked the handling of registers when using ThreadSanitizer, making ThreadSanitizer instrumentation usable on large programs. He has nearly finished merging the changes from his long-running ThreadSanitizer work (!10203).

  • Ben improved the documentation describing the memory-ordering properties of thunk update (!11007).

  • Ben changed how the stack alignment flag is passed to LLVM (#23870, !11124).

  • Ben fixed the AArch64 code generator’s treatment of sub-word-sized multiplication and improved coverage of these operations in the test-primops testsuite (#23721).

  • Ben started debugging a mysterious runtime crash (#23952).

Linking

  • Ben ensured the Windows linker interprets PE section numbers as unsigned (#22941, !10959).

  • Ben characterised the performance of dynamic linking on Darwin, prompted by user reports of poor code-loading performance (#23415).

Profiling

  • Andreas fixed a JSON character escape bug that was causing issues in profiling reports (#23924).

  • Finley implemented a change that will allow users to refine the set of constructors that -fdistinct-constructor-tables applies to, which can significantly reduce the sizes of binaries containing info-table provenance information (!10958).

  • Finley implemented the -fdistinct-constructor-tables-per-module flag, which can be used to obtain heap profiles containing per-module allocation stats for data constructors (!11068).

  • Finley added the -f{no-}info-table-map-with-{stack,fallback} flags, which allow users to control whether info tables for stack closures or info tables with defaulted source locations are included in the info table map (!10926).

  • Finley fixed a bug that caused the IPE information of some unconstrained instance dictionaries to be missing source locations (!11242).

  • Finley improved compiler performance when compiling with -finfo-table-map (!11023).

Libraries

  • Ben introduced two new libraries, ghc-internal and ghc-experimental. These were introduced as part of the Haskell Foundation tech proposal 51 in order to help stabilise the exposed interface of the base library.

  • Ben refactored the Exception Backtrace ghc-proposal 330 into distinct subproposals for consideration by the Core Libraries Committee under CLC proposal 164.

  • Ben opened CLC proposal 209 to add support for the EPOLLRDHUP event to the epoll backend of GHC’s event manager, and wired support for this event into wai. This will enable server applications to cancel on-going work when a client closes the receive side of its connection.

  • Jaro submitted CLC proposal 187 about using native operations to speed up Enum Word64 and Enum Int64 on 32-bit platforms. The implementation of the proposal (!10825) can improve performance by 1.5x on i386, and 5.6x with the JavaScript backend.

  • Ben documented that the readFloat function takes a runtime that is proportional to the magnitude of the input, and thus should be avoided on untrusted inputs (!11246).

  • Matthew tested the performance of 9.8 pre-releases on the benchmark suites of bytestring, text and containers, identifying a single regression (#23822).

Build system, CI and distribution

GHC build system

  • Matthew added a new reloc-binary-dist target which allows one to create relocatable binary distributions for GHC. This is especially useful on Windows, as the previous mechanism for creating a binary distribution was known to be broken.

  • Matthew implemented many fixes to ghc-toolchain to ensure that it produces compatible results with the configure script (!10976), fixing several issues (#23720, #23689, #23681, #23676).

  • Matthew has made good progress abstracting Hadrian to allow building stage 2 cross compilers. This brings cross-compilers into line with how we build normal compilers, and also allows us to build a stage 3 compiler for new architectures (#19174).

  • Ben added a check for C99 support to ghc-toolchain (!11131).

CI & testing

  • Matthew implemented several fixes to CI (!11089). In particular, this improved how labels attached to GHC MRs trigger additional pipelines; for example, one can specify to run a job only when either the full-ci label or the test-primops label is present.

  • Matthew added Debian 12 and Fedora 38 builds to CI.

  • Ben enabled test-primops to be run on 32-bit platforms.

Packaging

  • Matthew added aarch64-alpine and aarch64-deb11 bindists (!10594).

  • Matthew fixed a bug in which we were installing two copies of the same manpage (!11085).

  • Ben ensured LICENSE files are properly copied into binary distributions (!11294).

  • Matthew bumped MAXOSX_DEPLOYMENT_TARGET to 10.13 (#22938, !11210).

  • Zubin ensured that python files are correctly tracked as dependencies of the Hadrian docs target (!11072).

  • Ben restored compatibility of the users’ guide with older versions of Sphinx (!11073).

  • Ben fixed broken :base-ref: references in the users’ guide (!11337).

  • Zubin investigated a GHC build failure due to interactions between a bytestring submodule bump and RTS header files (#23789, !11088).