GHC activities report: August–September 2020

This is the second edition of our new GHC activities report, which is intended to provide regular updates on the work on GHC and related projects that we are doing at Well-Typed. This edition covers roughly the months August and September 2020. The first update covered June and July 2020.

A bit of background: One aspect of our work at Well-Typed is to support GHC and the Haskell core infrastructure. Several companies, including IOHK and Facebook, are providing us with funding to do this work. We also recently announced a partnership with Hasura with a focus on working toward better debugging tools. We are very grateful on behalf of the whole Haskell community for the support these companies provide.

If you are interested in also contributing funding to ensure we can continue or even scale up this kind of work, please get in touch.

Of course, GHC is a large community effort, and Well-Typed’s contributions are just a small part of this. This report does not aim to give an exhaustive picture of all GHC work that is ongoing, and there are many fantastic features currently being worked on that are omitted here simply because none of us are currently involved in them in any way. Furthermore, the aspects we do mention are still the work of many people. In many cases, we have just been helping with the last few steps of integration. We are immensely grateful to everyone contributing to GHC. Please keep doing so (or start)!

Release management

The GHC 9.0 release is getting closer. We have been working on various aspects of release engineering, including the first alpha release in the GHC 9.0 series. (Ben)

Runtime performance and measurement

• Revisiting default value of +RTS -A (!3678) (Andreas). Over the past decade cache sizes have continued to grow and cache hierachies have deepened. To take advantage of these changes, we are examining the default allocation area size used by GHC’s storage manager. After a significant amount of benchmarking looking at a variety of configurations, we have settled on raising the default storage allocation area size from 1 MiB to 4 MiB. Not only will this improve out-of-the-box scalability of parallel programs but it also has a nice non-trivial impact on GHC compilation time of larger programs.
• Revamping nofib build system (Andreas). nofib is one of the three performance benchmarks we use to characterise GHC’s performance. However, its dated build system makes usage slower and more difficult than necessary. Andreas has been working on finishing a rewrite of this build system in Shake, enabling exploitation of parallelism (useful when running tests under cachegrind), measurement of performance monitoring unit counters, and a significantly more convenient output format.
• Bottom cost-center profiling (#18566) (Ben). While working on GHC performance issues in the past we have often encountered situations where it would be helpful to identify all call-sites of a particular function. This is enabled by a feature implemented by Ben, introducing a new compiler flag allowing users to request that cost-centers be automatically added to all call-sites of a specified function.
• There have been some improvements to the selection of baseline values for performance tests (#18484) (Ben).

Portability and infrastructure

• Ben has done some initial work bringing up GHC on MacOS on ARM in our Apple Developer Transition Kit environment.
• Fix GHC binary distribution support on Raspberry Pi OS / Raspbian (#17856) (Ben). This allows GHC binary distributions to work out of the box on the ubiquitous Raspbian Linux distribution.
• Rework of Docker images (Ben). As the number of configurations encompassed by GHC’s continuous integration infrastructure has grown, various inconsistencies have crept into the Docker environments in which these builds run. To eliminate these we have consolidated the Dockerfiles responsible for these environments, generating them via Dhall.
• Updating Windows toolchain (Ben). GHC provides its own patched gcc toolchain on Windows to work-around various platform limitations (e.g. the MAX_PATH limit). Before every release we update this toolchain and fix the breakage (#18774) that often results.
• Test Hadrian via Stack (Ben). As part of the preparation to switch over to Hadrian as GHC’s primary build system, we want to raise assurances that users will be able to compile the build system. As the Stack build has broken numerous times in the past (e.g. #18726), we now test this via CI.
• Ben has resolved the last failing tests on Windows, allowing Windows to be marked as a mandatory CI-green platform.

Bug fixes

• Debugging Windows stack corruption and refactoring of initializers (#18548) (Ben). Fix GHC’s implementation of foreign function exports to avoid calls into libc during object loading. This fixes a stack corruption bug on Windows.
• Refactoring of object merging (Ben). GHCi’s linker tends to be quite slow when loading objects compiled with function sections enabled. For this reason, GHC also produces “GHCi objects” where function sections have been merged. However, this feature has introduced a number of unfortunate toolchain interactions (e.g. #17962, #18550). To avoid these we have refactored GHC’s object merging logic to ensure that we only pass self-consistent flag-sets to the linker.
• Further refinement of simplification around runRW# (#15127, #18291) (Ben).
• Refactoring of RebindableSyntax (#17582) (Alp). RebindableSyntax is a source of significant implementation complexity within GHC and has consequently been a significant source of bugs. Alp has been working on refactoring much of this complexity away by introducing a way to typecheck some desugared code while reporting the original code in error message. There is a working reimplementation of “rebindable if” that uses the new mechanism, as well as an almost complete reimplementation of “rebindable monad operations”.
• Implement unloading support for dynamic objects, fix unloading support for static objects (#16525) (Ben). GHCi has long supported unloading of static objects. However, there were known cases where such unloading could result in unsound garbage collection. Furthermore, dynamic object unloading was not supported at all. Ben has refactored the linker’s code unloading support, eliminating the potential for unsoundness and implementing unloading of dynamic objects.
• Diagnosing non-moving GC bug (#18587) (Ben). Currently, we are working with an issue reporter to diagnose apparent unsoundness in a STM-heavy parallel application when run with the non-moving garbage collector. This effort has already resulted in the fixing of a missing barrier in the TVar mutation codepath, although there is apparently more to be done here.

Debugging tools

Now that Well-Typed has partnered up with Hasura, we (primarily David) have started working toward better debugging tools. This effort is currently focused on the ghc-debug library which allows connecting to a remote Haskell process, pausing it, making queries to the remote heap, then resuming it again. To achieve this, we first need support from the runtime system. We’ve picked up work on GHC MR !1435 to implement the relevant API changes. So far we’ve made sure that the exposed API is simple, thread safe, and that correct usage is documented. We’re also working on the ghc-heap part of the MR which enables decoding remote heap objects without sending the garbage collector into a segfault.

GHC proposals

Decorate exceptions with backtrace information

We want to ensure that exceptions report provenance information by default without requiring action on the part of the thrower by leveraging the existing mechanisms for collecting backtraces (HasCallStack, GHC.Stack.CCS, and DWARF debug information). Furthermore, we want to ensure that this information is available for consumption in structured form by the user program, to allow use by logging libraries. (Ben, David)

Simplify DuplicateRecordFields

This proposal addresses an unsatisfactory aspect of DuplicateRecordFields, namely the unclear rules around when a field selector or update will be accepted, by entirely removing the type-directed name resolution aspect. This proposal is more restrictive than a previous (dormant) simplification proposal, allowing fewer programs, but correspondingly simpler. (Adam)