This seventh edition of our GHC activities report marks the one-year anniversary since that start of sending out these regular updates on the work on GHC and related projects that we are doing at Well-Typed. The current edition covers roughly the months of June and and July 2021.
You can find the previous editions collected under the ghc-activities-report tag.
A bit of background: One aspect of our work at Well-Typed is to support GHC and the Haskell core infrastructure. Several companies, including IOHK, Facebook, and GitHub via the Haskell Foundation, are providing us with funding to do this work. We are also working with Hasura on better debugging tools. We are very grateful on behalf of the whole Haskell community for the support these companies provide.
If you are interested in also contributing funding to ensure we can continue or even scale up this kind of work, please get in touch. If you are interested in working with us, we recently announced a hiring round.
Of course, GHC is a large community effort, and Well-Typed’s contributions are just a small part of this. This report does not aim to give an exhaustive picture of all GHC work that is ongoing, and there are many fantastic features currently being worked on that are omitted here simply because none of us are currently involved in them in any way. Furthermore, the aspects we do mention are still the work of many people. In many cases, we have just been helping with the last few steps of integration. We are immensely grateful to everyone contributing to GHC. Please keep doing so (or start)!
Currently, Ben Gamari, Andreas Klebinger, Matthew Pickering and Zubin Duggal are working primarily on GHC-related tasks. In addition, Alfredo Di Napoli has been doing some work on GHC in the last two months, next to other projects he is working on. Many others within Well-Typed, including Adam Gundry, Alp Mestanogullari, Douglas Wilson and Oleg Grenrus, contributed to GHC more occasionally.
Ben has been handling backports and release planning for the 9.2.1 and 9.0.2 releases.
Matt worked on the structure of the bindists produced by
hadrian, now they are much more like the ones produced by
make. This also fixes some other issues with the Windows packaging from the 9.0.1 release. (!6133)
Zubin has a patch in progress (!5965) that will allow the GHC library to be re-installed, so that GHC API clients (e.g. Haskell Language Server) are not restricted to the boot library versions that shipped with the compiler. This paves the way for smaller binary distributions of GHC, since users would be able to recompile the GHC library to have access to things like profiling and debug information, instead of having to ship all these configurations in the binary distribution.
Compiler error messages
Alfredo continued his work on GHC’s new diagnostic API (#18516, #19905). After completing the foundational work, he started to port existing GHC errors and warnings to the new API as well as fine-tuning the design (!6087, !6249, !6165, !6129, !5924, !5872, !5719). He also created a few newcomer-friendly tickets to help with the conversion work: these tickets have a lot of context to guide first-time GHC contributors towards their first merged MR. See for example #20118 and #20119.
Alfredo is also finalising an introductory blog post to the new GHC diagnostic API which will be published in the next few weeks.
Matt has started preliminary work on fixing a long standing bug where mixing optimisation levels would lead to optimisations not firing in some cases (#12847, #13002, #20021, #8635, #9370). With the patch (!6080), the pragmas are always read from interface files but we are careful to not look when optimisation is turned off. It turns out that using some information from interface files improves compiler performance because simpler code is produced.
Matt has continued on his crusade to refactor and modernise GHC’s driver code (!5987, !6178). This time the code that drives
--makehas been in his sights. Amongst other things the patch tries to separate the specification of the build graph from the execution of the build graph, so it is possible to describe different execution strategies. The patch also simplifies (and specifies) how module cycles are compiled which has long been a pain-point for people modifying this area.
Haddock and documentation
- Zubin rebased and improved the long pending
hiHaddock work, which should allow Haddock to generate documentation using only GHC interface (
.hi) files (!6224). This greatly simplifies Haddock’s implementation, and allows it to skip parsing, renaming and type-checking files if the appropriate information already exists in the interface files, speeding it up greatly in such cases. This also reduces Haddock’s peak memory consumption. Identifiers in Haddock comments will also be renamed by GHC itself, and the results are also serialized into
.hifiles for tooling to make use of. A number of Haddock bugs were fixed along the way (#20034, haddock 30, haddock 665, haddock 921).
GHCi and developer experience
Zubin improved GHCi completion to better support Unicode characters and operators, fixing a bug in the 9.2 pre-release, which erased the entire line the user typed if completion was triggered on an operator name (#20101, !6160).
Profiling and debugging
ghc-debugfor a test on a puzzling profile presented by a user (#20065) which seemed to have a large discrepancy between live bytes and the information reported in the profile. It turned out that the application had a severely fragmented heap, which was easy to diagnose and observe using
- Matt squashed a leak in the simplifier which should reduce maximum residency for all programs, and in particular reduced maximum residency in the test from 2GB to 1.3GB (!6202).
- Matt found a very subtle space leak caused by a reference being retained on the stack longer than necessary (!6185).
- Andreas improved register allocation performance under high register pressure (!6209).
- Ben characterised and worked to resolve a number of runtime performance regressions observed in GHC 9.0.1 (#19557, #19701, #19822, #19727, #19769, #19790).
Ben performed a refactoring of GHC’s “adjustor” mechanism used by some foreign calls, fixing a bug manifesting with some newer
libffiversions (#20051) while fixing a few nearby
libffi-related bugs (#19869).
Ben collected and characterised a number of issues manifesting on AArch64/Darwin which were ultimately found to be due to the rather peculiar ABI of that platform (#20079). He performed an audit (#20085) of Hackage packages looking for similar issues in common packages and wrote a blog post providing advice to users for writing portable, robust foreign library bindings.
Matt started looking into an old static pointers correctness issue (#16981) which a few users had commented on recently. We know what the problem is but it seems that to fix the ticket robustly a more invasive change will be needed to how static pointers are compiled.
CI and infrastructure
Matt has added support to head.hackage to run a test-suite of programs. This replaces tests in GHC’s testsuite which depended on external packages and hence were never executed during normal test runs. Now it will be straightforward to add tests with more complicated dependencies.
Ben migrated GHC build artifacts and Docker images to local storage to improve service availability.