GHC activities report: December–January 2020/2021

Ben Gamari, Andreas Klebinger, David Eichmann, Adam Gundry, Alfredo Di Napoli, Matthew Pickering – Wednesday, 10 February 2021

all community ghc ghc-activities-report  

This is the fourth edition of our GHC activities report, which is intended to provide regular updates on the work on GHC and related projects that we are doing at Well-Typed. This edition covers roughly the months of December 2020 and Janary 2021.

The previous editions are here:

A bit of background: One aspect of our work at Well-Typed is to support GHC and the Haskell core infrastructure. Several companies, including IOHK and Facebook, are providing us with funding to do this work. We also recently announced a partnership with Hasura with a focus on working toward better debugging tools. We are very grateful on behalf of the whole Haskell community for the support these companies provide.

If you are interested in also contributing funding to ensure we can continue or even scale up this kind of work, please get in touch.

Of course, GHC is a large community effort, and Well-Typed’s contributions are just a small part of this. This report does not aim to give an exhaustive picture of all GHC work that is ongoing, and there are many fantastic features currently being worked on that are omitted here simply because none of us are currently involved in them in any way. Furthermore, the aspects we do mention are still the work of many people. In many cases, we have just been helping with the last few steps of integration. We are immensely grateful to everyone contributing to GHC. Please keep doing so (or start)!

Release management

  • Ben Gamari put the finishing touches on GHC 9, planning for the release in early Februrary 2021. Meanwhile, he worked to fix a few remaining regressions (#19149, #19155) in preparation for a 8.10.4 release to hopefully close the 8.10 series.

  • Ben finished a major rebase of GHC’s haddock branch, allowing changes made upstream to be shipped with GHC 9.0 and future releases.

  • Ben started planning for the GHC 9.2 release cycle and started work on bumping GHC’s various core library dependencies to their current upstream versions.

Profiling and Debugging

  • Matthew Pickering has been picking up tickets related to the heap profiler in order to improve its usability. This includes implementing the ability for heap profiling to be controlled dynamically from within a program (!4570), user-interface polish, and continued work on info-table provenance profiling, as described in this blog post.

  • Ben has been working on the Ticky-Ticky profiler, introducing support and tooling for recording of counter samples in GHC’s eventlog (!3085). This allows for characterisation of the dynamic behavior of the program and has already proven useful in identifying performance bugs in the compiler. Some of the work will be described in a forthcoming blog post.

  • Ben implemented profiling support for pinned allocations, allowing proper cost-centers to be reported for such closures and fixing a long-standing blind-spot of the profiler (#7275).

Compiler Performance

  • Using his eventlog-enabled ticky-ticky work, Ben identified a number of compiler performance bugs in the compiler. While each change was individually small, overall they improved compile time of the Cabal library by over two seconds.

  • Ben further improved (!4823) GHC’s treatment of nullary type constructor applications, resulting in nearly 2% improvement of compile-time of the Cabal library.

  • A number of small memory-usage optimisations have been identified as a result of Matt applying ghc-debug to GHC. See #17292, #19190, #19194 for examples where opportunities for increased sharing were identified and #19156 and #19201 for some thunk accumulation problems.

  • Ben observed that call-arity analysis can take an undue portion of compilation time in some workloads (#18789). On further investigation he pinned the problem on the poor asymptotic behavior of the data structure used to track co-calls. A rework of the data structure reduced runtime of the call-arity pass from over 50 seconds to less than 2 seconds. In addition, a number of smaller constant-factor optimisations were performed.

  • A user identified the CmmSink pass as a compiler performance bottleneck in certain edge cases. Andreas Klebinger made some simple improvements (!4534) to the pass to improve compiler performance for this edge case. These resulted in a reduction in compiler allocations by a few percent. However the underlying algorithm is unchanged and remains quadratic (with a small constant). A ticket (#19012) now exists which both describes the remaining issue and discusses approaches to potential fixes this.

  • When dumping GHC’s intermediate passes a large amount of the output tends to be consecutive whitespace. Andreas wrote up a patch (!4508) to speed up this case. For some cases this improves compiler allocations by >= 4% when dumping output.

  • In certain situations (#18730), GHC ends up doing exponential inlining work. Andreas suggested we might avoid this by keeping track of the depth of an expression. SPJ created a proof-of-concept (!4617) for the idea which was then finished up by Andreas. This patch puts a bound on the work caused by inlining, greatly improving compiler performance for certain cases.

  • Ben and Andreas worked to improve code generation for derived instances of the Eq typeclass (#17241), eliminating a significant amount of generated code in favor of using the dataToTag# primop.

Runtime performance

  • Andreas worked to finish up documentation of his tag inference analysis, which allows GHC to elide tag check code when a closure is known to be evaluated (#16970).

  • Ben finished work to improve code generation for the dataToTag# primop (!2915), allowing it to use the pointer tag instead of dereferencing the info table.

  • Ben worked with Domen Kožar from Cachix to help better understand the pause-time behavior of his workload running under the non-moving garbage collector and implemented some additional instrumentation (!4631) to make pauses more transparent.

Compiler correctness

  • Andreas found and characterised a bug in GHC where GHC ignored GHC.Magic.noinline uses applied to functions taking a dictionary argument argument. The bug (#18995) was quickly fixed (!4616) by SPJ.

  • Andreas has been finishing a patch improving the compiler’s behavior when typechecking large floating-point literals (#15646).

  • After a internal refactoring using -ddump-asm-regalloc-stages with -fregs-graph used to cause a compiler panic. Andreas fixed this in !4516.

  • GHC’s testsuite allows tests to depend on external libraries. In order for this to work well with cabal-v2 Andreas added support for package-dbs (!4394) to the testsuite. This allows us to run tests depending on certain packages installed using cabal v2-install.

  • Ben and David Eichmann worked to fix (!4637) a number of race conditions in the eventlog implementation revealed by testing of ghc-eventlog-socket.

  • Ben worked to fix a number of testsuite regressions hidden by a subtle bug in the CI infrastructure (!2890).

  • Ben fixed a bug where the compiler would throw an internal error when an irrefutable match was used in a do block while the -XStrict language extension is enabled (#19027).

Compiler functionality and language extensions

  • Ben finished up the implementation of the BoxedRep proposal (#17526) in preparation for the 9.2 release.

  • Adam Gundry fixed a variety of bugs with DuplicateRecordFields, in particular to allow the extension to be used with PatternSynonyms (!4467).

  • The NoFieldSelectors extension, which will hide selector functions to make it possible to define expressions with the same name as field labels, is nearly ready to land (!4743).

  • Ben introduced a new runtime system flag, --eventlog-flush-interval, allowing that the user specify the maximum time which an eventlog event can remain buffered before being flushed to the event sink (!4621). This can be helpful in live-monitoring use-cases.

  • Ben implemented initial support for hugepages on Linux (#17134, !4523), reducing TLB pressure for programs with large heaps (benchmarks pending).

Compiler error messages refactoring