Working on build systems full-time at Meta
Neil Mitchell's Haskell Blog
by Neil Mitchell
2y ago
Summary: I joined Meta 2.5 years ago to work on build systems. I’m enjoying it. I joined Meta over two years ago when an opportunity arose to work on build systems full time. I started the Shake build system at Standard Chartered over 10 years ago, and then wrote an open source version a few years later. Since then, I’ve always been dabbling in build systems, at both moderate and small scale. I really enjoyed writing the Build Systems a la Carte paper, and as a result, started to appreciate some of the Bazel and Buck design decisions. I was involved in the Bazel work at Digital Asset, and afte ..read more
Visit website
Huge Project Build Systems
Neil Mitchell's Haskell Blog
by Neil Mitchell
3y ago
Summary: Shake won't scale to millions of files, this post says what would be required to make it do so. While Shake has compiled projects with hundreds of thousands of files, it's never scaled out to millions of files, and it would be unlikely to work well at that size. The most popular build systems that operate at that scale are Buck (from Facebook) and Bazel (from Google). In this post I go through the changes that would need to be made to make Shake scale. The first issue is covered in my previous post, that Shake doesn't know if you change the build rules themselves. As you scale up, it ..read more
Visit website
Small Project Build Systems
Neil Mitchell's Haskell Blog
by Neil Mitchell
3y ago
Summary: Forward build systems might work better for small projects. Yesterday's post talked about how Shake is a good medium sized build system - but what about smaller projects? Is the Shake model right for them? Shake can be considered a backwards build system. Each rule says how to produce a file, given some input files (which are dependencies) and an action. Almost all build systems (e.g. Make, Buck, Bazel, CMake, SCons, Ninja) fit this model, which is analysed in the Build Systems a la Carte paper. While this model works, it has two disadvantages: You have to explicitly list dependencie ..read more
Visit website
Reflecting on the Shake Build System
Neil Mitchell's Haskell Blog
by Neil Mitchell
3y ago
Summary: As a medium-sized build system, Shake has some good bits and some bad bits. I first developed the Shake build system at Standard Chartered in 2008, rewriting an open source version in my spare time in 2011. I wrote a paper on Shake for ICFP 2012 and then clarified some of the details in a JFP 2020 paper. Looking back, over a decade later, this post discusses what went well and what could be improved. The first thing to note is that Shake is a medium sized build system. If you have either 1 source file or 1 million source files, Shake probably isn't a good fit. In this post I'm going t ..read more
Visit website
Recording video
Neil Mitchell's Haskell Blog
by Neil Mitchell
3y ago
Summary: Use OBS, Camo and Audacity. I recently needed to record a presentation which had slides and my face combined, using a Mac. Based on suggestions from friends and searching the web, I came up with a recipe that worked reasonably well. I'm writing this down to both share that recipe, and so I can reuse the recipe next time. Slide design: I used a slide template which had a vertical rectangle hole at the bottom left so I could overlay a picture of my video. It took a while to find a slide design that looked plausible, and make sure callouts/quotes etc didn't overlap into this area. Camera ..read more
Visit website
Data types for build system dependencies
Neil Mitchell's Haskell Blog
by Neil Mitchell
3y ago
Summary: Monadic and early cut-off? Use a sequence of sets. In the Build Systems a la Carte paper we talk about the expressive power of various types of build systems. We deliberately simplify away parallelism and implementation concerns, but those details matter. In this post I'm going to discuss some of those details, specifically the representation of dependencies. Applicative build systems In an applicative build system like Make, all dependencies for a target are known before you start executing the associated action. That means the dependencies have no ordering, so are best represented a ..read more
Visit website
Turing Incomplete Languages
Neil Mitchell's Haskell Blog
by Neil Mitchell
4y ago
Summary: Some languages ban recursion to ensure programs "terminate". That's technically true, but usually irrelevant. In my career there have been three instances where I've worked on a programming language that went through the evolution: Ban recursion and unbounded loops. Proclaim the language is "Turing incomplete" and that all programs terminate. Declare that Turing incomplete programs are simpler. Have non-technical people conflate terminate quickly with terminate eventually. Realise lacking recursion makes things incredibly clunky to express, turning simple problems into brain teasers ..read more
Visit website
Don't use Ghcide anymore (directly)
Neil Mitchell's Haskell Blog
by Neil Mitchell
4y ago
Summary: I recommend people use the Haskell Language Server IDE. Just over a year ago, I recommended people looking for a Haskell IDE experience to give Ghcide a try. A few months later the Haskell IDE Engine and Ghcide teams agreed to work together on Haskell Language Server - using Ghcide as a library as the core, with the plugins/installer experience from the Haskell IDE Engine (by that stage we were already both using the same Haskell setup and LSP libraries). At that time Alan Zimmerman said to me: "We will have succeeded in joining forces when you (Neil) start recommending people use Ha ..read more
Visit website
Interviewing while biased
Neil Mitchell's Haskell Blog
by Neil Mitchell
4y ago
Interviewing usually involves some level of subjectivity. I once struggled to decide about a candidate, and after some period of reflection, the only cause I can see is that I was biased against the candidate. That wasn't a happy realisation, but even so, it's one I think worth sharing. Over my years, I've interviewed hundreds of candidates for software engineering jobs (I reckon somewhere in the 500-1000 mark). I've interviewed for many companies, for teams I was managing, for teams I worked in, and for other teams at the same company. In most places, I've been free to set the majority of the ..read more
Visit website
Which packages does Hoogle search?
Neil Mitchell's Haskell Blog
by Neil Mitchell
4y ago
Summary: Hoogle searches packages on Stackage. Haskell (as of 27 July 2020) has 14,490 packages in the Hackage package repository. Hoogle (the Haskell API search engine) searches 2,463 packages. This post explains which packages are searched, why some packages are excluded, and thus, how you can ensure your package is searched. The first filter is that Hoogle only searches packages on Stackage. Hoogle indexes any package which is either in the latest Stackage nightly or Stackage LTS, but always indexes the latest version that is on Hackage. If you want a Hoogle search that perfectly matches a ..read more
Visit website

Follow Neil Mitchell's Haskell Blog on FeedSpot

Continue with Google
Continue with Apple
OR