Going to rant about PATHs a bit.

The idea of search paths like PATH, LD_LIBRARY_PATH, PYTHONPATH, etc. dates back to MULTICS in 1969 and hasn’t really changed since.

Paths are cool because they let you (the user) swap new versions of things into your environment, but... 1/n https://twitter.com/tgamblin/status/1343626506824609794

Paths are also very uncool because they’re 1) global (at least in a single environment), 2) set by the user, and 3) a simple runtime heuristic for something much more complicated: dependency resolution.

Every time you search a path, you’re looking for a dependency. 2/n

Paths cause all kinds of reproducibility headaches. User having issues with a program? Set LD_LIBRARY_PATH! Oh, you already set it for that other program that has a conflicting version of that library. Too bad, one or the other of those programs will find the wrong version. 3/n

One script needs GNU sed and the other assumes BSD sed? Too bad, you get one PATH.

PATHs come with inbuilt assumptions about compatibility, which get passed on to the user and manifest as nasty configuration problems. 4/n

PATHs date back to an era where people pretty much knew their whole stack. That's impossible today because we reuse SO much code across different environments. The *user* (who controls the PATH) likely knows nothing about the requirements of their script/library/app. 5/n

The sad thing is that developers likely know very little about the requirements either. They just know it works on Ubuntu -- so containers were born. Can't specify precisely? Specify the universe it came from. Think of it as a horrible upper bound on the actual requirements. 6/n

Static linking, RPATH, resholve, containers are probably the best current solutions to this problem -- hard-code what the program needs. The trouble is you have to hard-code *everything* down to a really low level, and that's currently tough to reason about. 7/n

If you want to relink, or reconfigure an abspath'd program/environment/whatever, you're back to a resolution problem. Change one version in your system? Something might be incompatible with it. Now you need to search for that, and so on. Configuration (resolution) is NP-hard. 8/n

We have a very write-only development model right now -- human configures, snapshots, ships. If you want smooth changes or upgrades in a system like this, you need something that can reason about interfaces: functions, command-line options, ABI, etc. 9/n

In my ideal world you'd have better resolution. One program needs a different sed? Resolver reasons about whether that affects other stuff in your env based on requirements, and sets up your programs to use two seds if needed. 10/n

Need a new version of a library? Resolver looks at the ABI and checks it against other stuff in your environment. Finds you a compatible set. 11/n

This is way harder to think about than search paths, so you can see why they settled for them in 1969. Dependency resolvers only think about versions (not ABI, params, etc.), and scripts only know the *names* of commands they need. 12/n

To do better, we need way more provenance and way better *interface* specifications -- from CLI down to ABI. With that, we can start expressing things in a way that a machine could actually reason about, and we might see a system that can upgrade itself w/o manual config. 13/n

Without that, software will continue to be brittle, and we'll continue to settle for PATHs. PATHs are a *mechanism* for configuration, but they don't guarantee safety. We need automated reasoning to change complex systems safely *without* knowing everything about them. 14/14.