Read on Twitter
Thinking lately about how racket macros are very good, but that has remarkably little to do with what people would traditionally call a “macro system”; the most interesting pieces are supporting tech that racket people just happen to lump into the same bucket for history reasons.
When people hear “macro,” they think “ad-hoc syntax rewriting rule,” and “hygiene” just means “capture-avoiding rewriting,” which is all basically correct, even in Racket. But “the Racket macro system” actually does a whole bunch of other things besides expand macros.
For example, the Racket module system provides predictable separate compilation in an aggressively staged world. Racket tracks dependencies separately between compilation stages and enforces rules about compile-time side effects to allow control over compilation to span modules.
For another, Racket defines lots of protocols so that different stages of a program may receive egalitarian treatment from tooling, such as error reporting, jump-to-definition support, and interaction with debuggers.
And Racket has a fairly rich compile-time API that allows programs to introspect on program structure in addition to guiding compilation. You can attach arbitrary information to bindings and recover it later, even if it crosses several module boundaries along the way.
Most of these things don’t actually have much to do with “syntax rewriting.” Rather, they are a collection of modular pieces built in pursuit of a larger goal: making it possible to build programs out of towers of little languages, DSLs that aren’t bound by the host language.
In this universe, the “macro” is not the fundamental concept, but the language. You have little pattern languages, little class languages, little import languages, little linking languages, all of which are entities in their own right, not a pile of ad-hoc syntactic sugar.
Authors of these languages define a little grammar, a little parser, and a little compiler, and Racket-the-system provides scaffolding (and it takes a lot!) for them to all coexist together, intermingling freely. When we say “macro,” we’ve often already failed to communicate.
The big idea is not “macros,” it’s overcoming the perennial dilemma of “do I invent a totally new programming language or awkwardly cram my concepts into the one I already have?” because the first of those choices is usually a complete non-starter, so we usually get the latter.
People rail all the time against DSLs in other languages for trying to be too “magical,” but what they really mean is they resort to opaque tricks that make debugging impossible and break when you look at them the wrong way. Racket is willing to ask “what if they didn’t have to?”
