(thread)

There's a fair bit of (unintended) complexity that arises from moving to the "extremes" of modularization, as both ESM and component-oriented frontend frameworks tend to encourage...

For one, we experience more pressure/pain in terms of creating hierarchical directory structures to organize all these bits, which has the offshoot of permeating lots of deep relative path references (../../../../whatever/else).

All these paths CAN work, but they definitely create headaches for refactoring maintenance.

Maybe even worse, anything less localized than a single ../ in a path quickly makes it hard for a human to reason about the relationship between "here" and "there", so it becomes opaque.

IOW, the point of the hierarchy of directory structure is to create semantics around the relationships of assets, but it quickly gets so complex that our eyes glaze over and we stop paying any attention to the locations of things anyway, awash in the sea of ../'s.

To manage all that opaque relative pathing stuff, we often lean heavily on tools (IDEs, build tools, etc) to help us construct and adjust them.

Sadly, module systems like ESM prefer declarative static paths (for reasons), so we lose a lot of imperative tools to define paths.

There's a whole class of mechanics built up around managing all these paths that only has to exist because we decided that we should hyper-modularize and deeply heirarch our directories. If you haven't written code to figure out paths... it's hard.

This is unintended complexity.

The fact that ESM `import` rolled out with static paths only, and then later we had to "patch" JS to add dynamic `import(..)` so we (in part) can regain imperative control over our paths, is, in my mind, proof that we were over zealous in the post-ES6 move to "module everything".

Naming is another pressure we experience when we start modularizing everything. I can't be alone in my eyerolls when I see four and five word (often camel-cased) long component names (and their matching file names) strung out across an app.

Associated there is how easily component-oriented frontend architecture creates "div-itis", because every time we want to create a module boundary, we have to create a DOM boundary. SMH. At least the DOM lets us make these div boundaries anonymous so we don't have to name them.

Another unintended complexity of hyper modularization is the increase in circular dependency. the mechanisms can often handle circular dependency mechanically, but that doesn't mean it's clean/easy for humans to reason about such. I find it's often not easy.

True story: I was recently converting a small lib of mine (about 3k in file size) from a single module (UMD) to 3 separate modules (CJS, ESM)... I discovered a circular dependency that stems from an attempt I had previously made to "share" abstraction (to save file size).

This sharing was completely natural and easy when everything was in one file... but it was actually nearly impossible to express (that way) in separate modules that have static export semantics.

Why?

Because the utilities present on one module's API are actually used to define the methods on another module's API.

ESM, for its part, clearly discourages this. You're supposed to have static exports. You can do some funky work-arounds, but it was messy.

So what I thought was going to be a simple splitting of a single module into 3 turned out to be a fairly involved re-factoring, with some intentional duplication (reduction in sharing) to work out the separation. There's still circular dependency, but it's much more limited now.

And BTW, the choice to extract out those 3 modules from the one was based on a simple observation of namespaces... there were 3 logical collections of methods in my single module, that even had sub-namespace aliases.

So that modularization design was the clear obvious choice.

But I've now pondered what it would have been like to design this lib with *only* the deep modularization mindset from the start... would the opportunities to share behavior have even occurred to me, given the limitations of ESM?

And is my software "better" being split now?

I think a strong case could be made that if you find yourself struggling much with circular dependency, what you really have is a single module that you're trying to force into separate files, and that's awkward because it *should be*, because you shouldn't do that.

Remember, this was a tiny lib (3k in size), and only ended up as 3 modules (with a single circular dependency between two of them). Imagine how quickly such complexity balloons when you're doing substantial apps with this hyper-modulization in mind.

One aspect of this is, should the modularization of the thing -- that is, the splitting into separate namespaces, basically -- be driven top-down, or bottom-up?

IOW, should we draw the lines based on typical business-domain driven distinctions between "things", or...

...should we draw those lines around internal "boundaries" of functionality, keeping things grouped together that *use* each other, and only exposing module boundaries to the outside where there are clean breaks?

These are tough questions to answer. It depends.

I think most software is designed, in terms of modularity and organization, from the top-down... which has the unfortunate effect (as I described in my case) of sometimes constraining/limiting our implementations and abstractions.

But if you design purely bottom-up, it's very easy to accidentally "leak" your implementation choices outward, if even in just creating software that's more awkward for others to use.

If you pull back a bit from the extremes of hyper modularization, having fewer modules that are more deeply composed, you have more chances to balance these concerns. The more modularized you make a system, the harder that balance is to strike.

Linux, an OS basically composed of thousands of single-purpose apps, is often touted as the vanguard pinnacle of modularization superiority. But we forget how hard it was to design all that so it worked well, and we also ignore that there's still tons of rough edges even still.

Modularization (or componentization, if you prefer) is often touted as the "clearly better" way of designing software.

But it seems we're doing the same things devs always do: take a good idea in moderation, shift it up to the extreme, then patch up all the offshoot complexity.