Here is a story I heard from a friend who's a PhD in aerospace engineering:

He + team built a simple plane, calculating all the dimensions of wings, etc using all the textbook formulas

Plane didn't fly.

So they just increasing wingspan by a factor 1.5x and the plane worked

(Like, a small model plane, not anything with people in it!)

A theoretically unsound plane that flies beats a theoretically sound plane that doesn't fly!

My theory is that most sufficiently complex systems involves some theoretically unprincipled hacks developed through trial and error. Often these hacks have some "theoretical intuitions" behind them -- plane looks like it needs more lift so let's increase wingspan

Other examples of "hacks":

- Monetary policy looks like it's not affecting long-term rates, mortgages, enough, so let's buy long-term treasuries, MBS's directly.

- Single-unit auction theory doesn't account for budget constraints, which are important for ad auctions in practice: let's invent a "pacing" algorithm which basically figures out the bid which spends your budget over the time span you want to spend it on

- Most of ML: why in the *@#$ do SVMs/deep learning/regression trees/etc actually work? Maybe we know the answer better now, but the theory really seems to have followed the practice on this one

Theory (ex-post) is often useful in these settings: understanding, at some level, why things work, helps understand the limitations of hacks, and how to extend them

But, IMO, theory works best used hand-in-hand with experiments, each building on each other. Approaching the world totally theory-first, throwing out anything which doesn't neatly fit into models/theories, just doesn't seem to work well for solving practical problems

"Experiments" wasn't quite the right word here: I meant some combination of intuition/experimentation/hacks (thanks @TibiAyman )