Read on Twitter
Summary of「Systems Design that Explains the World」by @apenwarr https://apenwarr.ca/log/20201227
1. What is systems design?
2. Systems of control: hierarchies and decentralization
3. Chicken-egg problems
4. Second-system effect
5. Innovator's dilemmas
1. What is systems design?
2. Systems of control: hierarchies and decentralization
3. Chicken-egg problems
4. Second-system effect
5. Innovator's dilemmas
Before talking about what systems design is, an important observation of big tech's engineer promo ladder:
There are two types of misfits:
1. People who program well but can't produce designs.
2. People who excel in design but not coding.
1. People who program well but can't produce designs.
2. People who excel in design but not coding.
If both types of people start as junior devs, group 1 will rise faster but will stall at Senior. Group 2 has the potential of becoming executives but can't get promotions because they can't pass the early stage.
However, translating bigger and bigger business problems into designs requires a completely different skill set than a programmer, and "rising through the ranks doesn't prepare you for that job at all".
1. What is system design?
"It's fixing which promotion ladders your company even has, rather than trying to climb the ladders..."
"It's fixing which promotion ladders your company even has, rather than trying to climb the ladders..."
... "It's macroeconomics instead of microeconomics. It's knowing when a distributed system is or isn't appropriate, not just knowing how to build one. It's repairing the incentives in a political system, not just getting elected and passing your favorite laws."
2. Systems of control: hierarchies and decentralization
Command-and-control hierarchy vs. flat organization? A real system is never black-and-white. Even in the most flat, decentralized systems, there is always an implicit central force that enforces the rules.
Command-and-control hierarchy vs. flat organization? A real system is never black-and-white. Even in the most flat, decentralized systems, there is always an implicit central force that enforces the rules.
"Democracy depends on someone enforcing your right to vote. Capitalism depends on someone enforcing the rules of a 'free' marketplace."
Hierarchy or flat, it boils down to whether you have explicit leaders or implicitly enforced rules in your design.
Hierarchy or flat, it boils down to whether you have explicit leaders or implicitly enforced rules in your design.
3. Chicken-egg problems
Neither chicken or the egg came first. There were chicken-ish and egg-ish things before them in the long evolution all the way back to a single-cell organism.
Neither chicken or the egg came first. There were chicken-ish and egg-ish things before them in the long evolution all the way back to a single-cell organism.
A product has the chicken-egg problem because it is not useful until people use it, examples: phone and fax, the Internet, IPv6, social networks, game consoles. The next level up is 2-side marketplace like Uber, or 3-side like UberEats.
New systems must have a gradual introduction phase with proper design such as backward compatibility, aggressive user acquisition, and adoption, etc. Ignoring the chicken-egg problem, your product will likely fail.
4. Second-system effect
Scenario: a product that started small but became big, then a lot of hacks and compromises were added to optimize for iteration speed. Later when the eng org is big, people start to rewrite a "second system" from scratch. (A lot of big companies do this)
Scenario: a product that started small but became big, then a lot of hacks and compromises were added to optimize for iteration speed. Later when the eng org is big, people start to rewrite a "second system" from scratch. (A lot of big companies do this)
Result: projects take longer, new issues introduced despite solving old ones, dev time split between maintaining the old and building the new, the old system likely get shut down early and users are forced to switch.
Problems particularly bad if the team who built the old system left and new people working on the new one have no experience.
If you can avoid building the second system, don't.
If you can avoid building the second system, don't.
5. Innovator's dilemmas
Scenario: Big company A (Intel) had a high-margin product. Small company B (ARM) had a low-end low-margin product. A constantly shot down low-margin ideas and optimized for profits. B improved its product a tiny bit, A "retreats upmarket". Over and over.
Scenario: Big company A (Intel) had a high-margin product. Small company B (ARM) had a low-end low-margin product. A constantly shot down low-margin ideas and optimized for profits. B improved its product a tiny bit, A "retreats upmarket". Over and over.
Result: One day, B produces a comparable or better product than A, A is defeated.
The constant optimization for high-margin projects and profit creates a corporate environment where a product that can compete with B could never be created.
The constant optimization for high-margin projects and profit creates a corporate environment where a product that can compete with B could never be created.
「If you're a startup and think you have a disruptive innovation, that's great news for you. It's a perfect answer to that awkward investor question "What if [big company] decides to do this too?" because "their own politics will tear that initiative apart from the inside." 」
