1/ At @tryhaystack we have talked to over 500 engineering leaders, analyzed 400,000 private pull requests & 3M commits of non-opensource companies.

Here's our takeaway
2/ We've seen engineering leaders having the hardest time with the following questions:

- Are we getting better over time?
- Where does work get stuck?
- How effectively are we working?
- What did we accomplish last month?
- How do we assess the value of fixing tech debt?
3/ These questions are hard to answer because of the problem's nature, multiple variables effects affecting the outcome. We just don't have all the data.

To progress further, we need to change the way we think from localized problems to what's the main goal?
4/ Thinking from first principles, an organization is successful if they have 📈 Revenue 📉 Churn. These are Lagging Metrics. They give you feedback later than you want
5/ Each part of the organization have their KPIs which reflect Revenue.

- sales -> Sales Revenue..
- product -> MAU..
- marketing -> CAC and LTV..

What about Engineering? We found that this metric is

"# of Successful Iterations"
6/ YC recommends launching fast then iterate. @paulg mentions in his blog "13 Sentences" as #2 and #3 most important items.

@elonmusk calls this Iterative Development and tries to maximize the # of iterations for both @SpaceX and @Tesla

7/ It's obvious why this metric succeeds. You try something, get feedback on it, fix the issue, repeat. The more you iterate, the more likely you'll build the correct product, the more likely you'll be ahead of the market.
8/ The million-dollar question is "How do we track # of successful iterations?"

Deployment Frequency: The more you deploy, the more you iterate.

This is correct, but missing a piece
9/ @RoyOsherove gives a really good example in his GOTO talk

The manager asks: "How can we deploy faster?"
The team responds: "Let's not write tests!"

We are ignoring half of the statement:

# of "SUCCESSFUL" iterations
10/ We not should only track speed, but we should track quality along with it.

- Change Failure Rate
- # of Bugs
- MttR

are proxies for high-level app quality. There are more details on the correct way to track these, which I'll not get into. DM if you want to know more.
/11 Summarizing

Aim: # of successful iterations

- Speed:
* Deployment Frequency
* Cycle Time
- Quality:
* Change Failure Rate
* # of Bugs
* MttR

We call these Northstar Metrics ⭐️
12/ What about all these metrics:
- % coverage
- pr size
- build & deploy speed
- review time
- # of work in progress

These are "Leading Metrics". They are tools to get faster feedback on your process changes.

DON'T use Leading metrics as KPIs.
13/ Common pitfall most teams fell into is tracking exact numbers. Don't do this.

- Trends
- Distribution (aka outlier)

@CatSwetel has some interesting ways of visualizing trends and distribution.
14/ Continuous Delivery advocates like @samnewman, @thoughtworks know what works and what doesn't work well.

They started the CI/CD movement to help engineers "Iterate Faster Successfully"

We must not forget @jezhumble, @RealGeneKim, @nicolefv for their incredible research.
16/ Here are some of these great fellows who made the ecosystem move forward @martinfowler, @jpetazzo, @jesserobbins, @allspawm, @AdamTornhill, @eoinwoodz, @jcoplien, @davidfarley, @andrew_glover, @PaulDuvall, and many more 👏
You can follow @thellimist.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled:

By continuing to use the site, you are consenting to the use of cookies as explained in our Cookie Policy to improve your experience.