Read on Twitter
Some days it really seems like every significant “intuition” that managers have about metrics or systematisation of their workplace is at best useless, and more often actively harmful to the business’s goals.
If your workers are engaged in complex cognitive work, using optimisation tools designed for 19th-century factories will not just be ineffective, it will be devastating.
This is especially true if you start trying to measure and reward things that you think correlate with outputs you care about, rather than the outputs themselves.
In a complex organisation it is almost a certainty that the relationship between your chosen correlated variable and the things you actually care about is MUCH more complicated than you realise.
If you tell people that you want fewer incidents in production, you will get... fewer incidents in production. But it is MUCH easier to change how you count incidents than it is to actually fix complex engineered systems, so people will simply change how they count.
(Meanwhile, your customers are exactly as unhappy as they were before.)
If you tell people that you want fewer issues reported by customers — our own systems should detect these issues — you will get... exactly that. But the easiest way to achieve this is to dial up the sensitivity on all your monitoring so your false negative rate drops.
But what happens to your false positive rate? Through the roof. Now every alarm is firing all the time and most of the alerts are ignored. Now when an issue is reported by a customer we say “our monitoring caught it” but we omit “and it was lost in a sea of noise.”
I could list concrete examples all day. The point is that defining quantitative goals will always distort the behaviour of your organisation in ways you didn’t expect, and if done haphazardly usually in ways that are actively harmful to your real goals.
The only way organisations doing cognitive function effectively is through difficult and continuous conversations between leaders and their teams about goals and priorities, with the intent of devolving context-heavy decision-making to the people closest to the problem.
This means that the people “at the top” do not ever get to tell the people at the sharp end of the organisation how to do their jobs. Strategic leaders are not the experts in “how.” They have to think about “which direction” and “why” and relentlessly communicate the answers.
If the sharp end of the organisation is doing things in ways that are surprising or seem unhelpful to higher-level leaders, it is for the leaders to understand why.
There are several possibilities:
1. They don’t understand the goal.
2. They are being rewarded for working towards a different goal.
3. They are actually helping, even though it didn’t look like it to leadership.
1. They don’t understand the goal.
2. They are being rewarded for working towards a different goal.
3. They are actually helping, even though it didn’t look like it to leadership.
In all three of these cases, if there is any problem to solve here it’s a problem for the leadership to solve.
What KPIs/metrics/etc. do as management tools is move responsibility for the outcomes down to the sharp end of the organisation, absolving leaders of the responsibility to understand how their organisation actually works.
They can just point at a number and everyone can agree that the number is too high/low/variable and needs to change.
The problem is that wherever you move the responsibility, people will use whatever agency they have to change the number in the desired direction... even if that agency is unrelated to the actual goal you were trying to set. Hence the tendency to redefine thresholds, etc.
I will attempt to wrap up by pointing out that there is nothing wrong with leaders using quantitative methods to understand whether their strategy is leading the organisation in the right direction. But that analysis should be done AT THE LEADERSHIP LEVEL and not shared.
That analysis also must be done critically, and with an understanding that the relationships between the measures and the real behaviour of the organisation maybe extremely complicated and unclear.
If the numbers look “bad,” or indicate problems, leadership’s job is to dig into that complexity and understand what is behind it, in order to inform how they should change their messaging, investments, etc.
That is (surprise!) more hard cognitive work, and is unfamiliar and unpracticed by many managers, at least as far as I can see in the tech industry.
But the alternative is the abdication of that work and a decision to just push the problem down the organisation as a context-free KPI. For the problems with that approach, please return to the start of the thread.
FIN.
FIN.
