Read on Twitter
Surveys & Platform teams: a thread on what I've learnt @SquareEng
As a platform team, it's important to understand the needs of your customers, the engineers at your company. Surveys can help with that. They can also be utterly useless and a distraction.
As a platform team, it's important to understand the needs of your customers, the engineers at your company. Surveys can help with that. They can also be utterly useless and a distraction.
First, the obvious that people needs to be constantly reminded of: survey fatigue is real.
- Run surveys max 2 to 4 times a year
- Favor one survey for a large eng category (e.g. mobile, server, etc) vs several per product survey (e.g. gradle, IDE, etc)
- Keep surveys short.
- Run surveys max 2 to 4 times a year
- Favor one survey for a large eng category (e.g. mobile, server, etc) vs several per product survey (e.g. gradle, IDE, etc)
- Keep surveys short.
Also: keep surveys anonymous. Otherwise they'll just be another vanity metric.
Add an optional field where people can write their name if they're ok with you reaching out to clarify some comments.
Add an optional field where people can write their name if they're ok with you reaching out to clarify some comments.
Use surveys to evaluate and rank problems. Don't use them to analyze problems. Don't use them to evaluate solutions.
E.g. "rank what's slowing you most: local build, CI, IDE". Not "why is your CI slow?". Not "Grade this tool"
E.g. "rank what's slowing you most: local build, CI, IDE". Not "why is your CI slow?". Not "Grade this tool"
Every time we are gathering requirements to solve a new problem at the platform level, someone says "hey let's run a survey". NO. That's a cop out. Do your job, sit down and interview your eng customers.
Survey aren't a cheap trick to scale gathering requirements.
Survey aren't a cheap trick to scale gathering requirements.
The default stance from your customers is that your survey is useless and you won't act on it.
To fight this: publish survey results AND the team's interpretation. Associate publish OKRs / roadmap items with prior survey results.
To fight this: publish survey results AND the team's interpretation. Associate publish OKRs / roadmap items with prior survey results.
Carefully craft your survey questions. Keep the questions mostly constant over time: the value comes from trends over time.
Be very cautious of adding new questions. Keep the survey short!
Be very cautious of adding new questions. Keep the survey short!
If a question is an item selection, you most often should enable multiple choice, AND you should add a "Other" text field.
You have unconscious bias in how you're designing the question, leave room for that bias to be surfaced.
You have unconscious bias in how you're designing the question, leave room for that bias to be surfaced.
When you follow these guidelines, surveys are super useful. They help prioritize roadmaps and help justify the work of platform teams to execs and other teams.
I've learnt all this watching @lukestclair set this up during his time on the Square mobile platform team
I've learnt all this watching @lukestclair set this up during his time on the Square mobile platform team
