Read on Twitter
This is 100% true. This explains why every other tool out there feels vaguely the same to you, and why honeycomb feels so radically distinct.
They all started out stuffing metrics into RDBMS or TSDB. We spent the first 1-2 years building a storage engine for ultrawide events. https://twitter.com/perfclarity/status/1331077323307696128
They all started out stuffing metrics into RDBMS or TSDB. We spent the first 1-2 years building a storage engine for ultrawide events. https://twitter.com/perfclarity/status/1331077323307696128
We didn't *want* to write a db, ffs. I've spent my whole career yelling at people "DO NOT WRITE A DATABASE!" But we needed a lot of tradeoffs that no one else was making:
- immutable, append-only column store
- schemalessness
- fast and close to right is better than "correct"
- immutable, append-only column store
- schemalessness
- fast and close to right is better than "correct"
- aging out column data to S3
- high cardinality, high dimensionality
- extremely wide, arbitrarily-wide events
- indexlessness
Technically we didn't write a database, anyway, we wrote a
storage engine. come at me bro =p
- high cardinality, high dimensionality
- extremely wide, arbitrarily-wide events
- indexlessness
Technically we didn't write a database, anyway, we wrote a
storage engine. come at me bro =p
The author goes on to say this, which is also accurate.
You cannot achieve observability if your storage engine needs schemas, because observability is about handling unknown-unknowns. By definition, you cannot predict in advance what data you may need. https://twitter.com/perfclarity/status/1331079463723966466
You cannot achieve observability if your storage engine needs schemas, because observability is about handling unknown-unknowns. By definition, you cannot predict in advance what data you may need. https://twitter.com/perfclarity/status/1331079463723966466
You cannot achieve observability if you force people to define indexes, because that's just predicting in advance which dimensions will be valuable enough that you'll want to query efficiently on them.
I have written this shit up on many a time
https://www.honeycomb.io/blog/so-you-want-to-build-an-observability-tool/
I have written this shit up on many a time
https://www.honeycomb.io/blog/so-you-want-to-build-an-observability-tool/
HOWEVER. it may sound like i am bragging about how crazy smart we were, and lordy no, no I am not.
We did it all wrong, and we are so lucky to have (barely) survived our own stupidity.
We built this like engineers solving a problem, not like founders starting a viable business.
We did it all wrong, and we are so lucky to have (barely) survived our own stupidity.
We built this like engineers solving a problem, not like founders starting a viable business.
I remember about 8 or 9 months in to the first year, when an early investor was asking us about customer traction and who we had gotten started using it, and I was looking at at him like he was slow. "Dude, we haven't gotten there yet. First we gotta do this custom not-a-db...."
We didn't -- couldn't -- really start going out and start building the signup and user flow and integrations and all the other user-facing bits -- you know, that whole "product" thing -- until we were a couple years in, and on our second seed round. This definitely hurt us.
In retrospect, I made all the exact mistakes you'd expect an ops/infra engineer to make, all whilst feeling totally confident that I was far too self-aware and conscious of those potential pitfalls to fall in. Meh.
We are well out of the woods by now, of course. And I am grateful our investors kept rolling the dice on us long enough to let us wobble across the chasm, but I am even more grateful to our brilliant, loyal, extremely vocal and passionate users,
who spoke up again and again on our behalf, vouching for us to investors, telling them that yes, honeycomb really was something new and special and no, christine and i weren't just high on our own supply. 
The moral of the story is, of course, never write a database. Ever.
Unless you really, really, really have to. Then write a database. But it will probably cause you to fail if you do.
Unless you really, really, really have to. Then write a database. But it will probably cause you to fail if you do.
