i guess we’re all still on vendor assessments so here you go. a bunch of obvious stuff that everyone who has put 5 minutes into thinking about this already knows:

1. Everyone says theyre assessing something. Controls. Security Posture. Risk of loss of control. Blah blah. You aren’t doing any such thing. With a large staff of deep experts you can model risk of compromise to some degree, but that still isn’t those things.

You’re measuring, kinda, sorta two things at best: 1) is their security program around the right degree of maturity for what they’re doing? and 2) is their security program better than yours for the data they get from you?

I’ve built and rebuilt vendor assessments and yea they used questionnaires but they only point of the questionnaire was to give an expert a temp check on those two things, with a little cross checking of answers looking for BS.

2. Checking for BS is essential. I cannot tell you the number of times I’ve focused on something with a well known enterprise software vendor, assuming it’s a misunderstanding, only to find that no, their CTO is actually a dark cloud of lovecraftian madness from beyond time

one single example. a large, somewhat well known integration and systems management provider: “yea we encrypt the api keys in a mysql table and then the gateway decrypts them to compare to what you provided. haha, how else would you do it??”

3. You want to know if this expensive vendor is trustworthy with your sensitive data. Cool.

Are you trustworthy with it? If not then find someone more competent, sure. If so then don’t fucking give it away. Why the hell would you change state from “known good” to “ehhhhh?”?!?

the answer to why is terrible.

good security is expensive. the people are expensive. the process is expensive. everything about it is expensive.

no one outsources anything unless:

A. they’re deliberately paying for competence they don’t have and don’t want and not degrading other capabilities. this is usually more expensive than any other option, but reasonable. it’s only ever done for boring foundational elements (your cloud provider. email. etc.)

B. They’re deliberately shifting expense categories. They’re going to pay less to get more of a capability by skimping on other ones that are currently bundled. This is a huge % of most startup usage.

“They do X well and cheaper than we can!”

OK, what bundling do you lose?

C. Escaping the bundling is the point. This is the worst one. It means that people in your company are OK with increasing some types of risk to lower the ability of central services to muck about with their projects. Occasionally can be bad faith. A pattern signifies central rot.

the entire startup market is built on slight of hand around B with more than a little of C. they’ll tell you that everything else is fine and/or good enough and charge less to do a thing than your internal costs. assuming your internal costs are rational, you know they’re lying

ahh but capex vs opex and our employees expect that we use Foo because it’s cool and the orange website is full of people saying it’s better and the startup advisor that spoke at my school told us we should be learning it and

and here buried deep where no one will ever see it is that problem:

the startup ecosystem is mostly a multi-faceted scam. its hype cycles are orchestrated and/or faked. the cross-licensing across VC firms is brokered. the startup media and school pipelines are captured

i don’t mean “startups are scams flee back to as400 you fools!”

i mean the entirety of the tech media, tech schools, VC advising, new stuff “replacing” 30 year old systems functions, sales cycles, startup cross-licensing, conference talk circuits for cool CTOs thing is a scam

it’s a cycle to extract wealth from almost everyone involved, coming and going. not gonna solve it here. prolly not gonna solve it at all. anyway

point is, every time you fire up your “vendor assessment process” that is the ecosystem with which you are interfacing

there is no honest reality for you to assess because every part of the modern tech ecosystem reality is, fractically, a fiction that doesn’t care about your interests or even the interests of the layer above or below whatever piece of fiction you’re examining

so again:

you’re assessing if they’re doing the things you’d expect, in ways you’d expect, if you were they. expand the “you” to be additional smart people and make the call. use a process that’s post hoc understandable. try to use a process that’s repeatable.

also you’re assessing if they give a fuck about all this security/privacy stuff and if they’re allowed to make their engineers do so in ways the engineers would maybe prefer to not do

you’re assessing if they assess their vendors

that’s it. the rest is just a shared fiction