Yes, anyone who is a long-time Less-wrong/SL4 fan must surely have thought of this classic post by Eliezer.

I should pour some cold water on that though: this advance by Deepmind wasn't superintelligence, it was neural network science led by ordinary v1.0 humans at @DeepMind https://twitter.com/Cameloparadlis/status/1333428368323317762
In this post and I believe an accompanying talk, Eliezer outlined an "out of the box" way that a recursively self-improving superintelligence could literally take over the world pinky-and-the-brain-style.
You can find the story at this PDF page: http://intelligence.org/files/AIPosNegFactor.pdf
The gist of it is a superintelligent AI pays labs to make certain amino acid sequences (by, like, emailing them and paying with online money of some kind), then gets someone to mix the various solutions together.
Because the SAI has "cracked the protein folding problem", it can optimize the amino-acid sequence to make some kind of custom molecular machinery. This custom machinery could build more complex machinery and eventually get to advanced molecular nanotechnology....
And then with that advanced molecular nanotechnology it could build various robots, drones, grey goo etc that would eat up the entire planet (including us) if it wanted to.
You can follow @RokoMijicUK.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled:

By continuing to use the site, you are consenting to the use of cookies as explained in our Cookie Policy to improve your experience.