Weird post of the day: if you use the complexity prior to make decisions, weird simulated aliens will trick you into letting them break into our universe and eat you for breakfast... or worse. https://www.lesswrong.com/posts/Tr7tAyt5zZpdTwTQK/the-solomonoff-prior-is-malign

Unrealistic though, as the pure complexity prior is uncomputable and most of the scenarios where the simulated aliens start to exert influence involve accidentally simulating their entire universe, history and evolution, which is obviously very expensive in time and memory.

Oh, also, trigger warning for people who find this kind of thing unsettling.

If you want to know how this would work, it would be something like a Game of Life initial configuration that turned into an entire mini-universe inside the game of life, evolved intelligent aliens, ...

... and then started sending out messages that we could measure by turning certain cells of GoL on and off. They would make great predictions for us to use for various practical purposes, but those predictions might be malevolent and eventually give booby-trapped advice.

Of course the game of life example is a bit contrived. It probably wouldn't be exactly that.

In general this is a form of mesa-optimizer, i.e. an algorithm that develops an internal optimizer that has its own ideas about what it wants to do.

Human beings are a mesa-optimizer for evolution - we do all sorts of things that don't optimize the objective of evolution (such as using birth control).