Read on Twitter
Will re-read. https://worldmodels.github.io/
Really good summary of the article. Looking forward to learning about more recent developments with this work.
@hardmaru
@hardmaru
Seems like this work also provides compelling convergent support for Erik Hoel's model that dreams can be understood as training with higher temperature/uncertainty simulations, so preventing overfitting and promoting generalization. https://twitter.com/erikphoel/status/1285570452405002241
From the world models paper linked above:
"Unlike the actual game environment, however, we note that it is possible to add extra uncertainty into the virtual environment, thus making the game more challenging in the dream environment..." (continued below)
"Unlike the actual game environment, however, we note that it is possible to add extra uncertainty into the virtual environment, thus making the game more challenging in the dream environment..." (continued below)
"We can do this by increasing the temperature \\tauτ parameter during the sampling process of z_{t+1}z
t+1, as done in [35]. By increasing the uncertainty, our dream environment becomes more difficult compared to the actual environment." (continued below)
t+1, as done in [35]. By increasing the uncertainty, our dream environment becomes more difficult compared to the actual environment." (continued below)
"The fireballs may move more randomly in a less predictable path compared to the actual game. Sometimes the agent may even die due to sheer misfortune, without explanation." (continued below)
"We find agents that perform well in higher temperature settings generally perform better in the normal setting. In fact, increasing \\tauτ helps prevent our controller from taking advantage of the imperfections of our world model — we will discuss this in more depth later on."
A non-mutually exclusive model of dreaming as (emotionally-biased) elaborative encoding of memories, which seems consistent with a policy-refinement interpretation (hippocampus : SWS :: amygdala : REM?). https://pubmed.ncbi.nlm.nih.gov/24304746/
If something was repeated extensively during the day (or was highly salient), might we be more likely to rehearse what happened, and less likely to engage in more loosely-constrained imaginative dreams? https://academic.oup.com/scan/article/13/6/637/5032636
More hippocampal and prefrontal involvement in lucid dreamers as consistent with the rehearsal/salience model above, where more coherent/accessible/memory-constrained dreams are also more likely to promote meta-cognitive awareness? https://pubmed.ncbi.nlm.nih.gov/30542052/
"In a survey of 17 middle-school students, search for control over situations in waking life was positively correlated with frequency of lucid dreaming." https://journals.sagepub.com/doi/abs/10.1177/003151259508100258?journalCode=pmsb
Highly simplified/compressed model:
Posterior cortices: Predicting what is.
Prefrontal cortices (PFC): Predicting what will/ought-to-be.
PFC down-regulated while dreaming (with eyes closed or open); up-regulated during lucid dreaming.
Posterior cortices: Predicting what is.
Prefrontal cortices (PFC): Predicting what will/ought-to-be.
PFC down-regulated while dreaming (with eyes closed or open); up-regulated during lucid dreaming.
With relevance to the "World Models" paper (above):
Posterior cortices: autoencoder heterarchy.
PFC-hippocampus: RNN predictor of auto-encoded latent space, action selector, and long-short-term-memory router of posterior dynamics.
Basal ganglia: reduced-parameter controller.
Posterior cortices: autoencoder heterarchy.
PFC-hippocampus: RNN predictor of auto-encoded latent space, action selector, and long-short-term-memory router of posterior dynamics.
Basal ganglia: reduced-parameter controller.
If all areas of cortex implement a common predictive 'coding' algorithm, then they're all in the same game of modulating their activity in order to minimize their surprise. (more on this below)
Consciousness models emphasizing conscious access view PFC almost like a kind of (hierarchically) high-level sensory cortex on top of what and where/how pathways.
Not incompatibly, I tend to view PFC as an extended motor hierarchy and modulator of posterior sensory cortices.
Not incompatibly, I tend to view PFC as an extended motor hierarchy and modulator of posterior sensory cortices.
By virtue of their hierarchically higher positioning, do PFCs (in conjunction with the rest of the brain) not just minimize prediction-error via policy selection over overt actions and covert attention, but by modeling counterfactual futures and pasts?
https://knightlab.berkeley.edu/publications/detail/74/
https://knightlab.berkeley.edu/publications/detail/74/
That is, more sensory-decoupled cortical areas must construct integrated world models to minimize surprise, and also leverage those models to enact futures where they minimize prediction-error relative to homeostatic/reproductive/evolutionary priors. https://twitter.com/adamsafron/status/1280949190537986059
What kind of prediction-errors are generated during dreaming?
Or while awake, if my brain predicts something will happen, and then it doesn't happen, and if non-attentional predictions generally suppress (or "explain away") observations, where would prediction-errors come from?
Or while awake, if my brain predicts something will happen, and then it doesn't happen, and if non-attentional predictions generally suppress (or "explain away") observations, where would prediction-errors come from?
If predicting one thing more means predicting other things less (implicit normalization/softmax?), then would those under-predictions be where prediction-errors are generated?
@djheeger https://www.pnas.org/content/114/8/1773
@djheeger https://www.pnas.org/content/114/8/1773
E.g. I reach out to grasp a cup, but if the cup isn't where I thought it would be, it may not be my expectations of where the cup was that 'grabs' my attention, but bottom-up observations of emptiness or torque (from not encountering an opposing force) that were not anticipated.
But what is the source of bottom-up observations while dreaming?
It seems like the white noise used in this work (which generates alpha due to the impossibility of predicting randomness?) may be analogous to stochastic firing during sleep:
It seems like the white noise used in this work (which generates alpha due to the impossibility of predicting randomness?) may be analogous to stochastic firing during sleep:
However, while dreaming or imagining (i.e., consciousness decoupled from sensation), it may be the case that past experience causes seemingly stochastic activity to naturally generate a frequency/novelty/importance-weighted inversion of the overall generative model.
This inversion may generate predictions of varying degrees of coherence, which while non-lucid would be free from directed-entrainment/influence/enslavement by PFC as occurs during directed attention/imagination while awake/aware.
When coherence is reduced in PFC while asleep (or only partially awake), brains/minds may find themselves unable to use meta-cognition and temporally-deep modeling, and so they are unable to successfully predict inputs from posterior cortices. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3524543/
This may be similar to the conditions encountered when fatigued and/or confused and/or hypnotized while awake. Under such conditions, it may be that the proper thing for PFC to do is to listen, rather than trying to control, so enslaving the PFC present bottom-up inputs.
However, if the PFC is given time-space to adapt and acquire successfully predictive dynamics, gradients of reducing surprise should result in increased dopamine, contributing to disinhibition, potentially contributing to increased PFC control-energy/directional-entrainment.
Serotonergic signaling may have opposite effects, both directly suppressing dopaminergic pathways as well as increasing the power from posterior cortices. At extremes, this could cause hallucinations, or a tendency to "dream while awake."
@RCarhartHarris https://twitter.com/adamsafron/status/1284990554334470144
@RCarhartHarris https://twitter.com/adamsafron/status/1284990554334470144
"Individuals high in the personality trait Openness to Experience appear to engage spontaneously (during wake) in processes of elaborative encoding similar to those Llewellyn identifies in both dreaming and the ancient art of memory (AAOM)..." https://pubmed.ncbi.nlm.nih.gov/24304755/
(continued from above)
"...Links between Openness and dreaming support the hypothesis that dreaming is part of a larger process of cognitive exploration that facilitates adaptation to new experiences."
(potentially consistent article linked below) https://www.nature.com/articles/srep10964
"...Links between Openness and dreaming support the hypothesis that dreaming is part of a larger process of cognitive exploration that facilitates adaptation to new experiences."
(potentially consistent article linked below) https://www.nature.com/articles/srep10964
Dreaming styles/tendencies as reflecting (and mediating?) evolutionary life history strategies? https://twitter.com/adamsafron/status/1287469641527328768?fbclid=IwAR3C9_a87k2RY2hm8d_RhH5kmgv1zBqvPS1jUN-k5fxL9ffa6SjOKUg2H54
