Read on Twitter
Happy that our paper on “Tracking emotions: intrinsic motivation grounded on multi-level prediction error dynamics” has been accepted for presentation at @ICDL_EpiRob! https://arxiv.org/abs/2007.14632 Co-authored with @CIRIApsico @quitze, #MSCA @PredictiveRobot @SantAnnaPisa @uaemorelos
We propose a learning architecture that generates exploratory behaviours towards self-generated goals in a simulated robot, and that regulates goal selection and the balance between exploitation and exploration through a multi-level monitoring of prediction error dynamics.
The system is made of: a http://Conv.AE for unsupervised learning of low-dim. features from visual inputs; a self-organising map for online learning of visual goals; two deepNN, trained on line, encoding controller and predictor of the system. Memory replay is employed
A multi-level monitoring mechanism keeps track of two errors: (1) a high-level, general error of the system, i.e. MSE of the forward model calculated on a test dataset; (2) low-level goal errors, i.e. the prediction errors estimated when trying to reach each specific goal.
The system maintains a buffer of high-level MSE observed during a specific time window. After every update of the MSE buffer, a linear regression is calculated on the stored values over time, whose slope indicates the trend of the general error of the system.
This trend modulates computational resources (size of goal error buffers) and exploration noise: when overall performances improve, the necessity of tracking the goal error dynamics is reduced. On the contrary, the system widens the time window on which goal errors are monitored.
We discuss the tight relationship that PE dynamics may have with the emotional valence of action. PE dynamics may be fundamental cause of emotional valence of action: positive valence linked to an active reduction of PE and a negative valence to a continuous increase of PE.
