Read on Twitter
Here's an interesting discussion on information-based vs. dynamics-based approaches to network inference and my thead on what I learned from it 
https://twitter.com/dallastaylordrt/status/1335055931332288514
https://twitter.com/dallastaylordrt/status/1335055931332288514
To recap, Dane argues in favour of information-based (i.e., (dynamical-)model-free) approaches. Manlio and Tiago argue that such approaches lead to many false positives. Dane retorts with amazing accuracy values from using causation entropy his paper (Sun et al). How can this be?
Despite Dane contrasting causal inference and system identification, I think the key is that he and Manlio/Tiago assume different settings. Sun et al. assumed that a system whose dynamics can be modelled by a Markov process. Is that are weird thing to assume? I don't think so.
In fact, as someone who has dedicated many years to the Ornstein-Uhlenbeck process, I believe that we can learn a lot from stationary Markov processes. A whole lot of physics and applied mathematics can be summarized as studying small perturbations in systems at stationary state.
So for a stationary Markov process, Sun et al. (Dane's paper) demonstrated that one does not need to assume a correct governing differential equation for the system dynamics to accurately infer the system's network structure.
But Manlio and Tiago are right to be skeptical of information-based approaches. The word 'model-free' gets thrown around a lot and evokes misplaced confidence in methods that are, strictly speaking, only guaranteed to work on stationary Markov processes.
And while stationary Markov processes are quite common in theoretical studies, it's not so clear for which real-world systems stationary Markov processes are good models. (Take that, 'model-free'!)
I work on neural systems for which it is necessary to assume that they are nonlinear and they almost constantly get input from the outside world. If your brain is ever stationary and Markovian, you are probably *very* dead.
For non-stationary systems (Markovian or not), I don't know of any method that is proven to accurately infer networks without making assumptions about the dynamics. For these systems, I agree with Manlio and Tiago that we cannot infer structure while ignoring dynamics.
Does that mean that information-based methods are not useful in practice? No. Many data sets are observations of non-stationary systems, but sometimes these systems are very close to being stationary (e.g. things like quasi stationarity)
or we can preprocess data in a way that filters non-stationary aspects of a data set (e.g., removing a moving average) that leads to a data set that looks like it could come from an almost stationary system.
Then again, to know what kind of preprocessing will give us the data that we need to accurately infer an underlying network, it helps a lot to know the dynamics that govern the system...
My two take-aways: (1) When two people argue, it's quite possible that both a right within some frame of reference. (2) When it comes to network inference, what works and what doesn't depends a lot on what we assume about the system that we are looking at.
Some questions to ask about a system before deciding what inference methods are promising: Is it (almost) stationary? Is it Markovian? Is time discrete or continuous? Do variables take discrete values or continuous values? Can edges have different edge weights?
And some practical questions about the data: How many features (i.e., nodes) and how many observations do I get? How noisy are they and where does the noise come from? Can I assume that there are no features missing from the data?
I honestly think that the network-inference literature would be a lot easier to navigate if people would always be very clear (in the abstract already!) about the setting that they consider. Is this the mathematician speaking? Maybe!
But if assumptions are explained well, it can really help with understanding. The paper by Sun et al. is one that I found very easy to understand because it was very clear about its assumptions. And that helped me make sense of the discussions I've been following here on Twitter.
