Read on Twitter
New paper!
For the last century, we’ve misunderstood something fundamental about unsupervised learning.
This paper fixes it.
https://arxiv.org/abs/2004.05387
For the last century, we’ve misunderstood something fundamental about unsupervised learning.
This paper fixes it.
https://arxiv.org/abs/2004.05387
Have you heard of Factor Analysis (FA)? People used to scream about it
In 1935, Thurstone ("the father of FA") proposed factor rotations. It made statisticians (like Charles Spearman) *super angry*. They said that factors are "rotationally invariant" and thus can’t be estimated
In 1935, Thurstone ("the father of FA") proposed factor rotations. It made statisticians (like Charles Spearman) *super angry*. They said that factors are "rotationally invariant" and thus can’t be estimated
While there is no more screaming about this today, the mood remains:
"Substantively, [rotational invariance] should be rather troubling. If we can rotate the factors as much as we like without consequences, how on Earth can we interpret them?" :-/
http://www.stat.cmu.edu/~cshalizi/uADA/15/lectures/19.pdf
"Substantively, [rotational invariance] should be rather troubling. If we can rotate the factors as much as we like without consequences, how on Earth can we interpret them?" :-/
http://www.stat.cmu.edu/~cshalizi/uADA/15/lectures/19.pdf
But the tools of FA (e.g. Varimax) work really well.
I use Varimax. It makes factors way more interpretable.
How can it work?? The factors are rotationally invariant!
And as far as they see, they can offer
No explanation
-Natalie
I use Varimax. It makes factors way more interpretable.
How can it work?? The factors are rotationally invariant!
And as far as they see, they can offerNo explanation
-Natalie
Well, in p>n regression, there are lots of solutions b that minimize ||y - Xb||. What do we do? We pick the sparse one!
In 1935, long before the Lasso, Thurstone used the same idea. He developed a know-how to find the sparse factors and resolve the rotational invariance.
In 1935, long before the Lasso, Thurstone used the same idea. He developed a know-how to find the sparse factors and resolve the rotational invariance.
This know-how passed on through generations of Psychologists. This know-how inspired Kaiser's Varimax (which is loaded into base R). This know how is embedded into the way that base R doesn't print the small factor loadings (because we think they might actually be zeros).
We show how sparsity resolves the rotational invariance of factor analysis.
It gets better.
We show that PCA + Varimax estimates a huge class of "semi-parametric" models: SBMs, topic models, NMF, ICA, etc
Filling seminar slots? I’d love zooming to you
https://arxiv.org/abs/2004.05387
It gets better.
We show that PCA + Varimax estimates a huge class of "semi-parametric" models: SBMs, topic models, NMF, ICA, etc
Filling seminar slots? I’d love zooming to you
https://arxiv.org/abs/2004.05387
