Since you asked, yes, I will write a thread about the SVD.

The SVD solves the optimization problem in the screenshot (X is nxp).

The k columns of U and V are the 1st k left and right singular vectors of X, and the diagonal elements of D are the 1st k singular values

1/4 https://twitter.com/DavidSabatini2/status/1350592194936270848

"Nonnegative matrix factorization" is a bit broad but it is most often used to refer to the optimization problem in this screenshot, which -- as you can see -- is like a distant cousin of the SVD. Sometimes an orthogonality constraint may be placed on the columns of V.

2/4

Unlike the SVD, which has a unique solution (up to sign flips of a given left/right singular vector pair), the NNMF isn't guaranteed to be unique (!!!!!??!!) ... see e.g. https://web.stanford.edu/~vcs/papers/NMFCDP.pdf

3/4

The SVD has a lot of magical connections to fundamental ideas in statistics & linear algebra (and I enumerated some of those connections here https://twitter.com/WomenInStat/status/1285610321747611653?s=20) and I definitely don't mean to throw shade, but NNMF is *not* magical.

4/4

Oops... important detail missed ... in the optimization problems above, of course you need to MINIMIZE (w.r.t. U,D,V for the SVD, and w.r.t U,V for the NNMF)

5/4

@DavidSabatini2 @KenjiEricLee @govindgnair