Tag: pca
-
Maximum likelihood PCA
These are my derivations of the maximum likelihood estimates of the parameters of probabilistic PCA as described in section 12.2.1 of Bishop, and with some hints from (Tipping and Bishop 1999). Once we have determined the maximum likelihood estimate of $\mu$ and plugged it in, we have (Bishop 12.44)$$ L = \ln p(X|W, \mu, \sigma^2)…
-
Changing regularization
This morning it occurred to me that the problems we’re having with our equation \begin{align}S^2 Z^2 S^2 – S C S = \lambda (Z^{-1} – I)\label{main}\tag{1}\end{align} are due to the regularizer we use, $\|Z – I\|_F^2$. This regularizer makes the default behavior of the feedforward connections passing the input directly to the output. But it’s…
-
Estimating the intrinsic dimensionality of data with the participation ratio
Many datasets are samples of the values of a given set of $N$ features. We can visualise these data as points in an $N$-dimensional space, with each point corresponding to one of the samples. Visualization encourages geometric characterization. A basic geometric property is dimensionality: what is the dimension of the space in which the data…