Tag: pca

  • Changing regularization

    This morning it occurred to me that the problems we’re having with our equation \begin{align}S^2 Z^2 S^2 – S C S = \lambda (Z^{-1} – I)\label{main}\tag{1}\end{align} are due to the regularizer we use, $\|Z – I\|_F^2$. This regularizer makes the default behavior of the feedforward connections passing the input directly to the output. But it’s…

  • Estimating the intrinsic dimensionality of data with the participation ratio

    Many datasets are samples of the values of a given set of $N$ features. We can visualise these data as points in an $N$-dimensional space, with each point corresponding to one of the samples. Visualization encourages geometric characterization. A basic geometric property is dimensionality: what is the dimension of the space in which the data…