Tag: maximum likelihood
-
Understanding Expectation Maximization as Coordinate Ascent
These notes are based on what I learned from my first postdoc advisor, who learned it (I believe) from (Neal and Hinton 1998). See also section 4 of (Roweis and Ghahramani 1999) for a short derivation, and the broader discussion in Chapter 9 of Bishop, in particular Section 9.4 Introduction When performing maximum likelihood estimation…
-
Maximum likelihood PCA
These are my derivations of the maximum likelihood estimates of the parameters of probabilistic PCA as described in section 12.2.1 of Bishop, and with some hints from (Tipping and Bishop 1999). Once we have determined the maximum likelihood estimate of $\mu$ and plugged it in, we have (Bishop 12.44)$$ L = \ln p(X|W, \mu, \sigma^2)…
-
The inference model when missing observations
The inference model isn’t giving good performance. But is this because we’re missing data? In the inference model, the recorded output activity is related to the input according to $$ (\sigma^2 \II + \AA \AA^T) \bLa = \YY,$$where we’ve absorbed $\gamma$ into $\AA$. We can model this as $N$ observations of $\yy$ given $\bla$, where$$…