Tag: weighted least squares

In Section 4.3.3 of his “Pattern Recognition and Machine Learning,” Bishop discusses the application of iterative reweighted least squares (IRLS) to maximum likelihood estimation of the parameters in logistic regression. The expression we get after computing the Newton-Raphson update of the parameters is $$ \ww^\text{(new)} = (\bPhi^T \RR \bPhi)^{-1} \bPhi^T \RR \zz,$$ where $$ \zz…