The left two panels show $\\wt Z_{UU}$ returned by the optimization, and the corresponding recurrent weights $\\wt W_{UU}$. I uses $\\wt Z_{UU}$ to start the iteration. The plot in the middle shows that, after some transient divergence, the iterates get closer together (blue) and converge to a solution (red). The solution, shown on the right, satisfies $\\Eqn{eqn:main}$ at numerical precision. It’s clear that the initial and terminal solutions are quite different. For example $\\wt Z_{UU}$ seems to have a lot of off-diagonal elements, whereas these are suppressed in $\\wt Z_{UU}^*$. The latter also seems to have its first element almost at zero. This seems well located to cancel the effect of the sandwiching $S^2$ terms.<\/p>\n\n\n\n
What makes $\\Eqn{eqn:main}$ hard to solve is that $Z$ shows up in two places: as a $Z^{-1}$ on the righthand side, and as $Z^2$ on the left. The problems we were seeing last time were due (I believe) to errors in our estimate being amplified through the term on the left. Ideally, we’d like to be able to ignore one or the other term to have a hope of solving this equation. However, when we use the optimization solution, the term on the right is a little bigger than the term on the left. So we have to consider both.<\/p>\n\n\n\n
Interestingly, using the exact solution, the contribution of the term containing $Z^2$ is much smaller than that of the term on the right, by about a factor of 60 when measured in matrix norm. <\/p>\n\n\n\n
Is this just a happy coincidence?<\/em><\/p>\n\n\n\nIf we drop the $Z^2$ term, we get $$ \\wt Z_{UU}^{-1} – I = \\wt W_{UU} \\approx -{S \\wt C_{VV} S \\over \\la’}.$$ This is in fact in good agreement with the true value:<\/p>\n\n\n\nThis is definitely not the case if we use the solution returned by the optimization:<\/p>\n\n\n\nWe are trying to understand the connectivity solutions $Z$ found when minimizing the objective $$ {1 \\over 2 n^2 } \\|X^T Z^T Z X – C\\|_F^2 + {\\la \\over 2 m^2}\\|Z – I\\|_F^2.$$ Recap We found in the previous post that solutions satisfy$$ {1 \\over \\la’} \\left(S^2 \\wt Z_{UU}^2 S^2 – S \\wt C_{VV} S […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1,30],"tags":[41,36,47,38,40],"class_list":["post-1402","post","type-post","status-publish","format-standard","hentry","category-blog","category-notes-blog","tag-approximation","tag-calculation","tag-damping","tag-matrix-inversion","tag-work"],"acf":[],"_links":{"self":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts\/1402","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/comments?post=1402"}],"version-history":[{"count":24,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts\/1402\/revisions"}],"predecessor-version":[{"id":1430,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts\/1402\/revisions\/1430"}],"wp:attachment":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/media?parent=1402"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/categories?post=1402"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/tags?post=1402"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"\"](\"https:\/\/sinatootoonian.com\/wp-content\/uploads\/2024\/02\/damped_updates-1-1024x558.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/sinatootoonian.com\/wp-content\/uploads\/2024\/02\/W_vs_W_-1024x566.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/sinatootoonian.com\/wp-content\/uploads\/2024\/02\/W_vs_W__-1024x566.png\")
<\/figure>\n","protected":false},"excerpt":{"rendered":"