Lateral excitation: Notes on computational neuroscience
-
Differentiating scalar functions of matrices in 22 easy steps of Mathematica.
I frequently need to differentiate loss functions with respect to matrices. I usually do this manually, which can be time-consuming and error-prone. Therefore I wanted to see if I could use Mathematica to compute these symbolic derivatives automatically. Mathematica does not have such functionality built in, but ChatGPT suggested achieving it using pattern matching. The…
-
The closest rotation to a scaling
What is the closest rotation to a pure scaling? Intuitively, it should be the null rotation, the identity. One way to see this might be to consider starting with the identity scaling, for which it’s clearly true. If we then scale along one of the coordinate axes, there doesn’t seem to be any ‘torque’ that…
-
Wrangling quartics
The problem we have is to minimize $${1\over 2} \|A ^T Z^T Z A – B \|_F^2 + {\lambda \over 2} \|Z – I\|_F^2,$$ where $B$ is symmetric. This is quartic over $Z$ so its solution will be the roots of some cubic function of $Z$, which is likely intractable. However, this problem might have…
-
Nearest orthonormal matrix
We sometimes need to find the nearest orthonormal matrix $Q$ to a given square matrix $X$. This can easily be done using Lagrange multipliers, as I’ll show here. By ‘nearest’ we will mean in the element-wise sum-of-squares sense. This is equivalent to the squared Frobenius norm, so our optimization problem is $$ \argmin_Q \; {1…
-
Estimating the intrinsic dimensionality of data with the participation ratio
Many datasets are samples of the values of a given set of $N$ features. We can visualise these data as points in an $N$-dimensional space, with each point corresponding to one of the samples. Visualization encourages geometric characterization. A basic geometric property is dimensionality: what is the dimension of the space in which the data…
-
The nearest dataset with a given covariance
When analyzing neural activity I sometimes want to find the nearest dataset to the one I’m working with that also has a desired covariance $\Sigma$. In this post I’m going to show how to compute such a dataset. Let the dataset we’re given be the $m \times n$ matrix $X$, and the new dataset that…
Got any book recommendations?