{"id":8730,"date":"2026-11-10T08:25:53","date_gmt":"2026-11-10T08:25:53","guid":{"rendered":"https:\/\/sinatootoonian.com\/?p=8730"},"modified":"2026-11-10T10:16:11","modified_gmt":"2026-11-10T10:16:11","slug":"neurons-as-detectors-of-coherent-sets","status":"publish","type":"post","link":"https:\/\/sinatootoonian.com\/index.php\/2026\/11\/10\/neurons-as-detectors-of-coherent-sets\/","title":{"rendered":"Neurons as Detectors of Coherent Sets"},"content":{"rendered":"\n

These are my notes on some of the content in this paper by Pughe-Sanford and colleagues<\/a>, supplemented by these background materials provided by Claude<\/a>.<\/em><\/p>\n\n\n\n

Sensory systems must distill the complex dynamics of environments into useful information. What aspect of these dynamics are worth reporting? <\/p>\n\n\n\n

Deterministic dynamics can usefully partition state space into invariant sets<\/em> – subsets of the state space which the dynamics map into themselves. Invariant sets have the property that if a point starts inside one, it will remain inside it, and if it starts outside one, it will remain outside it, and both of these conditions will hold for all time.<\/p>\n\n\n\n

Identifying invariant sets is useful because they’re predictive<\/em>: knowing that the system is in the subset at some time $t$ guarantees that the system will stay in that subset in the future (though its exact future location in the subset might be harder to determine). They’re also retrospective<\/em>: knowing that a system is in the set guarantees that it’s been in the set for its entire history.<\/p>\n\n\n\n

Real environmental dynamics are rarely deterministic, and are usually stochastic. This stochasticity makes strict invariance less useful because noise can knock a point outside of a set, violating the invariance requirements.<\/p>\n\n\n\n

Coherent<\/em> sets <\/em>are a relaxation of invariant sets more suitable for stochastic dynamics. They relax the membership requirements of invariant sets in two ways:<\/p>\n\n\n\n

    \n
  1. They replace strict set membership with high-probability set membership.<\/li>\n\n\n\n
  2. Membership is only evaluated over a finite time horizon $\\tau$.<\/li>\n<\/ol>\n\n\n\n

    We therefore say that the set $A$ at time $t$ is coherent<\/em> with the set $B$ at time $t + \\tau$ if the points in $A$ tend to end up in $B$, and points not in $A$ ( i.e. in the complement of $A$, $A^c$) don’t end up in $B$ (i.e. end up in $B^c$).<\/p>\n\n\n\n

    We can express this in terms of probabilities as $$ \\text{coh}(A, B) \\triangleq P(X_{t + \\tau} \\in B | X_t \\in A) + P(X_{t+\\tau} \\in B^c | X_t \\in A^c).$$<\/p>\n\n\n\n

    This is easy enough to write because all of the dynamics of the system are contained in the conditional probabilities.<\/p>\n\n\n\n

    Similar to invariant sets for deterministic dynamics, partitioning the input space into coherent sets would be a useful operation for sensory systems to perform because it would inform about either the past or future state of the environment. <\/p>\n\n\n\n

    For example, if two sets $A$ and $B$ are coherent over some time interval $\\tau$, then identifying that the sensory system is in $A$ is useful because that indicates that it will soon ($\\tau$) likely be in $B$. Similarly, identifying that it is in $B$ means that it was likely to have recently been in $A$.<\/p>\n\n\n\n

    Computing Maximally Coherent Sets<\/h2>\n\n\n\n

    We’d now like to find maximally coherent sets. This seems hard to do from the definition above. But we’ll now see that we can express coherence in terms of some familiar quantities from linear algebra. This will then suggest a relaxation of the problem that will be readily computable by neurons.<\/p>\n\n\n\n

    To see the connection to linear algebra, let’s focus on the first conditional probability above. We first express it as the ratio of the joint distribution and the prior, $$ P(X_{t+\\tau} \\in B | X_t \\in A) = {P(X_{t + \\tau} \\in B, X_t \\in A) \\over P(X_t \\in A)}.$$<\/p>\n\n\n\n

    To compute the prior distribution, let the probability over states at time $t$ be given by $\\mu(X_t)$. The prior is then $$ P(X_t \\in A) = \\int_{x \\in A} \\mu(x) dx.$$<\/p>\n\n\n\n

    This is an integral over $A$. We can instead express it as an integral over the whole space by using an indicator function $1_A(x)$, which is $1$ when $x \\in A$ and 0 otherwise. Our prior becomes $$ P(X_t \\in A) = \\int 1_A(x) \\mu(x) dx.$$ <\/p>\n\n\n\n

    We can now express this as a weighted inner product. Recall that the weighted inner product of two functions simply applies a weight at each point along the product. So, given two functions $a(x)$ and $b(x)$, their inner product weighted by $w(x)$ is $$ \\langle a, b \\rangle_w = \\int a(x) b(x) w(x) dx.$$<\/p>\n\n\n\n

    We want to express our prior as such a weighted inner product. But the integral above has three terms in the integrand: the two functions being inner-producted, and the weighting function. Our expression for the prior only has two.<\/p>\n\n\n\n

    To produce the required third term in the integrand, we note that $1_A(x)^2 = 1_A(x)$. We can then write our prior probability as $$ P(X_t \\in A) = \\int 1_A(x)^2 \\mu(x) dx = \\int 1_A(x) 1_A(x) \\mu(x) dx = \\langle 1_A, 1_A \\rangle_\\mu.$$ This says that the probabilty that $X_t$ is in $A$ is the size of the set $A$, weighted by the occupancy of the dynamics at time $t$.<\/p>\n\n\n\n

    The joint distribution is a bit more interesting: $$ P(X_t \\in A, X_{t + \\tau} \\in B) = \\int_{X_t \\in A} \\int_{X_{t+\\tau} \\in B} P(X_t, X_{t+\\tau}) \\, dX_t dX_{t+\\tau}.$$ Let’s first write it in terms of indicators.
    $$ \\dots = \\int_{X_t} 1_A(X_t) \\int_{X_{t+\\tau}} 1_B(X_{t+\\tau}) P(X_{t+\\tau}, X_t) \\, dX_t dX_{t+\\tau}. $$<\/p>\n\n\n\n

    Let’s now write this “dynamically,” in terms of where a point at time $t$ ends up at time $t + \\tau$. Recalling that $P(X_t) = \\mu(X_t)$, \\begin{align*} \\dots &= \\int_{X_t} 1_A(X_t) \\int_{X_{t+\\tau}} 1_B(X_{t+\\tau}) P(X_{t+\\tau}|X_t) \\mu(X_t) \\, dX_t dX_{t+\\tau} \\\\ &= \\int_{X_t} 1_A(X_t) \\mu(X_t) \\left[\\int_{X_{t+\\tau}} P(X_{t+\\tau}|X_t) 1_B(X_{t+\\tau}) dX_{t+\\tau}\\right] dX_t \\\\ &= \\int_{X_t} 1_A(X_t) \\mu(X_t) \\, \\EE (1_B(X_{t+\\tau}) | X_t) dX_t. \\end{align*}<\/p>\n\n\n\n

    We now define the Stochastic Koopman Operator<\/strong>, $\\mathcal K$. This operator has three slots:<\/p>\n\n\n\n

      \n
    1. A function $\\color{blue}f\\color{black}$, that maps each state to a scalar;<\/li>\n\n\n\n
    2. A state, $\\color{red}x\\color{black}$, at time $t$;<\/li>\n\n\n\n
    3. A time interval, $\\color{orange}\\tau\\color{black}$.<\/li>\n<\/ol>\n\n\n\n

      The operator takes the function $f$ and reports its average value at time $t+\\tau$, conditioned on the state at time $t$: $$ \\mathcal{K}_{\\color{orange}\\tau\\color{black}} \\color{blue}f\\color{black} (\\color{red}x\\color{black}) \\triangleq \\EE ( f(X_{t+\\tau}) | X_t = x) = \\int f(X_{t+\\tau}) P(X_{t+\\tau} | X_t = x) \\, dX_{t+\\tau}. $$<\/p>\n\n\n\n

      If we think of functions on $X$ as observables or measurements, we can imagine that our measuring device reports the average value over all states at a given time. <\/p>\n\n\n\n

      The SKO then gives us the future value of the measurement, conditioned on where we are now.<\/p>\n\n\n\n

      We can also view the SKO as defining a new function, $\\mathcal K_\\tau f$, which takes the state at time $t$ as input, maps it forward to time $t + \\tau$, and reports the expected value of $f$ over those future states.<\/p>\n\n\n\n

      We can write our joint probability in terms of the SKO as \\begin{align*} P(X_t \\in A, X_{t + \\tau} \\in B) &= \\int_{X_t} 1_A(X_t) \\mu(X_t) \\, \\EE (1_B(X_{t+\\tau}) | X_t) dX_t \\\\ &= \\int_{X_t} 1_A(X_t) \\mu(X_t) \\mathcal{K}_\\tau 1_B(X_t) dX_t\\\\ &= \\langle 1_A, \\mathcal{K}_\\tau 1_B \\rangle_\\mu, \\end{align*} and we see again that this probability is also a weighted inner product.<\/p>\n\n\n\n

      Putting the pieces together, and applying these results analogously to the complementary sets, we get an expression for coherence in terms of inner products, $$ \\text{coh}(A,B) = {\\langle 1_A, \\mathcal K 1_B \\rangle_\\mu \\over \\langle 1_A, 1_A \\rangle_\\mu} + {\\langle 1_{A^c}, \\mathcal K 1_{B^c} \\rangle_\\mu \\over \\langle 1_{A^c}, 1_{A^c} \\rangle_\\mu}. $$<\/p>\n\n\n\n

      These quantities seem easier to work with than the raw probabilities – but we’ve simply re-expressed the problem in terms of the SKO – and we’re still left with searching over subsets $A$ and $B$ of the state space. How does this latest expression help?<\/p>\n\n\n\n

      To see how, we’re going to look at a relaxation of the problem. From here on, we’ll assume that the dynamics are stationary, so that both $X_t$ and $X_{t+\\tau}$ are distributed as $\\mu$.<\/p>\n\n\n\n

      Since we’re looking for coherent sets, we expect that the probability of $A$ and $B$ are the same, so $P_{t}(A) = P_{t+\\tau}(B) = p$. If we then define the mean-subtracted indicators $f = 1_B – p$, $g = 1_A – p$, it turns out that our coherence formula becomes $$ \\text{coh}(A,B) = 1 + {\\langle g, \\mathcal{K} f \\rangle \\over \\|f\\| \\|g\\|}.$$<\/p>\n\n\n\n

      Therefore, maximizing coherence means maximizing the second term over $f$ and $g$. These are supposed to be mean-subtracted indicators, which are no easier to work with than the original indicators. <\/p>\n\n\n\n

      Instead, we notice that if we were free to maximize that second term over all<\/em> $f$ and $g$, it would correspond to finding left and right singular function of $\\mathcal K$. So, we’ll relax the problem and find these singular functions, and then convert them to (approximate) indicators for the coherent sets $A$ and $B$.<\/p>\n\n\n\n

      And that is exactly what they propose neurons are doing: computing these singular functions on the state space, and then thresholding them to indicate when the system is in a coherent set.<\/p>\n\n\n\n

      It also indicates that neurons can be split into predictive<\/strong> and retrospective<\/strong> neurons, corresponding to the two time points, or equivalently the left and right singular functions. The predictive neurons compute $g(x)$, indicating that the state space has arrived in a coherent set all of whose elements have the same future. Retrospective neurons compute $f(x)$, indicating a common past.<\/p>\n\n\n\n

      Detecting Coherent Sets in Data<\/h2>\n\n\n\n

      How would neurons actually compute these singular functions?<\/p>\n\n\n\n

      We see from the relaxation that we need to maximize $\\langle g, \\mathcal K f \\rangle.$ We saw from our earlier derivation that this corresponds to computing $ \\mathbb E ( g(X_t) f(X_{t+\\tau})), $ where the expectation is relative to the joint distribution $P(X_t ,X_{t+\\tau})$. <\/p>\n\n\n\n

      Now suppose the inputs to our neurons are a linear combination of some fixed features of the input. In particular, suppose one neuron receives a linear combination of these features evaluated at the current time, so $$ g(X_t) = \\sum_i \\phi_i(X_t) g_i = \\Phi(X_t)^T \\mathbf g,$$ where $g_i$ are coefficients.<\/p>\n\n\n\n

      Let’s suppose we have another neuron, which takes the value of the features at an interval $\\tau$ in the future. (Obviously neurons can’t see in the future so presumably this neuron would actually be looking at the present and the neuron we defined above would be looking at an interval $\\tau$ in the past).<\/p>\n\n\n\n

      The future neuron would compute its own linear combination of the features, weighted by $f_i$. So $$ f(X_{t+\\tau}) = \\sum_i \\phi_i(X_{t+\\tau}) f_i = \\Phi(X_{t+\\tau})^T \\mathbf f.$$<\/p>\n\n\n\n

      Then the expectation above is $$ \\mathbb E(g(X_t) f(X_{t+\\tau})) = \\langle \\mathbf g^T \\Phi(X_t) \\Phi(X_{t+\\tau})^T \\mathbf f\\rangle = \\mathbf g^T \\langle \\Phi(X_t) \\Phi(X_{t+\\tau})^T \\rangle \\mathbf f.$$<\/p>\n\n\n\n

      If we define the covariance of the input features with their future values as $$ \\Sigma_\\tau \\triangleq \\langle \\Phi(X_t) \\Phi(X_{t+\\tau})^T \\rangle,$$ so that $$ \\Sigma_0 = \\langle \\Phi(X_t) \\Phi(X_{t})^T \\rangle$$ is the feature covariance at time $t$, then the system’s job becomes maximizing $$ \\mathbf g^T \\Sigma_\\tau \\mathbf f,$$ subject to $$ \\mathbf f^T \\Sigma_0 \\mathbf f = \\mathbf g^T \\Sigma_0 \\mathbf g = 1.$$<\/p>\n\n\n\n

      If we now switch to whitened coordinates e.g. $\\hat \\ff = \\sqrt{\\Sigma_0} \\ff$, the problem becomes simply maximizing $$ \\hat \\bg^T \\Sigma_0^{-1\/2} \\Sigma_\\tau \\Sigma_0^{-1\/2} \\hat \\ff, \\quad \\text{subject to} \\|\\hat \\ff\\| = \\| \\hat \\bg \\| = 1.$$<\/p>\n\n\n\n

      This is Past-Future CCA, for which plausible neural implementations exist. Therefore, these neurons will adjust their feature weights according to past-future CCA until they arrive at the left and right singular vectors of the present-future covariance matrix, under the $\\Sigma_0$ constraints. Their drives (the weighted summation of their feature values) will then approximate the singular functions applied to the input, and by generating spikes when these drives are positive, the neurons will act as approximate detectors of coherent sets.<\/p>\n\n\n\n

      $$ \\blacksquare $$<\/p>\n","protected":false},"excerpt":{"rendered":"

      These are my notes on some of the content in this paper by Pughe-Sanford and colleagues, supplemented by these background materials provided by Claude. Sensory systems must distill the complex dynamics of environments into useful information. What aspect of these dynamics are worth reporting? Deterministic dynamics can usefully partition state space into invariant sets – […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-8730","post","type-post","status-publish","format-standard","hentry","category-blog"],"acf":[],"_links":{"self":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts\/8730","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/comments?post=8730"}],"version-history":[{"count":138,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts\/8730\/revisions"}],"predecessor-version":[{"id":9151,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts\/8730\/revisions\/9151"}],"wp:attachment":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/media?parent=8730"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/categories?post=8730"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/tags?post=8730"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}