{"id":6713,"date":"2026-02-26T09:15:47","date_gmt":"2026-02-26T09:15:47","guid":{"rendered":"https:\/\/sinatootoonian.com\/?p=6713"},"modified":"2026-03-02T07:22:15","modified_gmt":"2026-03-02T07:22:15","slug":"unary-vs-binary-expressions-of-independence","status":"publish","type":"post","link":"https:\/\/sinatootoonian.com\/index.php\/2026\/02\/26\/unary-vs-binary-expressions-of-independence\/","title":{"rendered":"Unary vs. Binary Expressions of Independence"},"content":{"rendered":"\n

When I think of independence my natural tendency is to consider it as a relation between two items, a binary relation. For example, a node of interest $a$ is independent of $b$ when $$p(a,b) = p(a) p(b).$$ But this expression is symmetric in $a$ and $b$. We could have equivalently said $b$ is independent of $a$, even though our node of interest is $a$.<\/p>\n\n\n\n

However, when learning about Markov blankets<\/a>, an equivalent but different ‘unary’ perspective was more useful. In that setting, we want to determine the minimal set, called the Markov boundary<\/em>, that makes a node conditionally independent of all others.<\/p>\n\n\n\n

My initial approach was to consider my node of interest $a$ and compare it to some node $b$ outside the conditioning set $C$ using the binary form of independence above. So, to show that $$p(a,b|C) = p(a|C) p(b|C).$$<\/p>\n\n\n\n

This approach put my node of interest $a$ and the comparison node $b$ on equal footing, even though I wasn’t really interested in the latter.<\/p>\n\n\n\n

Reading Section 8.2.2 of Bishop, it turned out to be much easier to use a different, ‘unary’ expression for independence. It’s a reformulation of the above that says $$ p(a|b) = {p(a,b) \\over p(b)} = p(a).$$<\/p>\n\n\n\n

This is an equivalent expression to the first one, but is ‘unary’ in that the focus remains on $a$ as the variable of interest, with $b$ given a distinct role as the conditioning set.<\/p>\n\n\n\n

This ‘unary’ notion was useful because it kept the focus mostly on my node of interest $a$, and the conditioning set. We expressed conditional independence as $$ p(a| \\text{all other nodes}) = p(a|C).$$ This expression was also more appropriate in that it didn’t single out a particular node to check independence against, since no such nodes should be distinguished.<\/p>\n\n\n\n

The ‘binary’ definitions seem like an undirected notion of independence, where the two nodes are considered symmetrically in a joint distribution, while the ‘unary’ definition is a directed notion, where one node conditions the other.<\/p>\n\n\n\n

Anyway, I think it’s interesting to note that these two mathematically equivalent expressions seem to have different semantic content.<\/p>\n\n\n\n

$$\\blacksquare$$<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

I discuss the unary and binary expressions of independence and how their meanings are slightly different.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1,152],"tags":[],"class_list":["post-6713","post","type-post","status-publish","format-standard","hentry","category-blog","category-post"],"_links":{"self":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts\/6713","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/comments?post=6713"}],"version-history":[{"count":8,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts\/6713\/revisions"}],"predecessor-version":[{"id":6951,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/posts\/6713\/revisions\/6951"}],"wp:attachment":[{"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/media?parent=6713"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/categories?post=6713"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sinatootoonian.com\/index.php\/wp-json\/wp\/v2\/tags?post=6713"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}