Bayesian cyclic networks, mutual information and reduced-order Bayesian inference

We examine Bayesian cyclic networks, here defined as complete directed graphs in which the nodes, representing the domains of discrete or continuous variables, are connected by directed edges representing conditional probabilities between all pairs of variables. The prior probabilities associated with each domain are also included as probabilistic edges into each domain. Such networks provide a graphical representation of the inferential connections between variables, and substantially extend the standard definition of “Bayesian networks”, usually defined as one-directional (acyclic) directed graphs. In a binary system, the proposed representation provides a graphical expression of Bayes’ theorem. In higher-dimensional systems, further probabilistic relations can be recovered from the network cyclic properties and the joint probability of all variables. In particular, adopting a Markovian assumption leads to the theorem that the mutual information between any pair of variables on the network must be equivalent. Analysis of a hybrid Bayesian cyclic network of two continuous and two discrete variables of the form of a commutative diagram – provides a framework for more computationally efficient reduced-order Bayesian inference, involving initial simplification by an order reduction process.