We introduce an approach to inferring the causal architecture of stochastic dynamical systems that extends rate-distortion theory to use causal shielding—a natural principle of learning. We study two distinct cases of causal inference: optimal causal filtering and optimal causal estimation. Filtering corresponds to the ideal case in which the probability distribution of measurement sequences is known, giving a principled method to approximate a system’s causal structure at a desired level of representation. We show that in the limit in which a model-complexity constraint is relaxed, filtering finds the exact causal architecture of a stochastic dynamical system, known as the causal-state partition. From this, one can estimate the amount of historical information the process stores. More generally, causal filtering finds a graded model-complexity hierarchy of approximations to the causal architecture. Abrupt changes in the hierarchy, as a function of approximation, capture distinct scales of structural organization. For nonideal cases with finite data, we show how the correct number of the underlying causal states can be found by optimal causal estimation. A previously derived model-complexity control term allows us to correct for the effect of statistical fluctuations in probability estimates and thereby avoid overfitting.
Skip Nav Destination
Article navigation
September 2010
Research Article|
September 28 2010
Optimal causal inference: Estimating stored information and approximating causal architecture
Susanne Still;
Susanne Still
a)
1Information and Computer Sciences,
University of Hawaii at Mānoa
, Honolulu, Hawaii 96822, USA
Search for other works by this author on:
James P. Crutchfield;
James P. Crutchfield
b)
2Department of Physics and Complexity Sciences Center,
University of California at Davis
, One Shields Avenue, Davis, California 95616, USA
3
Santa Fe Institute
, 1399 Hyde Park Road, Santa Fe, New Mexico 87501, USA
Search for other works by this author on:
Christopher J. Ellison
Christopher J. Ellison
c)
2Department of Physics and Complexity Sciences Center,
University of California at Davis
, One Shields Avenue, Davis, California 95616, USA
Search for other works by this author on:
a)
Electronic mail: sstill@hawaii.edu.
b)
Electronic mail: chaos@cse.ucdavis.edu.
c)
Electronic mail: cellison@cse.ucdavis.edu.
Chaos 20, 037111 (2010)
Article history
Received:
June 21 2010
Accepted:
August 23 2010
Citation
Susanne Still, James P. Crutchfield, Christopher J. Ellison; Optimal causal inference: Estimating stored information and approximating causal architecture. Chaos 1 September 2010; 20 (3): 037111. https://doi.org/10.1063/1.3489885
Download citation file:
Sign in
Don't already have an account? Register
Sign In
You could not be signed in. Please check your credentials and make sure you have an active account and try again.
Sign in via your Institution
Sign in via your InstitutionPay-Per-View Access
$40.00