Emergence, Intrinsic Structure of Information, and Agenthood
Adaptive Systems Research Group, School of Computer Science,
Last modified: August 15, 2006
The centrality of the concept of emergence for the understanding of
complex systems has been the topic of a numerous number of
publications, covering a spectrum from category-theoretic to
synergetic approaches (Rasmussen et al. 2001; Haken 2000).
A particular interest has been devoted to finding
information-theoretic formalizations of the notion of emergence, since
the universality of information theory lends itself to a wide area of
applicability. Ideally, the notion of emergence would not have to be
considered ``in the eye of the beholder" (Harvey 2000), but would
arise naturally from the dynamics of the system.
Such a notion of emergence in time series has been developed in
(Crutchfield 1994); Shalizi 2001), based on the epsilon-machine
concept: a process emerges from another one if it has a greater
predictive efficiency than the second. Stated informally, this means
that, the ratio between prediction information (excess entropy) and
the complexity of the predicting epsilon-machine is better in the
emerging process. This fits smoothly into the perspective that
emergence should represent a higher-level coarse-grained view or
simplification of a more intricate fine-grained system dynamics.
A related, but different view is taken in the emergent description
model from (Polani 2004). Here, as above, the predictivity of a
time series is measured by mutual information, but we considered a
memoryless system, unlike above work using the epsilon machine
modeling maximal causal histories. The main difference, however, is
that we considered a decomposition of the total system into individual
independent informational submodes, thus providing a more intricate
picture about intrinsic structure the informational dynamics of the
total system. In (McGregor and Fernando 2005), higher-level
prediction models for partial aspects of a systems are suggested,
based on entropy measures, which can be interpreted as a simplified
version of the model from (Shalizi 2001) as it does not consider
causal states, or of our model as it does not require a full partition
into independent modes. On the other hand, a decomposition philosophy
for dynamical hierarchies into modes similar to ours, but based on
smooth mappings instead of information theory is taken in (Jacobi
Armed with above insights, here we wish to suggest some insightful
extensions of the picture and interpretation of emergent descriptions.
1. the emergent descriptions model can be sought by e.g.
Multiobjective Evolutionary Search algorithms that can optimize
for the several criteria of emergent descriptions (completeness,
predictivity and independence). In general, the criteria cannot be
simultaneously fulfilled, so a solution realizes a tradeoff which
can be picked at the Pareto front of the search.
2. Instead of considering a memoryless process, the individual
``coordinates" or modes can be equipped by an epsilon-machine
like extension, as to accomodate possible memory effects.
3. Instead of point 2, on the other hand, it is possible to
search for suitable inputs from other modes that would help in the
prediction of the next state. That induces a natural hierarchy in
the different modes, not unlike the algebraic model of semigroup
decomposition into ``coordinates" suggested in (Nehaniv 1997)
where subordinate modes feed into higher-level modes.
4. In (Klyubin et al. 2004) it has been shown that the
perception-action loop of an agent acting in an environment can be
modeled in the language of information. This is particularly
interesting for above considerations, as the agent/environment
system is a generalization of a time series (a time series can be
considered an agent without without the ability to select an
action, i.e. without the capacity for ``free will").
Using infomax principles, above agent/environment system can be
shown to structure the information flows into partly decomposable
information flows, a process that can be interpreted as a form of
concept formation. This gives a new interpretation for the
importance of emergence as the archetypical mechanism that allows
the formation of concept in intelligent agents and is thus perhaps
a key driving the creation of complexity in living systems.
Crutchfield, J. P. (1994). The calculi of emergence: Computation,
dynamics, and induction. Physica D, pages 11-54.
Haken, H. (2000). Information and Self-Organization. Springer Series
in Synergetics. Springer.
Harvey, I. (2000). The 3 es of artificial life: Emergence, embodiment
and evolution. Invited talk at Artificial Life VII,
1.-6. August, Portland.
Jacobi, M. N. (2005). Hierarchical organization in smooth dynamical
systems. Artificial Life, 11(4):493-512.
Klyubin, A. S., Polani, D., and Nehaniv, C. L. (2004). Organization
of the information flow in the perception-action loop of
evolved agents. In Proceedings of 2004 NASA/DoD Conference on
Evolvable Hardware, pages 177-180. IEEE Computer Society.
McGregor, S. and Fernando, C. (2005). Levels of description: A novel
approach to dynamical hierarchies. Artificial Life,
Nehaniv, C. L. (1997). Algebraic models for understanding: Coordinate
systems and cognitive empowerment. In J. P. Marsh,
C. L. Nehaniv, B. G., editor, Proceedings of the Second
International Conference on Cognitive Technology: Humanizing
the Information Age, pages 147-162. IEEE Computer Society
Polani, D. (2004). Defining emergent descriptions by information
preservation. In Proc. of the International Conference on
Complex Systems. NECSI. Long abstract, full paper under
review in InterJournal.
Rasmussen, S., Baas, N., Mayer, B., Nilsson, M., and Olesen, M. W.
(2001). Ansatz for dynamical hierarchies. Artificial Life,
Shalizi, C. R. (2001). Causal Architecture, Complexity and
Self-Organization in Time Series and Cellular Automata. PhD
thesis, University of Wisconsin-Madison.