Emergence, Intrinsic Structure of Information, and Agenthood
Daniel Polani
Adaptive Systems Research Group, School of Computer Science,
Full text:
PDF
Last modified: August 15, 2006
Abstract
The centrality of the concept of emergence for the understanding of
complex systems has been the topic of a numerous number of
publications, covering a spectrum from categorytheoretic to
synergetic approaches (Rasmussen et al. 2001; Haken 2000).
A particular interest has been devoted to finding
informationtheoretic formalizations of the notion of emergence, since
the universality of information theory lends itself to a wide area of
applicability. Ideally, the notion of emergence would not have to be
considered ``in the eye of the beholder" (Harvey 2000), but would
arise naturally from the dynamics of the system.
Such a notion of emergence in time series has been developed in
(Crutchfield 1994); Shalizi 2001), based on the epsilonmachine
concept: a process emerges from another one if it has a greater
predictive efficiency than the second. Stated informally, this means
that, the ratio between prediction information (excess entropy) and
the complexity of the predicting epsilonmachine is better in the
emerging process. This fits smoothly into the perspective that
emergence should represent a higherlevel coarsegrained view or
simplification of a more intricate finegrained system dynamics.
A related, but different view is taken in the emergent description
model from (Polani 2004). Here, as above, the predictivity of a
time series is measured by mutual information, but we considered a
memoryless system, unlike above work using the epsilon machine
modeling maximal causal histories. The main difference, however, is
that we considered a decomposition of the total system into individual
independent informational submodes, thus providing a more intricate
picture about intrinsic structure the informational dynamics of the
total system. In (McGregor and Fernando 2005), higherlevel
prediction models for partial aspects of a systems are suggested,
based on entropy measures, which can be interpreted as a simplified
version of the model from (Shalizi 2001) as it does not consider
causal states, or of our model as it does not require a full partition
into independent modes. On the other hand, a decomposition philosophy
for dynamical hierarchies into modes similar to ours, but based on
smooth mappings instead of information theory is taken in (Jacobi
2005).
Armed with above insights, here we wish to suggest some insightful
extensions of the picture and interpretation of emergent descriptions.
1. the emergent descriptions model can be sought by e.g.
Multiobjective Evolutionary Search algorithms that can optimize
for the several criteria of emergent descriptions (completeness,
predictivity and independence). In general, the criteria cannot be
simultaneously fulfilled, so a solution realizes a tradeoff which
can be picked at the Pareto front of the search.
2. Instead of considering a memoryless process, the individual
``coordinates" or modes can be equipped by an epsilonmachine
like extension, as to accomodate possible memory effects.
3. Instead of point 2, on the other hand, it is possible to
search for suitable inputs from other modes that would help in the
prediction of the next state. That induces a natural hierarchy in
the different modes, not unlike the algebraic model of semigroup
decomposition into ``coordinates" suggested in (Nehaniv 1997)
where subordinate modes feed into higherlevel modes.
4. In (Klyubin et al. 2004) it has been shown that the
perceptionaction loop of an agent acting in an environment can be
modeled in the language of information. This is particularly
interesting for above considerations, as the agent/environment
system is a generalization of a time series (a time series can be
considered an agent without without the ability to select an
action, i.e. without the capacity for ``free will").
Using infomax principles, above agent/environment system can be
shown to structure the information flows into partly decomposable
information flows, a process that can be interpreted as a form of
concept formation. This gives a new interpretation for the
importance of emergence as the archetypical mechanism that allows
the formation of concept in intelligent agents and is thus perhaps
a key driving the creation of complexity in living systems.
Bibliography
Crutchfield, J. P. (1994). The calculi of emergence: Computation,
dynamics, and induction. Physica D, pages 1154.
Haken, H. (2000). Information and SelfOrganization. Springer Series
in Synergetics. Springer.
Harvey, I. (2000). The 3 es of artificial life: Emergence, embodiment
and evolution. Invited talk at Artificial Life VII,
1.6. August, Portland.
Jacobi, M. N. (2005). Hierarchical organization in smooth dynamical
systems. Artificial Life, 11(4):493512.
Klyubin, A. S., Polani, D., and Nehaniv, C. L. (2004). Organization
of the information flow in the perceptionaction loop of
evolved agents. In Proceedings of 2004 NASA/DoD Conference on
Evolvable Hardware, pages 177180. IEEE Computer Society.
McGregor, S. and Fernando, C. (2005). Levels of description: A novel
approach to dynamical hierarchies. Artificial Life,
11(4):459472.
Nehaniv, C. L. (1997). Algebraic models for understanding: Coordinate
systems and cognitive empowerment. In J. P. Marsh,
C. L. Nehaniv, B. G., editor, Proceedings of the Second
International Conference on Cognitive Technology: Humanizing
the Information Age, pages 147162. IEEE Computer Society
Press.
Polani, D. (2004). Defining emergent descriptions by information
preservation. In Proc. of the International Conference on
Complex Systems. NECSI. Long abstract, full paper under
review in InterJournal.
Rasmussen, S., Baas, N., Mayer, B., Nilsson, M., and Olesen, M. W.
(2001). Ansatz for dynamical hierarchies. Artificial Life,
7:329353.
Shalizi, C. R. (2001). Causal Architecture, Complexity and
SelfOrganization in Time Series and Cellular Automata. PhD
thesis, University of WisconsinMadison.


