Agency

In modern cognitive science and artificial intelligence research, the concept of an "Agent" is often used intuitively but rarely defined strictly. What distinguishes a simple computer program from a "subject"?

This study proposes a unifying theory that views agency not as hard-coded software but as an emergent property. We demonstrate that agency inevitably arises in a system attempting to balance two fundamental needs: the drive to learn the new (Curiosity) and the drive to control the environment (Empowerment), forming Active Inference.

Part 1. Existing Definitions

In contemporary literature, various definitions of agency can be found.

1.1. Agency as Incomputability

The first definition concerns predictability. If an entity's behavior can be described by a short mathematical formula (for example, F = ma for a stone), we do not consider it an agent. It is simply an object obeying predictable laws of physics or an algorithm. It lacks freedom.

If an agent possesses true freedom of choice, its behavior cannot be compressed into a simple algorithm. Consequently, any autonomous agent capable of achieving arbitrary goals in a complex environment must be Turing complete.
To ensure incomputability, the Kolmogorov complexity (the amount of information required to describe the object) of the agent's life trajectory must grow linearly with time. This means the agent is constantly generating new information, creating a unique history that could not have been predicted in advance [1].

1.2. Agency as Relative Subjectivity

The second definition, proposed in works on cybernetics (e.g., by Watson), views the agent as an optimizer. It considers subjectivity as the system's ability to use knowledge about its parts to overcome short-term difficulties for the sake of long-term goals—that is, to plan and execute plans [2].

This can be described as the ability to withstand "tension" (a high value of the error gradient), escaping local optima (small joys) to find a global optimum (a major goal) [3].

In other words, we require an agent to be free from predictability and to possess will.

Here arises an Apparent Paradox:

If an agent is an ideal optimizer (according to Watson), it always acts rationally and predictably.
If it is predictable, it becomes computable (loses complexity).
Therefore, an ideal agent loses freedom (according to the first definition).

The paradox is resolved if we accept that an Agent is not simply its "source code" (DNA or neural network) but a Continuous Representation of accumulated experience. The agent's history, in this case, cannot be decomposed into Markov states, because in any single distinct state, the agent's actions would be predictable.

To predict the actions of such an agent, knowing the formula of its optimizer is insufficient. An observer needs to possess the entire database of its past experience. It is precisely this history that makes its behavior incomputable to an external observer while preserving the internal logic of optimization.

Part 2. Active Inference

Why does an agent do anything at all? The fundamental goal of any living system is Autopoiesis, maintaining its own existence and fighting against Entropy [4].

According to the theory of Active Inference, an organism must stay within a narrow range of acceptable states (homeostasis). Any deviation causes "surprise" (mathematically, Negative Log Likelihood).

To minimize surprise (to avoid death), an agent has two paths:

Change its model of the world (Perception): Acknowledge that the world has changed and update its knowledge.
Change the world to fit the model (Action): Perform an action so that reality matches expectations.

Usually, in AI, the Categorical Cross-Entropy loss function is used. It teaches the system well how to explain data (Perception) but, by itself, does not provide the motivation to actively change the world or seek new data. True agency requires additional incentives.

When two specific components are added to the survival equation's loss function formalizing incomputability and planning conditions for sustainable Self-Preservation are created according to Active Inference.

2.1. Curiosity — $α$

"I go where I don't know."

This is an intrinsic reward for prediction error. The agent specifically seeks situations where its model of the world performs poorly in order to improve it [5].

L_{C u r i o s i t y} = H (p, q) = - \sum_{x} p (x) \log q (x)

Where:

$p (x)$ is the actual probability distribution of the event occurring in the environment (Ground Truth). In a deterministic observation, this is 1 for the observed state and 0 for others.
$q (x)$ is the probability distribution predicted by the agent's internal model (Belief).

It promotes Generalization. In experiments, agents rewarded for curiosity learn to navigate levels (e.g., in games) they have never seen before faster than agents taught simply to "win." They develop a universal skill of exploration. This is particularly useful in environments with sparse rewards.

Via adding Generalization we automatically ensures that 1.2 becomes part of loss function, because this is let the model to achive lower minimums, closer to global ones.

2.2. Empowerment — $β$

"I go where I can do more."

This is the drive to maximize the information capacity of the control channel. The agent wants to be in a state where:

It has many options for action.
Each action has a predictable result.

Mathematically, Empowerment is calculated via Mutual Information [6] between a sequence of actions $A^{n}$ and the future state $S_{t + n}$ :

L_{E m p o w e r m e n t} = E = max_{p (a^{n})} I (A^{n}; S_{t + n})

$A^{n}$ : a sequence of $n$ actions $(a_{t}, a_{t + 1}, \dots, a_{t + n - 1})$ .
$S_{t + n}$ : the state of sensors after $n$ steps.
$I (A^{n}; S_{t + n})$ : the mutual information between actions and the result.
$max_{p (a^{n})}$ : we are looking for the probability distribution of actions that gives us maximum information (maximum channel capacity).

If we expand this via Entropy:

L_{E m p o w e r m e n t} = H (S_{n e x t}) - H (S_{n e x t} | A)

The agent strives to increase diversity ( $H (S_{n e x t})$ ) to have access to a large number of states.
The agent strives to decrease uncertainty ( $H (S_{n e x t} | A)$ ) so that control is reliable.

This is the evolutionary reason for the development of sensors and limbs: they expand the channel of influence on the world. Via adding Empowerment we sum up the loss to the Active Inference, implementing self-presence, needed for sustain Incomputability.

Part 3. Synthesis

Now we can formulate the final definition.

Agency arises naturally in a system that minimizes the following loss function:

L o s s = α \cdot L_{C u r i o s i t y} + β \cdot L_{E m p o w e r m e n t} + γ \cdot L_{M e s a}

In this system, a continuous dynamic conflict occurs:

Curiosity pushes the agent into the unknown, forcing its history (Kolmogorov complexity) to grow. This ensures incomputability and novelty.
Empowerment forces the agent to structure this chaos, turning it into manageable order. This ensures subjectivity (movement toward a goal).
In the case where the subject's behavior does not affect main loss components, we can say that in this specific behavior it is not acting as an agent, but is pursuing the goals of a mesa-optimizer(s).

As a result, we can define an Agent as a Continuous Representation, self-sustaining (autopoietic) through the balance between exploration (curiosity) and exploitation (control).

References:

[1] https://arxiv.org/pdf/2505.04646: Computational Irreducibility as the Foundation of Agency: A Formal Model Connecting Undecidability to Autonomous Behavior in Complex Systems

[2] https://pmc.ncbi.nlm.nih.gov/articles/PMC10920425/: Agency, Goal-Directed Behavior, and Part-Whole Relationships in Biological Systems

[3] https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/lehman_ecj11.pdf: Abandoning Objectives: Evolution through the Search for Novelty Alone

[4] https://www.frontiersin.org/journals/bioengineering-and-biotechnology/articles/10.3389/fbioe.2021.720652/full Goals as Emergent Autopoietic Processes

[5] https://arxiv.org/pdf/1705.05363: Curiosity-driven Exploration by Self-supervised Prediction

[6] https://uhra.herts.ac.uk/id/eprint/282/1/901241.pdf: Empowerment: A Universal Agent-Centric Measure of Control