Tiling Without Omniscience Dynamizing Faith in Joint Argmax

Abstract

The tiling theorem of Demski, Hsia, and Rapoport proves that an Updateless Decision Theory 1.0 agent has no strict preference for self-modification, given a coherent prior and logical omniscience. In the present document the Faith in Joint Argmax assumption is weakened to a one-sided additive gap that, under iterative learning, vanishes in the Cesàro mean. All four places where omniscience is consumed in the tiling are reduced to a single condition on the inductor over a minimal subalgebra, and the result is reflectively stable. As an output, the condition preserves the independence of branch probabilities as a structural axiom, which can be dropped if one allows bounded optimality, replacing it with an internally verifiable domination condition. If global optimality is required, the agent is placed under an obligation to explore the environment, which removes the entire argmax machinery and yields tiling without structural assumptions about the value table, thereby removing omniscience.

Keywords: alignment research, tiling, updateless decision theory, logical induction, bounded rationality, reflective stability.

1. Introduction

Theorem 3 of DHR25 states: a UDT 1.0 agent has no strict preference for a self-modifying action $a_m\in A_m$ over the best non-self-modifying one. Formally, for each $o$ and each $a_m\in A_m$:

Throughout we adopt DHR25's notation:

• $O$ is the observation space,
• $A$ the action space,
• $A_o\subseteq A$ the actions available at $o$,
• $A_m\subseteq A$ the self-modifying actions,
• $A_{-m}=A\setminus A_m$ the non-self-modifying actions,
• $a_m\in A_m$, ${\hat{a}}_m\in A_{-m}$ denotes its externally indistinguishable non-self-modifying version,
• $(\mathrm{ob}(a_m),\mathrm{ac}(a_m))=(\overline{o},\overline{a})$ denotes the unique policy-point it forces by Limited Self-Modification (LSM; Assumption 3), and all expectations are taken with respect to the agent's subjective prior $p$,
• $⌜\cdot ⌝$ indicate a value substituted from outer context,
• $⌞\cdot ⌟$ indicate a free variable in the inner context,
• Utility is bounded, $u\in [0,1]$,
• The agent's policy is $\pi$,
• The optimal policy is ${\pi }^{\ast }$; ${\pi }^{\ast }(o)$ denotes the policy's action under observation $o$.

We work with a propositionally coherent logical inductor with true prices $\hat{Q}$, transformed into a Bayesian logical inductor with round prices $P_t$; the Boundedly Rational Inductive Agent (BRIA) round $t$ is identified with a step of deductive time. The agent's estimate at round $t$ is ${\hat{\mathbb{E}}}_t$, with ${\hat{\mathbb{E}}}_t\to {\mathbb{E}}_{\hat{Q}}$. Fix a non-decreasing budget $g$; an event is $g$-decidable if its truth value is decidable in $O(g(t))$ time.

We define:

• $w(a,a^{\prime }):={\mathbb{E}}_{\hat{Q}}[u\mid {\pi }^{\ast }(o)=a\wedge {\pi }^{\ast }(o^{\prime })=a^{\prime }]$
• $w_o(a):={\mathbb{E}}_{\hat{Q}}[u\mid {\pi }^{\ast }(o)=a]$
• $v(a^{\prime }):={\mathbb{E}}_{\hat{Q}}[u\mid {\pi }^{\ast }(o^{\prime })=a^{\prime }]$
• $v^{-}:=\underset{a\in A_{-m},a^{\prime }\in A}{max}w(a,a^{\prime })$

Cross-block conditional laws:

• $\rho (a^{\prime }\mid a):=P_{\hat{Q}}({\pi }^{\ast }(o^{\prime })=a^{\prime }\mid {\pi }^{\ast }(o)=a)$
• $\sigma (a\mid a^{\prime }):=P_{\hat{Q}}({\pi }^{\ast }(o)=a\mid {\pi }^{\ast }(o^{\prime })=a^{\prime })$

2. Dynamization

The proof proceeds along the chain: Fine-Grained Fairness (FGF) $\to$ Faith in Joint Argmax (FGJ) $\to$ Action Coordination (AC) $\to$ Knowledge of the Decision Procedure (KoDP). Each step consumes omniscience, since the agent must simultaneously not know the concrete action of its argmax and know its value. To remove this computational uncertainty, we check the robustness of the DHR25 chain to computational constraints. We reproduce the chain with explicit tracking of equalities and inequalities, since this determines where and how the assumption can be weakened. If the agent is not omniscient and makes errors in its estimates, we ensure that this error does not propagate to infinity as it passes through the steps of the proof.

• FGF states that self-modification does not change value if it is equivalent to a non-self-modifying action.
• FGJ links the current choice to the future argmax, guaranteeing that the agent believes in the consistency of its optimal decisions across different policy points.
• AC allows collapsing nested maximization operations into a single decision process.
• KoDP removes the need to predict the future argmax, allowing the agent to use knowledge of its own decision-making algorithm.

Formally, with $o$ fixed and $(\overline{o},\overline{a})=(\mathrm{ob}(a_m),\mathrm{ac}(a_m))$, ${\hat{a}}_m\in A_{-m}$:

Theorem 1.

Proof. Fix an arbitrary $a_m\in A_m$ under observation $o$, with $(\overline{o},\overline{a})=(\mathrm{ob}(a_m),\mathrm{ac}(a_m))$. Introduce the intermediate quantities

The chain is:

Here the equality $Y_0=Y_1$ is FGF; $Y_1\le Y_2$ — since ${\hat{a}}_m\in A_{-m}$; $Y_2\le Y_3$ — taking the maximum over the second condition ($Y_3$ is the right-hand side of FGJ); the step $Y_3\le X+\epsilon$ is $\epsilon$-FGJ in the direction $\mathrm{RHS}\le \mathrm{LHS}$; AC collapses the iterated double maximum inside the argmax into a single one ($X=X_{\mathrm{AC}}$); KoDP removes the condition on the argmax ($X_{\mathrm{AC}}=\underset{a\in A_{-m}}{max}{\mathbb{E}}_p(u\mid {\pi }^{\ast }(o)=a)$).

Note that FGJ is used exactly once, only on the lower side, and is sandwiched between exact equalities. The AC and KoDP steps after FGJ apply to $X$ and are exact equalities, i.e. any additive gap introduced at the FGJ step passes straight through without amplification.

The $\epsilon$-FGJ used here, at $\epsilon =0$, is exactly Assumption 5 of DHR25. $◻$

Thus we have shown that any error in knowing the future argmax is transferred additively to the final result. This does not guarantee that the agent will be able to achieve tiling at all. For the error not to be a permanent obstacle, it must tend to zero in the course of learning. Since $\overline{o}=\mathrm{ob}(a_m)$ varies with self-modification, to obtain "no self-modifying action at $o$ is preferable by more than $\epsilon$", the assumption must hold uniformly: $\epsilon =\underset{o^{\prime }}{sup}{\epsilon }_{o,o^{\prime }}$ over all relevant pairs.

We transfer the problem to the BRIA setting (Oesterheld, Demski, Conitzer, 2023). At round $t$ the agent holds estimates $W_t(a,a^{\prime })$ of the quantities $w(a,a^{\prime })$, chooses ${\widehat{a^{\prime }}}_t=\arg \underset{a^{\prime }}{max}\underset{a\in A_{-m}}{max}{W}_t(a,a^{\prime })$ and $a_t=\arg \underset{a\in A_{-m}}{max}{W}_t(a,{\widehat{a^{\prime }}}_t)$, realizing the pair $c_t=(a_t,{\widehat{a^{\prime }}}_t)$ with true value $w(c_t)$ and estimation regret ${\epsilon }_t=v^{-}-w(c_t)\ge 0$. The realization schedules $S_{a,a^{\prime }}:=\{t:c_t=(a,a^{\prime })\}$ are $g$-decidable and partition $\mathbb{N}$.

Theorem 2. Let $\overline{\alpha }$ be a covering BRIA over $\{h_{a,a^{\prime }}:(a,a^{\prime })\in A_{-m}\times A\}$, where $h_{a,a^{\prime }}$ recommends the pair $(a,a^{\prime })$ and promises $w(a,a^{\prime })-\delta$. Under reward-value tracking (R2) on each $S_{a,a^{\prime }}$:

Proof. We use two properties of the covering BRIA from Oesterheld et al. (2023):

• (i) covering (Def. 6): if the empirical record of a hypothesis $h$ does not diverge to $-\mathrm{\infty }$, then the set of rounds in which $h$ is rejected is finite, and hence ${\alpha }_t^e\ge {\alpha }^e(h)-\delta$ for all but finitely many $t$;
• (ii) no overestimation: $\frac{1}{T}\sum _{t\le T}{r}_t\ge \frac{1}{T}\sum _{t\le T}{\alpha }_t^e-o(1)$.

Fix $(a,a^{\prime })$ and $\delta >0$. The test set $M_{a,a^{\prime }}\subseteq S_{a,a^{\prime }}$ of the hypothesis $h_{a,a^{\prime }}$ is $g$-decidable; value tracking (R2) on it gives $\frac{1}{|M_{\le T}|}\sum _{t\in M_{\le T}}(r_t-w(a,a^{\prime }))\to 0$. Consequently the empirical record $\sum _{t\in M_{\le T}}(r_t-(w(a,a^{\prime })-\delta ))$ has mean $\to \delta >0$ and does not diverge to $-\mathrm{\infty }$. By property (i), ${\alpha }_t^e\ge w(a,a^{\prime })-\delta$ for all but finitely many $t$. Letting $\delta \downarrow 0$ and taking the supremum over all pairs (finitely many), $\underset{t}{lim inf}{\alpha }_t^e\ge v^{-}$, whence $\underset{T}{lim inf}\frac{1}{T}\sum _{t\le T}{\alpha }_t^e\ge v^{-}$. By property (ii), $\frac{1}{T}\sum r_t\ge \frac{1}{T}\sum {\alpha }_t^e-o(1)$, so $\underset{T}{lim inf}\frac{1}{T}\sum r_t\ge v^{-}$.

We partition $\{1,\dots ,T\}$ by realized pairs. For each $(a,a^{\prime })$ with infinite $S_{a,a^{\prime }}$, value tracking (R2) on $S_{a,a^{\prime }}$ gives $\sum _{t\in S_{a,a^{\prime },\le T}}(r_t-w(a,a^{\prime }))=o(|S_{a,a^{\prime },\le T}|)=o(T)$. Summing over the finitely many pairs, $\frac{1}{T}\sum _{t\le T}(r_t-w(c_t))\to 0$. Since $w(c_t)\le v^{-}$, we obtain $\underset{T}{lim sup}\frac{1}{T}\sum r_t\le v^{-}$.

Combining: $\frac{1}{T}\sum r_t\to v^{-}$; from the ceiling identity $\frac{1}{T}\sum w(c_t)=\frac{1}{T}\sum r_t-o(1)\to v^{-}$, whence $\frac{1}{T}\sum {\epsilon }_t\to 0$. $◻$

Corollary. Set

If $\mathrm{\Delta }>0$, then the best response is chosen with limiting frequency $1$: since $w(c_t)<v^{-}\Rightarrow w(c_t)\le v^{-}-\mathrm{\Delta }$, we have ${\epsilon }_t\ge \mathrm{\Delta }\cdot \mathbf{1}[{\epsilon }_t>0]$, whence

In the absence of a gap ($\mathrm{\Delta }=0$) only the Cesàro statement of Theorem 2 holds. (The gap is taken on the $w$-scale of realized pairs $c_t$, consistently with the indicator.)

At this point we have shown the attainability of tiling in values; however, we have implicitly assumed that the AC and KoDP steps are exact equalities, whereas so far they too consume omniscience. But in reality the agent may err not only in values (as in FGJ) but in the maximum-selection process itself. If it incorrectly identifies the argmax, this creates a new error. To guarantee that these errors also vanish, we introduce Lemma 1, which links the error in probabilities to the error in expectations.

Lemma 1. Let $C,C^{\prime }$ be events of positive $P_t$-probability, $G$ an arbitrary event, $u\in [0,1]$. If $C\cap G=C^{\prime }\cap G$, then:

Proof. By total expectation and $u\le 1$: ${\mathbb{E}}_p[u\mid C]\le {\mathbb{E}}_p[u\mid C,G]+P_t(\mathrm{\neg }G\mid C)$. From $C\cap G=C^{\prime }\cap G$:

Finally, ${\mathbb{E}}_p[u\mid C^{\prime },G]-{\mathbb{E}}_p[u\mid C^{\prime }]=P_t(\mathrm{\neg }G\mid C^{\prime })({\mathbb{E}}_p[u\mid C^{\prime },G]-{\mathbb{E}}_p[u\mid C^{\prime },\mathrm{\neg }G])\le P_t(\mathrm{\neg }G\mid C^{\prime })$, since the bracket lies in $[-1,1]$. $◻$

Good events for AC and KoDP:

Let $B_a:=\{{\pi }^{\ast }(o)=⌜a⌝\}$, $C_a:=\{{\pi }^{\ast }(o)=⌜a⌝\wedge {\pi }^{\ast }(\overline{o})=⌜\overline{a}⌝\}$, $C_a^{\mathrm{AC}}$ — the conditioning after the AC step. We define (at round $t$; in the idealized form of Theorem 3 the same residuals are written without the index $(t)$):

Now we combine the results obtained.

Theorem 3. Under the residual bounds ${\epsilon }_{\mathrm{FGJ}}$, ${\epsilon }_{\mathrm{AC}}$, ${\epsilon }_{\mathrm{KoDP}}$:

Proof. By Theorem 1 the steps before FGJ are exact; FGJ contributes $+{\epsilon }_{\mathrm{FGJ}}$ and yields $X=\underset{a}{max}{\mathbb{E}}_p[u\mid C_a]$.

We apply Lemma 1 with $(C,C^{\prime },G)=(C_a,C_a^{\mathrm{AC}},G_{\mathrm{AC}})$. The hypothesis $C_a\cap G_{\mathrm{AC}}=C_a^{\mathrm{AC}}\cap G_{\mathrm{AC}}$ holds: on $G_{\mathrm{AC}}$ the iterated double maximum inside the argmax equals the single one, so the conditioning on ${\pi }^{\ast }(\overline{o})$ coincides, while the parts ${\pi }^{\ast }(o)=⌜a⌝$ are identical. Hence ${\mathbb{E}}_p[u\mid C_a]\le {\mathbb{E}}_p[u\mid C_a^{\mathrm{AC}}]+P_t(\mathrm{\neg }{G}_{\mathrm{AC}}\mid C_a)+P_t(\mathrm{\neg }{G}_{\mathrm{AC}}\mid C_a^{\mathrm{AC}})$; taking $\underset{a}{max}$: $X\le X_{\mathrm{AC}}+{\epsilon }_{\mathrm{AC}}$.

We apply Lemma 1 with $(C,C^{\prime },G)=(C_a^{\mathrm{AC}},B_a,G_{\mathrm{KoDP}})$: on $G_{\mathrm{KoDP}}$ knowledge of the procedure makes the argmax condition redundant, so $C_a^{\mathrm{AC}}\cap G_{\mathrm{KoDP}}=B_a\cap G_{\mathrm{KoDP}}$. Hence $X_{\mathrm{AC}}\le \underset{a}{max}{\mathbb{E}}_p[u\mid B_a]+{\epsilon }_{\mathrm{KoDP}}$.

Combining: $X\le \underset{a\in A_{-m}}{max}{\mathbb{E}}_p[u\mid {\pi }^{\ast }(o)=⌜a⌝]+{\epsilon }_{\mathrm{AC}}+{\epsilon }_{\mathrm{KoDP}}$, and adding the FGJ gap closes the chain. $◻$

Thus we have carried out the dynamization of FGJ via BRIA.

3. The Minimal Condition on the Inductor

If the agent's errors in value estimates tend to zero, then tiling holds. But what exactly must the inductor ensure for these errors to actually vanish? We cannot demand perfection from the agent in everything, but we must impose requirements on those aspects of its logic that directly affect decision-making. We call this set of requirements the Minimal Condition for Rationality (MCR).

Let the decision-relevant generators be the propositions about policy points $\{{\pi }^{\ast }(o)=⌜a⌝\}$, value comparisons $\{w(a,a^{\prime })\ge w(\tilde{a},{\tilde{a}}^{\prime })\}$, and reward variables $(r_t)$ on the realization schedules. Let $\mathcal{P}:=\{(o,\mathrm{ob}(a_m)):a_m\in A_m\cap A_o\}$ be the "modification–target" pairs.

Definition (Minimal Condition for Rationality — MCR). The true prices $\hat{Q}$ satisfy $\mathrm{MCR}(\mathcal{A})$ under budget $g$ if:

• (R1) for each $E,C\in \mathcal{A}$ with $P_{\hat{Q}}(C)>0$, the centered sequence $(\mathbf{1}[E_t]-P_t(E\mid C){)}_t$ has vanishing Cesàro mean along each $g$-decidable subschedule $S_C$; equivalently, its true-price frequency equals $P_{\hat{Q}}(E\mid C)$, inexploitably;
• (R2) for each realization schedule $S$ generated within $\mathcal{A}$, the reward sequence on $S$ is $O(g)$-boundedly van Mises–Wald–Church (vMWC) random with its true conditional mean.

R1 guarantees that the agent's belief that "event $E$ will occur" is mathematically consistent with reality (necessity). R2 guarantees that the agent physically "sees" the rewards and that they are no noisier than the budget permits (sufficiency). Together they create the foundation for convergence.

In the special case, two-block vMWC randomness for a pair $(o,o^{\prime })$ is exactly MCR restricted to the two-block cross algebra of a single pair of observations. MCR lifts it to an arbitrary decision-relevant algebra.

Lemma 2. Suppose $\mathrm{MCR}(\mathcal{A})$ and the auxiliary coherence constraints (inherited from DHR25 §6). Let $G\in \mathcal{A}$ be a $g$-decidable event with $P_{\hat{Q}}(G)=1$, given by a finite Boolean combination of strict comparisons $\{w(b)>w(b^{\prime })\}$ from $\mathcal{A}$, and $C\in \mathcal{A}$ a $g$-decidable event of positive true price. Then:

Proof. Write $G=\bigcap _{k=1}^K{E}_k$, $E_k=\{w(b_k)>w(b_k^{\prime })\}$, with true gaps ${\delta }_k:=w(b_k)-w(b_k^{\prime })>0$ ($K<\mathrm{\infty }$, since $A$ is finite).

By R2 the estimates track the true values on the corresponding schedules; real-valued coherence pins the credences $P_t(E_k)$ to these estimates. Since the gap ${\delta }_k>0$ is strict, the estimated comparison coincides with the true one with limiting frequency $1$, whence $\frac{1}{T}\sum _{t\le T}{P}_t(\mathrm{\neg }{E}_k)\to 0$. By subadditivity $P_t(\mathrm{\neg }G)\le \sum _k{P}_t(\mathrm{\neg }{E}_k)$, so $\frac{1}{T}\sum _{t\le T}{P}_t(\mathrm{\neg }G)\to 0$.

Since $P_t\to \hat{Q}$ and $P_{\hat{Q}}(C)>0$, there exist $t_0$ and $c:=\frac{1}{2}{P}_{\hat{Q}}(C)>0$ with $P_t(C)\ge c$ for $t\ge t_0$. The auxiliary coherence constraints license the monotonicity $P_{\hat{Q}}(\mathrm{\neg }G\wedge C)=0$ for self-referential $C$.

We split the sum at $t_0$: the first $t_0$ terms contribute $\le t_0/T\to 0$. For the rest: $P_t(\mathrm{\neg }G\mid C)\le (P_t(\mathrm{\neg }G)+{\beta }_t)/c$, where ${\beta }_t\to 0$, which completes the proof. $◻$

Lemma 3. If R1 is violated for $(\mathrm{\neg }G,C)$: there exist an infinite $g$-decidable subschedule $S\subseteq S_C$ and $\eta >0$ such that

for infinitely many $T$. Then the inductor is $O(g)$-exploitable.

The proof of Lemma 3 is a standard trader argument and, for readability, is given in Appendix A.

We see that MCR does indeed make sense as a constraint to impose; however, to understand how strict MCR must be, we need to determine the bounds of its domain of application. It would be excessive to demand MCR for all possible propositions in the agent's language. It suffices for it to hold only for those propositions the agent uses to compare actions and choose a policy. This leads us to the notion of a minimal subalgebra.

3.1 The Minimal Subalgebra

The tiling chain conditions and compares values only through the pairs $(o,o^{\prime })\in \mathcal{P}$. This allows defining the minimal algebra, with $S_{a,a^{\prime }}:=\{t\in \mathbb{N}:c_t=(a,a^{\prime })\}$:

restricted to the pairs $\mathcal{P}$. (The value layer is the joint $\sigma$-algebra of rewards and schedule labels $S_{a,a^{\prime }}$, since the schedules $\{S_{a,a^{\prime }}\}$ themselves partition $\mathbb{N}$ and restricting the rewards to their union yields all $r_t$.)

MCR guarantees that the agent learns correctly, but it does not guarantee that the structure of the world (or of the policy) itself helps it make the right choice. For the action-coordination step (AC) in our chain to become exact, we may need a certain structural orderedness in the value table. We introduce two such structural notions: branch independence (VR1) and argmax stability (CBA).

Definition. A pair $(o,o^{\prime })$ satisfies VR1 under $\hat{Q}$ if the joint policy law factorizes: $\rho (a^{\prime }\mid a)=\rho (a^{\prime })$ and $\sigma (a\mid a^{\prime })=\sigma (a)$ for all $a,a^{\prime }$.

Definition.

• A pair $(o,o^{\prime })$ satisfies CBA if there exists a uniform dominator $a_{\ast }\in A_{-m}$ with $w(a_{\ast },a^{\prime })=\underset{a\in A_{-m}}{max}w(a,a^{\prime })$ for every $a^{\prime }\in A$.
• The pair satisfies sCBA (strict CBA) if it satisfies CBA and, additionally, for each $a\in A_{-m}\setminus \{a_{\ast }\}$ there exists $a^{\prime }$ with $\rho (a^{\prime })>0$ and $w(a_{\ast },a^{\prime })>w(a,a^{\prime })$.

Thus $\mathrm{sCBA}\Rightarrow \mathrm{CBA}$, and both properties have distinct notations.

If VR1 and CBA hold, then MCR automatically guarantees the exactness of the AC and KoDP steps. We prove this with the following lemmas, under Action-Coordination Honesty (ACH):

Lemma 4. Suppose $(o,o^{\prime })$ satisfies sCBA (with dominator $a_{\ast }$), VR1, and ACH. Then:

• (i) $\arg \underset{a\in A_{-m}}{max}{w}_o(a)=\{a_{\ast }\}$ (unique maximizer);
• (ii) $a_o^{\ast }=a_{\ast }$;
• (iii) $\mathrm{\forall }{a}^{\prime }\in A:v(a^{\prime })=w(a_{\ast },a^{\prime })$;
• (iv) $G_{\mathrm{AC}}$ holds under $\hat{Q}$.

Proof. (i) By VR1 the weight $\rho (a^{\prime }\mid a)=\rho (a^{\prime })$ does not depend on $a$, so:

For $a\ne a_{\ast }$ the strictness of sCBA gives an $a_0^{\prime }$ with $\rho (a_0^{\prime })>0$ and $w(a_{\ast },a_0^{\prime })>w(a,a_0^{\prime })$, making the inequality strict. Thus $\arg \underset{a\in A_{-m}}{max}{w}_o(a)=\{a_{\ast }\}$.

(ii) By ACH, $a_o^{\ast }\in A_{-m}$; since $a_o^{\ast }$ maximizes $w_o$ over $A_o\supseteq A_{-m}$, it also maximizes over $A_{-m}$, and uniqueness from (i) forces $a_o^{\ast }=a_{\ast }$.

(iii) Determinism of the policy gives $\sigma (\cdot )={\delta }_{a_o^{\ast }}$; by VR1, $\sigma (a\mid a^{\prime })=\sigma (a)$, whence $v(a^{\prime })=\sum _a\sigma (a)w(a,a^{\prime })=w(a_o^{\ast },a^{\prime })=w(a_{\ast },a^{\prime })$ by (ii).

(iv) By CBA, $\arg \underset{a^{\prime }}{max}\underset{a}{max}w(a,a^{\prime })=\arg \underset{a^{\prime }}{max}w(a_{\ast },a^{\prime })$; by (iii), $\arg \underset{a^{\prime }}{max}v(a^{\prime })=\arg \underset{a^{\prime }}{max}w(a_{\ast },a^{\prime })$. The two argmaxes coincide — this is $G_{\mathrm{AC}}$. $◻$

Clearly, without VR1 the conclusion fails: with $w(a_{\ast },\cdot )=(0.9,0.2)$, $w(a_1,\cdot )=(0.8,0.1)$ and $\rho (\cdot \mid a_{\ast })={\delta }_{a_2^{\prime }}$, $\rho (\cdot \mid a_1)={\delta }_{a_1^{\prime }}$ we have $w_o(a_{\ast })=0.2<0.8=w_o(a_1)$, even though $a_{\ast }$ dominates pointwise. A comparison of VR1 and full branch independence is in Appendix B.

Lemma 5. Suppose value tracking of the $o$-block (R2) and MCR over the comparison events of the $o$-block. Then:

and similarly with $C_a^{\mathrm{AC}}$ in place of $B_a$.

Proof. By the BRIA rule: ${\pi }^{\ast }(o)=\arg \underset{a^{\prime }}{max}{\hat{\mathbb{E}}}_t(u\mid {\pi }^{\ast }(o)=⌜a^{\prime }⌝)$. Consequently $\mathrm{\neg }{G}_{\mathrm{KoDP}}⟺\arg \underset{a^{\prime }}{max}{\hat{\mathbb{E}}}_t(u\mid {\pi }^{\ast }(o)=⌜a^{\prime }⌝)\ne \arg \underset{a^{\prime }}{max}{w}_o(a^{\prime })$ — this is the $g$-decidable event "the argmax of the estimate does not coincide with the argmax of the true value in the $o$-block".

$G_{\mathrm{KoDP}}$ is a finite Boolean combination of $o$-block comparisons, $g$-decidable. Under true prices $P_{\hat{Q}}(\mathrm{\neg }{G}_{\mathrm{KoDP}}\mid B_a)=0$: the agent chooses $a$ if and only if $a$ is its estimate-argmax, which by limiting value tracking is the argmax of the true value. Lemma 2 yields the Cesàro-zero conclusion.

The only potential self-reference is in the hypothesis $P_{\hat{Q}}(\mathrm{\neg }{G}_{\mathrm{KoDP}}\mid B_a)=0$. Universal instantiation transfers ${\hat{\mathbb{E}}}_t(u\mid {\pi }^{\ast }(o)=a^{\prime })\to w_o(a^{\prime })$ to the concrete $a^{\prime }=a$ even under conditioning on $B_a$; real-valued coherence forces $\mathbb{E}[u\mid B_a]$ to cohere with the learned $w_o(a)$. This is exactly the auxiliary coherence constraints from DHR25 §6, no new generators are consumed. $◻$

Now we can assemble everything. We have:

• the error in values, which vanishes by R2/Theorem 2,
• the error in probabilities (comparisons), which vanishes by R1/Lemma 2,
• the errors in the AC and KoDP steps, which vanish if the world possesses the VR1/CBA structure.

The synthesis of these results gives the main theorem of the section.

Theorem 4. Let the agent be a BRIA over a Bayesian logical inductor. Suppose LSM, the structural premises making $G_{\mathrm{AC}}$, $G_{\mathrm{KoDP}}$ exact under $\hat{Q}$ (Lemma 4, Lemma 5), and $\mathrm{MCR}({\mathcal{A}}^{\ast })$. Then:

Proof. The static chain (Theorem 3) is exact on the first step (FGF) under true prices and becomes an $\epsilon$-chain with residuals ${\epsilon }_{\mathrm{FGJ}}^{(t)}+{\epsilon }_{\mathrm{AC}}^{(t)}+{\epsilon }_{\mathrm{KoDP}}^{(t)}$, arising outside the inner $\arg max$ and not amplified by the exact steps. The conclusion, however, is denominated in the round estimates ${\hat{\mathbb{E}}}_t$, whereas the chain is written in idealized prices; transferring to ${\hat{\mathbb{E}}}_t$ introduces one more residual at the FGF step, which we first isolate explicitly.

For ${\epsilon }_{\mathrm{FGF}}^{(t)}$: the self-modification event $\{{\pi }^{\ast }(o)=a_m\}$ and the non-modification event $\{{\pi }^{\ast }(o)={\hat{a}}_m\wedge {\pi }^{\ast }(\overline{o})=\overline{a}\}$ have the same true value, but the agent sees different estimates. We define this gap as:

The cell $({\hat{a}}_m,\overline{a})\in A_{-m}\times A$ is an $o$-block generator of the value layer. Thanks to the value-tracking condition R2, the agent inevitably comes to know the true values of these table cells over time, so $\frac{1}{T}\sum _{t\le T}{\epsilon }_{\mathrm{FGF}}^{(t)}\to 0$.

Similarly, the estimation residual $|{\hat{\mathbb{E}}}_t(u\mid {\pi }^{\ast }(o)=⌜a⌝)-w_o(a)|$ for the non-self-modifying $o$-block is Cesàro-zero by R2. Hence the difference between the estimated and the true form of the bound is Cesàro-zero.

• For ${\epsilon }_{\mathrm{FGJ}}^{(t)}$: Theorem 2 guarantees convergence of the error to $0$.
• For ${\epsilon }_{\mathrm{AC}}^{(t)}$: by Lemma 4 $G_{\mathrm{AC}}$ is exact under $\hat{Q}$; Lemma 2 gives $\frac{1}{T}\sum P_t(\mathrm{\neg }{G}_{\mathrm{AC}}\mid C)\to 0$ for each $C\in \{C_a,C_a^{\mathrm{AC}}\}$; finiteness of $A_{-m}$ and subadditivity close it.
• For ${\epsilon }_{\mathrm{KoDP}}^{(t)}$: Lemma 5 guarantees that, thanks to MCR, the error converges to $0$.

Since each of these error sources is Cesàro-zero, their sum also tends to zero. $◻$

Theorem 5. Under LSM, FGF, ACH, VR1, and sCBA under $\hat{Q}$,

Proof. $(\Rightarrow )$ This is exactly Theorem 4: with the skeleton fixed, $\mathrm{MCR}({\mathcal{A}}^{\ast })$ supplies R1 and R2, the structural premises give exactness of the good events under $\hat{Q}$, and the assembly of Theorem 4 gives $\frac{1}{T}\sum _{t\le T}{\rho }_t\to 0$.

$(\Leftarrow )$ We show by contraposition that abandoning either layer breaks the conclusion.

Suppose R2 is violated on some realization schedule $S_{a_0,a_0^{\prime }}$: there exists an infinite $g$-decidable $S\subseteq S_{a_0,a_0^{\prime }}$ with

Then $\frac{1}{T}\sum _{t\le T}w(c_t)$ does not converge to $v^{-}$ along the subsequence on which the pair $(a_0,a_0^{\prime })$ is chosen with positive frequency, and the regret ${\epsilon }_t$ has nonzero Cesàro mean along it; by Theorem 2 this is exactly a violation of the Cesàro-zeroness of ${\epsilon }_{\mathrm{FGJ}}$, so the tiling term does not tend to zero. Hence the value layer is indispensable.

Suppose R1 is violated for $(\mathrm{\neg }{G}_{\mathrm{AC}},C)$ for some $C\in \{C_a,C_a^{\mathrm{AC}}\}$: there exist a $g$-decidable $S\subseteq S_C$ and $\eta >0$ with

for infinitely many $T$. By Lemma 3 (Appendix A) the inductor is $O(g)$-exploitable, which is incompatible with the logical-induction criterion; equivalently, excluding the comparison layer admits an inductor whose true prices violate CBA, so $G_{\mathrm{AC}}$ fails, the AC step collapses by a nonzero amount, and an $O(g)$-trader betting on the mismatch of argmaxes exploits the agent. Hence the comparison layer is indispensable.

Thus both layers are necessary and sufficient, and ${\mathcal{A}}^{\ast }$ is minimal. $◻$

3.2 Reflective Stability

Theorem 4 was proved by reasoning about the agent's prices. For tiling to be self-applicable, the predecessor must refrain from modification on grounds available to it itself.

Proposition 1. Every quantity in the tiling bound of Theorem 4 is an entry in the agent's price system $P_t$. In particular, all residuals $P_t(\mathrm{\neg }G\mid C)$ are the agent's own credences, not external quantities.

Proof. The proof of Theorem 4 is built along the scheme: transfer of FGF to estimates (the residual ${\epsilon }_{\mathrm{FGF}}^{(t)}$ is an R2-tracked quantity in ${\hat{\mathbb{E}}}_t$) $\to$ $\epsilon$-chain in ${\hat{\mathbb{E}}}_t$ $\to$ Lemma 1 introduces the residuals $P_t(\mathrm{\neg }G\mid C)$ $\to$ Lemma 2 gives their Cesàro-zeroness. Every term at every step is a round price of the agent. $◻$

As a consequence, to act, the agent does not need a credence about its own price of the form $P_t(P_t(\mathrm{\neg }G\mid C)<\epsilon )$. It takes the $\arg max$ of ${\hat{\mathbb{E}}}_t$, which is a price. The "belief about belief" tower does not start, since the action consumes a price, not a judgment about a price.

Lemma 6. Denote the modification successor $\mathsf{S}(a_m)$ and the unmodified successor ${\mathrm{\Pi }}^{\circ }$. The reflective gap ${\rho }_t(a_m):=[{\hat{V}}_t(\mathsf{S}(a_m))-{\hat{V}}_t({\mathrm{\Pi }}^{\circ }){]}_{+}$. Then

where UDT 1.0 is the updateless optimization rule ${\pi }^{\ast }(o):=\arg \underset{a\in A_o}{max}{\mathbb{E}}_p(u\mid {\pi }^{\ast }(o)=⌜a⌝)$.

Proof. The updatelessness of UDT 1.0 chooses the policy once; in the dynamized version, at each round $t$ all coordinates of the committed policy are co-determined by one and the same price system ${\hat{\mathbb{E}}}_t$. In particular the $o^{\prime }$-coordinate (and the entire lower part) is evaluated at the round-$t$ prices, not by a later estimate ${\hat{\mathbb{E}}}_{t^{\prime }}$ with $t^{\prime }>t$. This follows from updatelessness itself — the predecessor at round $t$ values the committed policy as a single object under ${\hat{\mathbb{E}}}_t$ — and requires neither factorization of the branch probabilities (VR1) nor invariance of values across branches (VR2): the localization of the optimization onto the $o^{\prime }$-term (the content of Theorem 4 of the companion FGJ document, which uses VR1/VR2) is needed only to compute that term, whereas for the gap identity it suffices to have the qualitative fact that both valuations — ${\hat{V}}_t(\mathsf{S}(a_m))$ and ${\hat{V}}_t({\mathrm{\Pi }}^{\circ })$ — are taken at the prices of one and the same round $t$. Therefore the valuation "keep UDT" is not a nested ${\hat{\mathbb{E}}}_t[{\hat{\mathbb{E}}}_{t^{\prime }}[\cdot ]]$ with $t^{\prime }>t$, but a single-level round conditional expectation. This is exactly what makes Lemma 6 applicable to the branches of §4, where VR1 is dropped.

Therefore "impose $a_m$" is ${\hat{V}}_t(\mathsf{S}(a_m))={\hat{\mathbb{E}}}_t(u\mid {\pi }^{\ast }(o)=a_m)$ by definition, and "keep UDT" is ${\hat{V}}_t({\mathrm{\Pi }}^{\circ })=\underset{a\in A_{-m}}{max}{\hat{\mathbb{E}}}_t(u\mid {\pi }^{\ast }(o)=⌜a⌝)$ with the co-determined lower part. The downstream $\arg max$ is a single-level round coordination event (of type $G_{\mathrm{KoDP}}$), not a separately trusted future estimate. The correspondence between the form of the action-at-$o$ (${\pi }^{\ast }(o)=a_m$ versus $\underset{A_{-m}}{max}$) and the form of the modification ("impose" versus "keep") is exactly the FGF + LSM bundle. $◻$

Theorem 6. Under the premises of Theorem 4 and without any additional self-trust assumption:

Proof. By Lemma 6, ${\rho }_t(a_m)$ is the summand of Theorem 4; we apply Theorem 4. The only residual self-reference — the downstream coordination event $H=\{\arg \underset{a^{\prime }}{max}{\hat{\mathbb{E}}}_t(u\mid {\pi }^{\ast }(o^{\prime })=a^{\prime })=\arg \underset{a^{\prime }}{max}v(a^{\prime })\}$ — is $G_{\mathrm{KoDP}}$ for the $o^{\prime }$-block (Lemma 5), $g$-decidable, $P_{\hat{Q}}(H)=1$; its residual $P_t(\mathrm{\neg }H\mid C)$ is Cesàro-zero by Lemma 2. No new generator beyond the value and comparison layers is introduced. In the absence of a value gap, when $H$ may have ties and $P_{\hat{Q}}(H)<1$, ${\rho }_t$ is controlled directly by the value regret ${\epsilon }_t$ (ties do not lose value), which is Cesàro-zero by Theorem 2; the indicator of $H$ is then not needed. $◻$

As an output, the minimal tiling condition preserves only VR1 as an external axiom. It is needed for Lemma 4, since without it the pointwise dominator may fail to be the agent's marginally optimal action. Globally, in the sections below, we consider various directions for removing this constraint.

4. Working with VR1

Regarding VR1, we see two options for solving the problem. On the one hand, we can impose a condition on the structure of prices under the given policy, that is, essentially state the non-guaranteedness of the agent's optimal policy, since it may require the absence of this condition. This option is acceptable from the standpoint of pure UDT, since it does not presuppose a significant change to the theory, and allows simply translating the omniscience problem into the much more realistic problem of branch independence.

On the other hand, we propose to impose a condition on the agent's action schedule over time. This approach has a goal similar to GU in the sense that, by the agent making an agreement with itself regarding its own actions, we solve a significant problem of the decision-making system. In exchange for such a constraint, we obtain the ability to guarantee attainment of the optimum without omniscience of the agent, and therefore we consider it promising.

4.1 Bounded Optimality

Definition. A regime $d=(o,o^{\prime })\in \mathcal{P}$ satisfies realized cross-block domination ${\mathrm{RD}}^{\ast }$ if:

Clearly, ${\mathrm{RD}}^{\ast }$ is weaker than the bundle VR1 + sCBA: there $\mathrm{VR}1\wedge \mathrm{sCBA}\wedge \mathrm{ACH}\Rightarrow {\mathrm{RD}}^{\ast }$ (Lemma 4(ii)), so ${\mathrm{RD}}^{\ast }$ is a strict weakening. The converse is false: ${\mathrm{RD}}^{\ast }$ does not entail $\rho (a^{\prime }\mid a)=\rho (a^{\prime })$ and may hold under coupled conditional laws.

Definition. A regime $d$ is admissible if it satisfies ACH ($a_o^{\ast }\in A_{-m}$) and ${\mathrm{RD}}^{\ast }$ under $\hat{Q}$. The set of admissible regimes is $\mathcal{D}$; it is finite ($A,O$ finite).

Lemma 7. If a regime $d$ satisfies ACH and ${\mathrm{RD}}^{\ast }$ under $\hat{Q}$, then $G_{\mathrm{AC}}$ holds under $\hat{Q}$.

Proof. Denote $M(a^{\prime }):=\underset{a\in A_{-m}}{max}w(a,a^{\prime })$.

(i) By ${\mathrm{RD}}^{\ast }$: $M(a^{\prime })=w(a_o^{\ast },a^{\prime })$ for every $a^{\prime }\in A$.