Skip to content

State Management

This section explains how dspyer coordinates agent state transitions, performs Copy-on-Write (COW) performance optimizations, and resolves merge conflicts when reconciling parallel execution branches.

The State Model

To stay compatible with standard DSPy tracers and optimizers, dspyer represents agent execution states as a series of stateless transitions. Mutating state objects in-place inside forward passes is a common cause of tracing bugs and optimization parameter lookup failures.

State transitions are modeled mathematically as:

$$S_{t+1} = S_t \oplus \Delta_t$$

Where: * $S_t$ is the current ImmutableState snapshot. * $\Delta_t$ is a dictionary merge patch generated by executing node $t$ (equivalent to Pydantic's model_dump()). * $\oplus$ is the merge operator.

Copy-on-Write (COW) Optimization

In version 0.3.2, we replaced global state cloning (deepcopy) with a high-performance Copy-on-Write (COW) dictionary state merge algorithm.

During graph execution, instead of copying the entire state dictionary, dspyer does the following: 1. Touch only the dictionary pathways where keys are being modified, added, or deleted. 2. Clone only the immediate parent namespaces of those modified paths. 3. Untouched keys and nested branches are referenced as-is.

This results in a significant reduction in memory overhead and computation latency, which is crucial when handling large query payloads, complex document histories, or high-concurrency web servers.

Conflict Resolution Policies

When parallel branches execute concurrently, they start from the same base state $S_t$ and produce separate patches $\Delta_1$ and $\Delta_2$. To reconcile these divergent states back into a single unified timeline, dspyer provides the merge method with three configurable conflict resolution policies:

from dspyer import ImmutableState

state_a = ImmutableState({"user_id": 123, "tags": ["active"]})
state_b = ImmutableState({"user_id": 123, "tags": ["premium"], "score": 95})

# Reconcile states
merged = state_a.merge(state_b, policy="combine_lists")
print(merged.to_dict())
# Output: {'user_id': 123, 'tags': ['active', 'premium'], 'score': 95}

Supported Policies

  1. last_write_wins (Default):
    • Conflicting keys in the target state are overwritten by values from the incoming state.
  2. combine_lists:
    • If a conflicting key holds lists in both states, the lists are concatenated.
    • For other types, it falls back to last_write_wins.
  3. raise:
    • Raises a ValueError if a key exists in both states with mismatched values, ensuring developers catch unintended execution collisions early.