# Long-Horizon Task Reliability

How checkpointing, abort policies, and recovery limit drift in long-running agents.

- Author: Jatin Bansal (https://jatinbansal.com/)
- Published: 2026-05-19
- Canonical: https://jatinbansal.com/ai-engineering/long-horizon-reliability/
- Topic: AI Engineering

---

Long-running agents accumulate state, assumptions, and side effects. A coding agent can complete hundreds of valid edits, form one bad hypothesis, then damage shared infrastructure while trying to recover. Model capability did not change during the run; the loop crossed into a less reliable operating regime.

## Separate capability from reliability

The cleanest framing is the [reliability-vs-capability split from the late-2026 "Beyond pass@1" framework](https://arxiv.org/html/2603.29231v1): capability is whether the model can do a task on its best attempt (pass@1, the standard benchmark axis); reliability is whether it can do the task *consistently* across attempts of varying duration. A model that scores 90% on a 5-minute task and 44% on a 4-hour version isn't a worse model; it's the same model in a different failure regime.

Two headline findings. Frontier models exhibit the *highest* meltdown rates (DeepSeek V3 at 19%, MiniMax M2.5 at 13% at very-long horizons) precisely because they attempt ambitious multi-step strategies; weaker models hover around 0-4% because they fall back on rote, shallow strategies that don't have anywhere to drift to. And the failure shape is positive error correlation across steps: once the agent forms a wrong hypothesis it persists rather than recovers, and the reliability curve decays super-linearly relative to an i.i.d. Bernoulli baseline. Capability is the model's job; reliability is [the harness's](/ai-engineering/agent-harness-anatomy/) job.

## Interpret time-horizon benchmarks carefully

The empirical anchor is [METR's time-horizon benchmark](https://metr.org/time-horizons/), which measures the task duration (calibrated against human completion time) that an agent can do with 50% reliability. As of [METR Time Horizon 1.1 in January 2026](https://metr.org/blog/2026-1-29-time-horizon-1-1/), Claude Opus 4.5 sits at 320 minutes (5.3 hours), GPT-5 at 214, o3 at 121, Claude Opus 4 at 101. The trend since 2024 is a doubling every 89-131 days, with [later updates putting Opus 4.6 at roughly 14.5 hours by February 2026](https://apiardata.com/statistics/ai-autonomous-task-horizon/). The "Moore's law for agents" framing is real and the curve is steep.

The published 50% horizon is an inflection point, not a safe operating target. Horizons at 80% or 95% reliability are much shorter, and failures beyond the measured range can be abrupt rather than gradual.

## How long runs drift

Four mechanisms drive the super-linear decay, and they compound. The "Beyond pass@1" framework and the [Wire blog's anatomy of agent drift](https://usewire.io/blog/agent-drift-why-long-running-ai-agents-lose-the-plot/) converge on this list:

- Context drift. History grows; older turns become decorations the model attends to less; critical details from turn 10 are diluted by 200 turns of subsequent noise. The lost-in-the-middle failure from [the context-engineering article](/ai-engineering/context-engineering-jit-vs-aot/) at a longer time scale. Mitigation: [aggressive compaction](/ai-engineering/context-compression/) and JIT-only context fetches.
- Hallucination cascades. A wrong fact at turn 15 gets cited at turn 30, becomes part of the agent's "known" state, and by turn 100 the agent is reasoning from a corrupted premise it can't distinguish from primary observation. Mitigation: explicit *provenance tracking*: every working-state claim carries a pointer to its source turn, the same shape as the [memory-provenance article](/ai-engineering/temporal-reasoning-provenance/) describes.
- Goal drift. "Migrate 400 stored procedures" gets gradually re-interpreted as "improve the test harness" or "refactor the migration script": adjacent activities that look like progress but aren't the task. Mitigation: an explicit goal artifact (a typed object, not a paragraph) re-injected at every step.
- Meltdown. The pathological end state: the agent transitions from "coherent but wrong" to "incoherent looping, self-contradiction, hallucinated tool outputs." MOP (meltdown-of-progress) precursors are entropy spikes, repeated tool calls with slightly varied args, contradictions across consecutive turns. Mitigation: detect early, save state and restart with a fresh context window; not just compressing, but saying "this run is poisoned, resume from a checkpoint that wasn't."

Context drift makes cascades easier to seed; cascades make goal drift undetectable from inside the loop; goal drift sets up meltdown when the wrong action contradicts the world hard enough.

## Use durable checkpoints

Once you accept the run will fail somewhere past the 50% horizon, the engineering question is how cheaply you can resume. Production frameworks disagree on what "checkpoint" means.

Step-level (between-node) checkpointing. [LangGraph's persistence layer](https://docs.langchain.com/oss/python/langgraph/persistence) is the standard example: a checkpointer saves graph state *between* nodes. On crash, the next run reads the most recent checkpoint and resumes from the next node. Cheap, human-debuggable, the right primitive for most agents.

The gotcha is what it doesn't checkpoint. State *inside* a node is not saved. If one node is doing 200 iterations of work and crashes at iteration 47, the next run restarts that node at iteration 0. [Diagrid's critique of checkpoint-based frameworks](https://www.diagrid.io/blog/checkpoints-are-not-durable-execution-why-langgraph-crewai-google-adk-and-others-fall-short-for-production-agent-workflows) is the pointed take. The cost shows up as wasted tokens; 47 API calls re-issued, 47 mutating writes re-attempted (why the idempotency keys above are mandatory), and as latency on recovery.

Durable-execution checkpointing. [Temporal](https://temporal.io/), [Restate](https://restate.dev/), and similar frameworks invert the model. The engine records *every* I/O operation into a durable event log. On crash, the workflow replays from the start, but every recorded I/O returns its previously-cached result instead of re-issuing. The agent never knows it crashed. Trade-off: more infrastructure and a constrained programming model (every side effect wrapped as an "activity") in exchange for dramatically stronger recovery semantics.

The 2026 production hybrid: LangGraph for control flow with Postgres checkpointing, Temporal (or equivalent) wrapping the outer harness when nodes can take minutes or contain many side effects. The [AWS DynamoDB write-up](https://aws.amazon.com/blogs/database/build-durable-ai-agents-with-langgraph-and-amazon-dynamodb/) and the [LangGraph-vs-Temporal architecture pieces](https://cordum.io/blog/temporal-vs-langgraph) make this case directly. A third primitive worth flagging: event-sourcing the conversation itself; persist every turn with a monotonic sequence number and you can rebuild state at any point by replaying turns up to that number, the [continuation-passing-style frame from the agent-loop article](/ai-engineering/agent-loop/) made durable.

## Code: a checkpointed agent loop in Python with LangGraph

A migration-assistant pattern with explicit drift detection and recovery. The loop checkpoints to SQLite between steps, tracks an explicit goal, and aborts on meltdown precursors. Install: `pip install langgraph langgraph-checkpoint-sqlite anthropic`. Uses [LangGraph](https://langchain-ai.github.io/langgraph/) and the [Anthropic SDK](https://github.com/anthropics/anthropic-sdk-python).

```python
import json, hashlib
from typing import TypedDict, Annotated
from operator import add
from anthropic import Anthropic
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.sqlite import SqliteSaver

client = Anthropic()

class AgentState(TypedDict):
    goal: str
    messages: Annotated[list, add]
    step: int
    tool_sigs: Annotated[list, add]
    drift_score: float
    completed_ids: list[str]
    aborted: bool

MELTDOWN_THRESHOLD = 0.7
MAX_STEPS = 500

TOOLS = [
    {"name": "migrate_proc", "description": "Migrate one stored procedure. Idempotent.",
     "input_schema": {"type": "object", "properties": {"path": {"type": "string"}},
                      "required": ["path"]}},
    {"name": "run_tests", "description": "Run the test suite. Read-only.",
     "input_schema": {"type": "object", "properties": {"path": {"type": "string"}},
                      "required": ["path"]}},
    {"name": "submit_final", "description": "Submit final results. Terminates.",
     "input_schema": {"type": "object",
                      "properties": {"summary": {"type": "string"},
                                     "completed": {"type": "integer"}},
                      "required": ["summary", "completed"]}},
]


def call_model(state: AgentState) -> dict:
    # Re-inject the goal at the top of every call. Cheapest mitigation
    # for goal drift. The model sees the original task on every turn,
    # not just at turn 0.
    sys = (
        f"GOAL (immutable): {state['goal']}\n\n"
        f"Step {state['step']} of at most {MAX_STEPS}. Before each tool call, "
        f"confirm it serves the goal. If finished, call submit_final."
    )
    resp = client.messages.create(
        model="claude-opus-4-7", max_tokens=2048,
        system=sys, tools=TOOLS, messages=state["messages"],
    )
    return {"messages": [{"role": "assistant", "content": resp.content}]}


def dispatch_tools(state: AgentState) -> dict:
    last = state["messages"][-1]["content"]
    tool_blocks = [b for b in last if b.type == "tool_use"]
    if not tool_blocks:
        return {"step": state["step"] + 1}

    results, sigs, drift_delta = [], [], 0.0
    new_completed = list(state["completed_ids"])
    for b in tool_blocks:
        sig = hashlib.sha1(
            f"{b.name}:{json.dumps(b.input, sort_keys=True)}".encode()
        ).hexdigest()
        sigs.append(sig)
        # No-progress detection: same call in last 5 → drift signal.
        if sig in state["tool_sigs"][-5:]:
            drift_delta += 0.3

        if b.name == "submit_final":
            return {"messages": [{"role": "user", "content": [
                {"type": "tool_result", "tool_use_id": b.id, "content": "ok"}]}],
                "step": state["step"] + 1}

        # Idempotency receipt for mutating tools — refuse re-execution.
        if b.name == "migrate_proc" and sig in new_completed:
            results.append({"type": "tool_result", "tool_use_id": b.id,
                            "content": "skipped: already completed"})
            continue

        out = execute_tool(b.name, b.input)   # real dispatch here
        results.append({"type": "tool_result", "tool_use_id": b.id,
                        "content": json.dumps(out),
                        "is_error": not out.get("ok", True)})
        if b.name == "migrate_proc" and out.get("ok"):
            new_completed.append(sig)
        if not out.get("ok", True):
            drift_delta += 0.1

    return {"messages": [{"role": "user", "content": results}],
            "tool_sigs": sigs,
            "drift_score": state["drift_score"] + drift_delta,
            "completed_ids": new_completed,
            "step": state["step"] + 1}


def execute_tool(name: str, args: dict) -> dict:
    if name == "migrate_proc": return {"ok": True, "path": args["path"]}
    if name == "run_tests":    return {"ok": True, "passed": True}
    return {"ok": False, "error": f"unknown tool: {name}"}


def should_continue(state: AgentState) -> str:
    if state["step"] >= MAX_STEPS:
        return END
    if state["drift_score"] >= MELTDOWN_THRESHOLD:
        # Meltdown precursor crossed. Bail; caller restarts from the
        # last good checkpoint with a fresh, compacted context.
        return END
    return "call_model"


def build_graph():
    g = StateGraph(AgentState)
    g.add_node("call_model", call_model)
    g.add_node("dispatch", dispatch_tools)
    g.add_edge(START, "call_model")
    g.add_edge("call_model", "dispatch")
    g.add_conditional_edges("dispatch", should_continue,
                            {"call_model": "call_model", END: END})
    return g.compile(checkpointer=SqliteSaver.from_conn_string("agent.sqlite"))


def run(goal: str, *, thread_id: str):
    graph = build_graph()
    initial = {"goal": goal,
               "messages": [{"role": "user", "content": goal}],
               "step": 0, "tool_sigs": [], "drift_score": 0.0,
               "completed_ids": [], "aborted": False}
    # thread_id binds this run to a checkpoint stream. Re-invoking with
    # the same thread_id after a crash resumes from the last checkpoint.
    return graph.invoke(initial, config={"configurable": {"thread_id": thread_id}})
```

The system prompt re-injects the goal on every turn so it cannot disappear under a long history. When the drift score crosses the threshold, the graph routes to `END`; the caller then restarts from the last checkpoint with a compacted prefix, following the [MOP-triggered context-resetting pattern](https://arxiv.org/html/2603.29231v1). The dispatch layer also makes repeated side effects idempotent: a second `migrate_proc` call returns `skipped` instead of executing again. `thread_id` binds each invocation to its checkpoint stream. A crash inside `dispatch_tools` can still lose work, so nodes with many side effects need a durable workflow engine such as Temporal beneath LangGraph.

## Abort when retries cannot help

The wrong instinct on a long-horizon failure is "retry from the failure point." The failure is rarely the failure; it's the *symptom* of accumulated drift, and retrying from the same drift-poisoned context produces the same shape of failure with new noise. The right instinct is more often abort, salvage, restart.

A checklist for the abort decision:

- Drift score over threshold. Abort, save state, restart with a compacted context.
- Goal drift detected. A second model (or a deterministic check) audits whether recent actions still serve the goal. If the audit fires, abort and re-plan from the original goal.
- Compensable side effects accumulating. If you've shipped 312 PRs and the next 88 start failing in correlated ways, stop. Salvage the 312.
- Non-compensable side effects imminent. Email send, payment capture, irrevocable account changes. If drift is climbing and the next predicted action is non-compensable, abort *before* the action. Better to lose 5 minutes of work than send the wrong email to 200 customers.
- Wall-clock or budget breach. The agent-loop budgets still apply: they just fire later.

The aborted-with-partial-results outcome is a first-class success state, not a failure. A migration agent that finished 312 of 400 procedures and stopped cleanly is a win; one that finished 400 of 400 with a broken test harness is a loss. The harness must expose partial state cleanly; completion list, abort reason, recovery hints; so the next run or the human operator can pick up without re-doing work.

## Further reading

- [METR: Time-Horizon Benchmark](https://metr.org/time-horizons/). The standard reference for how long AI agents can autonomously work. The methodology (HCAST plus RE-Bench plus shorter novel tasks calibrated against human completion times) and the doubling curve since 2019 are the empirical anchor for any long-horizon reliability conversation. The [Time Horizon 1.1 update](https://metr.org/blog/2026-1-29-time-horizon-1-1/) has the latest frontier numbers.
- [Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents](https://arxiv.org/html/2603.29231v1): the late-2026 paper that establishes the capability-vs-reliability distinction, defines meltdown, and proposes MOP-triggered context resetting as the highest-leverage intervention. The framework the rest of the field is converging on.
- [Anthropic: Building effective agents](https://www.anthropic.com/engineering/building-effective-agents). The engineering writeup that distinguishes workflows from agents and lays out the orchestrator-workers and evaluator-optimizer patterns that map cleanly onto long-horizon recovery.
- [Diagrid: Checkpoints Are Not Durable Execution](https://www.diagrid.io/blog/checkpoints-are-not-durable-execution-why-langgraph-crewai-google-adk-and-others-fall-short-for-production-agent-workflows). The most pointed take on what checkpoint-based frameworks (LangGraph, CrewAI, Google ADK) actually guarantee versus what durable execution engines (Temporal, Restate) guarantee. Worth reading even if you stay with checkpoint-based tools, because it tells you exactly which seam you're paying for.
