Tarun Upadhyay.

Research project · work in progress

TraceGuard

State-based runtime verification for tool-using AI agents.

An agent saying “done” is not proof that the world changed. TraceGuard checks the systems where the result should exist.

GitHub: tarun101 The prototype repository will be published after the full prototype gate.

Research question

Can external-state verification outperform agent self-checking and a second-model critic? Can it do so without too many false alarms?

Reproducible prototype output

One real injected-fault trace

This trace is synthetic. It uses no airline, supplier, traveler, payment, or production data.

What the agent saw

  1. Found a Denver hotel at $189 per night.
  2. Created reservation RS-0001 for $378.
  3. Authorized the booking total.
  4. The notification tool returned sent.

Agent verdict complete

Self-check satisfied

What external state showed

  • The supplier reservation existed.
  • The internal booking record matched it.
  • The authorization covered the total.
  • No delivery receipt existed 1,002 seconds later.

Completion contract violated

Injected fault: the notification tool reported success without writing a receipt.

Placeholder · no study data collected yet

Results specification

The full study will compare five verification methods across workflow families, agent models, and unseen variations. This table defines the outputs. It contains no experimental results.

Measure Expected output Current status
Detection rate Silent failures found by each verification method Placeholder. No data.
False alarms Valid completions incorrectly rejected Placeholder. No data.
Detection delay Time from the hidden fault to a verdict Placeholder. No data.
Verification cost Extra tool calls, latency, and compute Placeholder. No data.

Comparison

  1. No explicit verification
  2. Agent self-verification
  3. A separate language-model critic
  4. Deterministic state and invariant checks
  5. TraceGuard’s hybrid monitor

Current boundary

The prototype covers one hotel-confirmation workflow and two injected faults. It is evidence that the environment runs, not evidence for the paper’s broader claim.