The Statistical Firewall for
Data & AI Agents.
Treat datasets and agent trajectories like code. StatGit generates 5KB statistical signatures to catch silent tabular drift via Zero-ETL, while AgentGit tracks cognitive and behavioral drift across prompts, tools, and reasoning chains—100% on-device.
| Feature | PSI | JS Div | Status |
|---|---|---|---|
| user_age | 0.021 | 0.004 | PASS |
| income_bracket | 0.184 ↑ | 0.051 ↑ | DRIFT DETECTED |
| email_address | - | - | PII LEAK |
| transaction_vol | 0.080 | 0.012 | PASS |
--anonymize flag suggested to
redact email_address.
| Metric | v1.2 | v1.3 | Δ (B − A) |
|---|---|---|---|
| TUEI (Tool Efficiency) | 0.8120 | 0.9340 | +0.1220 ✓ |
| Decision Path Entropy | 1.4200 | 1.1800 | -0.2400 ✓ |
| Hallucination Score | 0.080 | 0.015 | GROUNDED |
| Avg Tokens / Latency | 920 (3.2s) | 780 (2.8s) | -140 tok ✓ |
DriftAlert regression)..agentgit/agentgit.duckdb.Zero-ETL Cloud Warehouses & On-Device Agent Stack
Four Pillars of Data Integrity
StatGit replaces fragile DAGs and heavy ETL validation pipelines with a lightweight, cloud-native statistical engine.
Zero-ETL Cloud Native
Snapshot data directly from S3, GCS, or Snowflake. Computation happens natively in your cloud warehouse using push-down queries. Only the 5KB mathematical signature stays local.
Privacy-First Diffing
Automated PII scanning blocks sensitive columns. Differential Privacy (DP) noise injection means you version metrics, not raw user data.
Model-Aware Sensitivity
Prioritize alerts based on feature importance. Stop waking up for noise in unused columns; start acting on what drives predictions.
Automated CI/CD Remediation
Connect StatGit & AgentGit to your deployment gates. Automatically quarantine corrupted data, fail pipelines if agent TUEI regresses >10%, and trigger retraining DAGs.
Meet AgentGit: Behavioral Version Control for AI Agents
Where StatGit detects silent drift in tabular datasets, AgentGit detects behavioral and cognitive drift in locally running AI agents—tracking how changing a system prompt, tool schema, or model affects decision-making quality over time. Everything runs on-device; no traces ever leave your machine.
Agent Traces & Prompt Versions
Every complete run is captured as a Pydantic v2 AgentTrace ("Agent Signature") containing inputs,
reasoning chains, timed tool calls, and outputs—tagged against SHA-256-hashed system prompt
commits (v1.2 → v1.3).
TUEI & Decision Path Entropy
Quantify trajectory stability mathematically before deploying a new prompt:
-
TUEI:
success_rate × (1000ms / avg_latency) - Path Entropy: Shannon entropy of step-type sequences to catch erratic loops.
On-Device LLM-as-a-Judge
A local Ollama model asynchronously evaluates traces post-run for Hallucination
Risk, Goal Alignment, and Reasoning
Coherence, feeding a rolling z-score detector that fires DriftAlerts on regression.
(1-Halluc)×.35 + Align×.30 + Coher×.20 + Eff×.15
Local-First Architecture
Instrument custom agent loops or drop in the AgentGitCallbackHandler for LangChain. If the FastAPI
server (:7432) is unreachable, the SDK automatically
falls back to .agentgit/fallback_traces.jsonl with zero
data loss.
Built for the Modern ML & Agent Stack.
Integrate StatGitClient into orchestration tools like Airflow and Dagster, or wrap autonomous agent runs with AgentTracker and LangChain callbacks in just a few lines of Python.
AgentGitCallbackHandler
directly into LangChain or instrument custom reasoning loops with context
managers.
from agentgit import AgentTracker
from agentgit.integrations.langchain import AgentGitCallbackHandler
SYSTEM_PROMPT = "You are a quantitative research assistant..."
# 1. Context Manager for custom agent loops
with AgentTracker(
agent_id="research-agent",
system_prompt=SYSTEM_PROMPT,
model_name="gpt-4o",
tags=["production"],
) as tracker:
tracker.set_input("Find recent papers on LLM drift")
tracker.planning("Search recent literature, then synthesize.")
with tracker.timed_tool_call("web_search", {"q": "LLM drift"}) as tc:
tc.output = web_search("LLM drift")
tracker.reasoning("Primary cause is fine-tuning on conflicting data.")
tracker.finish("Based on recent papers, the primary cause is...")
# 2. Or one-line LangChain drop-in
with AgentTracker(agent_id="lc-agent", system_prompt=SYSTEM_PROMPT) as t:
agent.run(query, callbacks=[AgentGitCallbackHandler(t)])from statgit import StatGitClient
# Connects to your local or remote (S3/GCS) registry
client = StatGitClient()
# Generate privacy-safe synthetic data
df = client.synthesize("reqs.yaml", rows=5000)
# Validate incoming data pipeline
res = client.check("s3://incoming/today.csv", "reqs.yaml")
if not res["passed"]:
print(f"Pipeline blocked. {len(res['issues'])} issues.")
# Query structured drift history
drift_data = client.diff("baseline.csv", "current.csv")
if drift_data["features"]["income"]["status"] == "DRIFTED":
trigger_retraining_dag()statgit serve
agentgit serve
Interactive React & Streamlit Dashboards
Deploy visual observatories in seconds. Inspect marginal distribution overlays in StatGit,
or step through interactive TrajectoryTimelines, EvalPanel judge
scores, and sparkline health trends in AgentGit's React UI.
agentgit commit v1.3
statgit trend
Statistical & Behavioral Version Control
Track continuous feature decay across your dataset history or run
agentgit diff <hash_a> <hash_b> to compare two system prompt
versions across latency, token usage, TUEI, and reasoning entropy.
Unified CLI Workflow
Git-style semantics for both tabular pipelines and autonomous agent loops.
Drag or swipe slider to reveal hidden subpopulation drift
Expose Hidden Drift with Subpopulation Slicing.
Global averages lie. When you average out features across a massive dataset or aggregate agent latency across task types, dangerous regressions cancel each other out—a phenomenon rooted in Simpson's Paradox.
StatGit automatically slices your data across categorical dimensions (like Region or
Age Group)
and
runs statistical tests on every segment.
-
Prevent Silent Failures Catch a 10% performance drop in a specific cohort before it impacts downstream models, even when global metrics appear stable.
-
Automated Cohort Discovery No manual slice definitions required. The engine isolates high-variance dimensions and anomalous agent tags automatically.
Architected & Engineered by Aarya Chaurasia
AI Systems Engineer, Quantitative Researcher, and Statistics & Computer Science student at the University of British Columbia (UBC). Explore my full engineering portfolio, quantitative research systems, and production AI projects.