MidSphere

Autonomous Context Circuit-Breaker for Data Platforms

Stack: Python · FastAPI · Google Antigravity SDK · Gemini · DataHub MCP · Cloud Run · React Flow

Overview

Closed-loop autonomous circuit-breaker engine for enterprise catalogs. Audits schema drift and PII leaks, scores selective blast radius, and applies advisory quarantine to protect downstream AI agents.

Project Metrics

Closed-loop stages: 5
Decision latency: <4s
Upstream contribution: Skills #88

System Architecture & Implementation

Overview

MidSphere is an autonomous context circuit-breaker engine for enterprise data platforms. Built for the DataHub ecosystem using Google Antigravity SDK and Gemini on Vertex AI, it protects downstream AI agents from consuming corrupted, drifted, or non-compliant catalog data.

Organizations deploy AI agents faster than they can maintain data catalog trust. When schema drift, quality defects, unmasked PII, or freshness violations hit a table, standard data catalogs only record an alert. Downstream AI agents continue reading the catalog and inherit the damage.

MidSphere replaces passive alerting with a closed-loop circuit-breaker: it audits context, calculates selective topological blast radius, applies advisory quarantine to DataHub graph state, generates dry-run remediation artifacts, and produces an MBOM (Model Bill of Materials) attestation.

The 5-stage closed loop

MidSphere organizes autonomous intelligence into five specialized stages orchestrated via Google Antigravity SDK:

Audit (ContextSentinel): Detects schema drift, unmasked PII, and freshness violations across tables using the official DataHub MCP server.
Topology (CascadeAnalyzer): Traverses upstream and downstream lineage graphs, scoring selective blast radius and risk (0-100).
Break (CircuitBreaker): Writes advisory quarantine tags and structured properties to DataHub graph state. Crucially, it isolates impacted tables while keeping parallel unimpacted lineage branches open.
Fix (FixArchitect): Synthesizes verifiable dry-run SQL and dbt remediation patches.
Inherit (MBOMEngine): Generates an MBOM attestation and saves document records to DataHub so downstream humans and agents inherit verified data provenance.

Consumer gate

MidSphere exposes a high-speed Consumer Gate endpoint (GET /v1/gate/{urn}). When a client coding or data agent attempts to query a catalog resource, the gate checks active quarantine status:

Unimpacted datasets: Returns ALLOWED with lineage provenance.
Quarantined datasets: Returns BLOCKED with the active violation reason and blast radius score, preventing prompt injection or hallucination cascades before the agent executes.

Technical architecture

Backend: FastAPI, Pydantic, and Uvicorn running on Google Cloud Run with Vertex AI integration.
Agent Workforce: Google Antigravity SDK (google-antigravity) coordinating multi-stage tools on Gemini with deterministic pipeline fallback.
Catalog Integration: Official DataHub MCP server (mcp-server-datahub) executing real catalog operations (search, get_lineage, list_schema_fields, add_tags, remove_tags, save_document).
Command Center: React 18, Vite, TypeScript, Tailwind, and React Flow deployed on Vercel with real-time interactive graph topology and state scrubbers.
Upstream Contribution: Contributed official skills to DataHub: datahub-project/datahub-skills#88 (datahub-circuit-breaker, datahub-quality-circuit).

Why build this

Scanners and gateways inspect tool calls at the proxy level and block requests. But they have zero awareness of data quality, schema drift, or column-level PII inside the catalog itself.

MidSphere starts where API gateways stop. By embedding circuit-breaker patterns directly into catalog graph state, it ensures that downstream autonomous agents never inherit toxic context.