· Updated · 6 min read
AI Governance for Database Agents: Control Consequences
Database-agent permissions should be based on the records an action can affect and whether the change is reversible, not on valid code or plausible answers.
A seemingly correct query can still affect the wrong records. Heads of data and platform leads must decide how far agents may access live data systems. Valid code or plausible answers do not justify production write access. An agent earns authority by predicting its blast radius and staying within reversible boundaries.
What is AI governance for database agents?
AI governance for database agents assigns permissions based on each action’s predicted blast radius and reversibility, while people approve changes with lasting or wide effects. It governs what an agent may do to live data systems, not merely whether its output appears valid.
Broad AI governance frameworks cover far more ground. The U.S. National Institute of Standards and Technology (NIST) publishes the AI Risk Management Framework. It is a voluntary guide that helps any organization manage the risks of its AI systems, regardless of sector or use case. Governance for database agents is narrower: it determines what an agent may do to live data.
What should AI governance control?
Governance should control what an agent's action will touch, not only whether its code runs. An agent can write a correct query and still not know which records the query will change. These are two different skills. MongoBench is a benchmark from a November 2026 arXiv paper that tests the second skill. It shows a model some records and a query, and asks which records the query selects.
The best of the four models that it tested found exactly the right records in about nine of ten cases (92%). The weakest model found them in about seven of ten cases (70%). The models also wrote test questions of their own, and they got up to three of ten of these wrong. A query that looks correct does not show that the agent knows what the query will change.
MongoBench tested single read queries, not changes to data. But a change selects its records in the same way as a query. So an agent that misjudges what a query selects also misjudges what a change affects. Governance must then answer this question: how much authority does an agent get when it cannot reliably say what an action touches?
How should AI governance frameworks set permissions?
Production authority follows potential impact and reversibility, not one permanent trust label. Authority changes with the action under review. A narrow, reversible action can remain inside established boundaries, while broad or lasting effects require human approval.
Risk combines the probability of an event with the magnitude of its consequences. Potential impact should therefore affect priorities and controls. However, both likelihood and magnitude can be difficult to calculate for novel AI risks. An exact set of affected records measures one part of a database action’s consequence, not its complete risk.
A small predicted set does not justify broader authority by itself. At least one model missed 93% of benchmark items with an empty reference set. The equivalent figure was 37% for items with non-empty sets. The result shows that consequence prediction can fail even when the engine-verified blast radius is zero.
How can enforceable boundaries contain generated code?
Enforceable boundaries let generated code change how fast a system works, but not which results it returns. A correctness firewall is a design from a November 2026 arXiv paper. It keeps a database component that an agent generated inside a trusted system. That trusted system still decides what a correct result is. This gives more protection than a review of the component's output alone.
The paper presents three case-specific patterns. This table shows when each applies, how the system enforces it, and what checking costs.
| Pattern | Source example | Applicability condition | Enforcement | Checking cost |
|---|---|---|---|---|
| Advisory | Cardinality estimate | Equivalent alternatives exist | Advice cannot change results | None |
| Guarded | Searching sorted data | Checker and fallback exist | Check output or use fallback | One check per output |
| Domain certified | Data compression | Small, finite, known at synthesis time | Check every input before deployment | One-time certification |
These patterns do not form a general selection rule. Bad advice can slow execution while results stay unchanged. A guard adds its check cost and the fallback cost after rejection. Certification requires a small domain known during synthesis. All three assume deterministic programs that halt, do not crash, and do not change state outside their scope.
Which decisions still need a person?
People retain approval for broad, lasting, or irreversible actions. Agents perform volume within established boundaries. This separates execution capacity from the judgment that sets production authority.
Separation of duties supports that rule. NIST calls it a best practice to separate people who build and use models from those who verify and validate them. Correctness firewalls provide a technical middle ground because generated components contribute without controlling result correctness.
The firewall examples keep a trusted responsibility outside each generated component. The table identifies the input, output, boundary, and retained responsibility.
| Generated component | Input | Output | Enforced boundary | Retained responsibility |
|---|---|---|---|---|
| Cardinality estimator | Database and representative workload | Cardinality estimate | Advice cannot change results | Optimizer and execution engine |
| Searching sorted data | Search key | Proposed search result | Checker accepts or rejects | Checker and fallback |
| Data compression | Known finite domain | Compression behavior | Certification before deployment | Specification and deployment |
These retained responsibilities are system controls, not undocumented human review steps. People still decide whether broad or hard-to-reverse production changes receive permission. A failed production pipeline shows how this division works in an operational setting.
How does dltHub keep agent diagnosis read-only?
dltHub keeps diagnosis read-only in two ways. Each agent has a declaration of what it may access and do. The agent also runs under a credentials profile that has no production access. The agent inspects and proposes a fix, and a person decides whether to apply it. The job-inspector agent, currently in public preview, as of 10 November 2026, diagnoses failed batch jobs. A deployed job.fail: selector can start an inspection after a production pipeline job fails.
Its documented workflow keeps diagnosis inside these boundaries:
- Input: The agent reads the
run record,logs,job definition,dlt trace, andworkspace source. - Output: It returns a
classification, citedevidence,confidence,open_points, and aproposed_fix. - Access: Its declaration permits read-only local and context access. It excludes destination access, shell execution, writes, deployment, job cancellation, and job reruns.
- Deployment: The agent job declaration pins the
accessprofile withrequire={"profile": "access"}, which keeps production-profile credentials out of its environment. The agent definition separately limits allowed operations.
inspector = run.agent( "dlthub-platform:job-inspector", trigger="job.fail:tag:ingest", require={"profile": "access"}, )
- Human decision: A person reviews the classification, confidence, evidence, proposed fix, target, change, and open points. The person then decides whether to apply it.
Local execution uses the active profile unless the operator supplies a profile.
How does AI governance for database agents relate to AI ethics and compliance?
AI governance for database agents answers the narrower question of what an agent may do to live data systems. Consequence-based permissions can support governance and security, but they do not establish compliance. They also do not replace wider responsibilities for fairness, accountability, transparency, privacy, explainability, safety, or applicable legal requirements.
Book a demo to discuss dltHub.
What is dltHub?
The agentic data layer on top of any data warehouse: one governed layer and a managed runtime to build, govern, heal and serve data for AI, without growing a platform team.
Was this page helpful?