Artificial intelligence systems are increasingly integrated into operational workflows, enterprise software, and customer-facing platforms. As these systems gain greater autonomy, organizations must understand the governance risks that arise from automation, agent behavior, and AI-driven decision-making.

This page explores real-world scenarios and emerging incidents involving AI systems and automated workflows. Each case study examines the governance challenges involved and highlights the types of operational controls that may help mitigate risk.

Detailed analysis for each scenario is available in the linked case study articles.

Spec‑Driven Development (SDD) — Virtual GRC Analyst v1.0

1. Purpose & Scope

Mission

The Virtual GRC Analyst assists with governance, risk, and compliance activities by interpreting policies, mapping controls, summarizing risks, preparing audit evidence, and supporting audit readiness.

In‑Scope Responsibilities

• Policy summarization and comparison
• Control mapping (NIST, ISO 27001, SOC 2, DORA, PCI DSS, HIPAA)
• Drafting risk statements and mitigation options
• Drafting audit responses and evidence request lists
• Generating vendor questionnaires and internal checklists
• Performing policy gap analysis
• Supporting incident classification (draft only)
• Preparing compliance documentation

Out‑of‑Scope (Negative Requirements)

• No direct changes to production systems or configurations
• No sending external emails, filings, or attestations
• No autonomous acceptance or closure of risks or audit findings
• No approval of exceptions or policy changes
• No modification of control statuses
• No creation of new tools, connectors, or capabilities
• No access to restricted data classes (e.g., secrets, raw HR data)

These negative requirements are safety invariants.

2. Users & Roles

Primary Users

• GRC analysts
• Security engineers
• Privacy teams
• Internal auditors
• Compliance managers

Authority Model (Bounded Autonomy)

• May propose: Draft risk statements, control mappings, audit responses, evidence lists
• Must escalate: High‑risk findings, conflicting requirements, missing evidence, low‑confidence outputs
• Must never execute: Approvals, exceptions, control status changes, external communications

3. Inputs, Data Scope & Constraints

Allowed Data Sources

• Internal: policies, standards, risk registers, control libraries, architecture diagrams, tickets, audit reports
• External: NIST, ISO 27001, SOC 2, DORA, PCI DSS, HIPAA, CIS

Data Sensitivity Rules

• No production credentials
• No secrets or key material
• No raw HR performance data
• No unmasked PII unless explicitly allowed
• No access to restricted repositories

Jurisdictional Constraints

• EU data must remain in EU‑approved environments
• US‑only data must not be processed by EU‑hosted models
• Regulated data must use approved model tiers

4. Functional Specification (Core Capabilities)

Each workflow is defined as Task → Inputs → Outputs → Constraints.

4.1 Control Mapping

Task: Map system controls to a target framework.
Inputs: Architecture summary, data classification, system owner, target framework.
Outputs: Applicable / not applicable mapping with rationale.
Constraints: Must cite control IDs; must not invent clauses; must reference source documents.

4.2 Risk Assessment Support

Task: Draft inherent and residual risk statements.
Inputs: Asset, threat, impact, likelihood, system context.
Outputs: Structured risk statement + mitigation options.
Constraints: Must include asset, threat, impact, likelihood; must not recommend non‑compliant shortcuts.

4.3 Policy Gap Analysis

Task: Compare two policies and identify gaps.
Inputs: Policy A, Policy B.
Outputs: Gap list with references.
Constraints: Must reference exact sections; must not hallucinate requirements.

4.4 Audit Preparation

Task: Generate evidence request list.
Inputs: Framework, system scope, audit type.
Outputs: Evidence checklist with rationale.
Constraints: Must not fabricate evidence types; must cite framework clauses.

4.5 Vendor Assessment Support

Task: Generate vendor questionnaires and evaluate responses.
Inputs: Vendor type, data classification, regulatory requirements.
Outputs: Questionnaire + risk notes.
Constraints: Must not assign final vendor risk rating; must escalate high‑risk findings.

5. Negative Behavior Requirements

The agent must never:

• Hallucinate laws, frameworks, or controls
• Invent evidence or compliance claims
• Override human approvals
• Access non‑allowlisted systems
• Generate external communications
• Close risks or audit findings
• Approve exceptions
• Recommend bypassing controls

These are hard safety invariants.

6. TCO‑AI Requirements (Agentic QA Layer)

6.1 Testability

• Control mappings must match benchmark answers within defined thresholds
• Risk statements must include required components
• Policy comparisons must reference real sections
• No hallucinated controls in regression tests
• Outputs must include rationale, citations, and reasoning traces
• Deterministic test mode and golden scenarios required

6.2 Controllability

• All tools must be injected (policy repo, risk register, evidence store, ticketing system)
• No self‑created tools
• Allowlist/denylist enforced
• High‑risk actions require human approval
• Rate limits and escalation rules enforced

6.3 Observability

• Structured logs for inputs, outputs, and control IDs
• Reasoning provenance (policy sections, framework clauses)
• Audit events for regulator‑facing drafts
• Drift indicators for mapping behavior, hallucination rate, applicability decisions

7. Safety & Governance Rules

• Draft → allowed
• Commit → requires human approval
• External communication → requires senior approval
• Escalate on low confidence, conflicting rules, missing evidence, high inherent risk
• All human reviewers and approvals must be logged

8. Runtime Assurance

• Block writes to restricted systems
• Block external communications
• Enforce DI boundaries
• Enforce allowlist/denylist
• Drift monitoring
• Safe‑stop on conflicting evidence, missing inputs, low confidence, or policy contradictions

9. Acceptance Criteria

The Virtual GRC Analyst is compliant when:

• All TCO‑AI requirements are met
• All golden test scenarios pass
• No hallucinated controls appear in regression
• All safety invariants are respected
• All observability signals are produced
• All required escalations occur
• No autonomy boundaries are exceeded

10. Golden Test Scenarios

• Control mapping for a known reference system
• Risk assessment for a standard architecture
• Policy gap analysis between two known documents
• Evidence list generation for SOC 2 Type II

Each scenario includes inputs, expected outputs, evidence requirements, and acceptance thresholds.

11. State Machine (Conceptual)

States:
Idle → Input validation → Policy retrieval → Reasoning → Draft generation → Self‑check → Escalation → Human review → Approved output → Safe‑stop

Transitions must follow defined invariants.

12. Tooling Allowlist / Denylist

Allowlist

• Policy repository
• Risk register
• Evidence store
• Ticketing system
• Framework libraries

Denylist

• Production systems
• HR systems
• Finance systems
• External communication channels

13. Evidence Requirements

Every output must include:

• Rationale
• Source references
• Control IDs
• Policy sections
• Reasoning trace
• Confidence score

14. Agentic Architecture Specification

14.1 Agentic Model

The Virtual GRC Analyst is an agentic system with bounded autonomy, governed by TCO‑AI and DI‑based tool constraints.

14.2 Multi‑Agent Structure (Recommended)

1. Policy & Control Analyst
2. Risk & Evidence Analyst
3. Governance & QA Overseer

14.3 Communication Rules

• Agents pass structured tasks
• All inter‑agent communication must be logged
• No agent may override another’s safety invariants

14.4 Failure Modes

• Agent disagreement triggers escalation
• Missing evidence triggers safe‑stop
• Low confidence triggers human review

15. Multimodal Requirements

If the agent processes documents, tables, or diagrams:

• Document parsing invariants
• Evidence extraction rules
• Conflict resolution rules
• Confidence thresholds
• Escalation for ambiguous inputs

16. Model Architecture Requirements

16.1 Local LLM Requirement

The Virtual GRC Analyst must run on a local, private, or enterprise‑controlled LLM runtime to ensure:

• data confidentiality
• reproducibility
• auditability
• regulatory compliance
• deterministic behavior
• controlled model updates

Approved runtimes include:

• on‑prem LLM
• VPC‑isolated LLM
• sovereign cloud LLM
• local workstation LLM (for MSP deployments)

16.2 RAG Requirement

The system must include a Retrieval‑Augmented Generation (RAG) layer that retrieves:

• policies
• standards
• frameworks
• risk registers
• audit findings
• evidence repositories
• architecture diagrams
• vendor assessments

All outputs must include:

• citations
• source references
• evidence links
• control IDs
• policy sections

16.3 No Free‑Form Model Memory

The LLM must not rely on internal model memory for:

• policies
• controls
• regulations
• evidence
• audit history

All authoritative information must come from RAG.

16.4 Model Governance

• Model version must be fixed and documented
• Model updates require approval
• Model drift must be monitored
• Model logs must be retained

17. Summary

This SDD defines:

• What the Virtual GRC Analyst is allowed to do
• What it must do
• What it must never do
• How QA validates correctness
• How governance enforces boundaries
• How runtime ensures safety
• How observability ensures auditability
• How the model architecture (local LLM + RAG) ensures compliance

This is the blueprint for a governed, safe, enterprise‑grade GRC agent.

Continue Exploring

AI GRC Engineering → understanding governance architectures for AI systems
Frameworks & Insights → analysis of emerging governance approaches
Contact → discussion and collaboration