Hi all, we recently published a paper on operationalizing AI policy by turning policy prose into testable, machine-readable rules that downstream enforcement and evaluation tools can use. We will be presenting this more on this work at the AAAI AI GOV 26 workshop next month!

TLDR; Many governance frameworks and standards produce useful guidance, but that guidance still has to be translated into explicit, checkable constraints before it can be enforced or evaluated in real systems. Our work proposes an extensible way to make that translation more systematic

The gap we are trying to close

AI governance today is largely expressed in prose. Major sources such as the EU AI Act, the NIST AI RMF, sector guidance, and enterprise standards articulate obligations and prohibitions in natural language, and they do so deliberately. Their aim is to remain broadly applicable across contexts, which means they often specify what should hold in principle without committing to a concrete, system-specific pass fail check.

In deployment, however, governance becomes actionable only when it is instantiated as something an engineering workflow can run and verify. The practical tooling ecosystem reflects this. Runtime guardrails, evaluation harnesses, and policy engines such as OPA and Rego can enforce constraints and measure compliance, but they largely presuppose that the relevant constraints already exist as structured rules and tests. Closing that gap requires expert labor. Practitioners must interpret policy clauses, resolve scope and exceptions, determine what evidence counts, and translate the result into executable checks. This translation layer becomes a persistent bottleneck. It is slow, error-prone, difficult to scale across documents and domains, and costly to maintain as policies and systems evolve.

Policy→Tests (P2T) is our attempt to make that missing layer explicit and reproducible by providing a bridge from policy prose to normalized, machine-readable rules suitable for downstream enforcement and evaluation.

What P2T does

P2T converts policy documents into a normalized corpus of atomic rules that downstream systems can consume. It is a pipeline plus a compact JSON DSL.

At a high level, the pipeline

ingests and chunks policy text into addressable spans with stable provenance
mines candidate spans likely to contain obligations and prohibitions
performs schema-guarded extraction

...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments