Curriculum

Defining AI Guardrails and Agent Security

Safety, trust and governance appears in about 30 percent of AI PM postings (83 of 273, across 39 companies), and the report's verdict is right-size: keep guardrails and responsible-AI as a working PM skill (define guardrails in the PRD, require human approval for irreve

1. 12.02

   What an agent may do without asking

   You set what an agent may do unattended from reversibility times measured reliability, write that as a per-action authority table, give it a non-human identity with scoped expiring credentials, and treat money as the irreversible action with spend limits enforced at the payment layer.
2. 12.03

   Agent security

   You treat everything the agent reads as untrusted data rather than a command, and reason about the lethal trifecta, prompt injection, tool misuse, permission escalation and memory poisoning as an agent's own attack surface, distinct from generic responsible-AI.
3. 12.04

   Guardrails in the request path

   You put input and output classifiers in the request path, pay their cost in latency and compute, and account for their false positives against an over-refusal budget, not just a block rate.
4. 12.05

   Red-team the obvious modes

   You run the known attacks on your own product as a written plan, accept that a clean run proves risk and not safety, and keep an incident runbook for the quiet degradation that no outage alarm will catch.
5. 12.06

   Right-sized governance

   You keep the governance habits that earn their weight, answer buyer security reviews by mapping your existing artifacts to NIST AI RMF and by classifying EU AI Act risk from the use case, match controls to reversibility and legal risk, hold the launch veto, document when not to build, and note that governance and safety appear in about 30 percent of postings, 83 of 273 across 39 companies.
6. 12.07

   INTEGRATION CASE

   From repeated runs, you can decide which tool calls may run without asking, and write the memo that defends it.