1.Why AI Governance for Delivery Teams is worth getting right
Lessons cover writing approval and policy checks as backlog acceptance criteria, a matching exercise connecting different AI features to the specific governance concern they raise, model version drift, data provenance, output monitoring, and a supply chain lesson on the specific risks of depending on a third-party or open-weight model whose own governance practices the team needs to understand rather than assume. A scenario walks through an AI feature that passed every automated eval but produced a harmful output in production anyway, forcing the team to examine whether their evals were actually testing the right thing or just the easy thing.
2.How it works in practice
The module closes on incident response for AI-specific failures, which differs from a typical production incident because the failure mode is often subtle degradation rather than a clean outage, and because a fix frequently requires retraining or re-prompting work that does not resolve in minutes the way rolling back a deployment does. The throughline is that governance, done well, is invisible most of the time because it is baked into the same Sprint habits the team already runs, not a separate process people learn to route around.
3.Role reality
Governance sounds like paperwork until the first incident. Every line on the right is what actually holds up when a model misbehaves in production.
| Textbook theory | Delivery reality |
|---|---|
| A governance board reviews AI features before release. | Boards that meet monthly review features that shipped three Sprints ago. By the time they look, the review is theatre. |
| Novel AI use cases need the most scrutiny. | Blast radius, not novelty, decides scrutiny. A boring internal tool with no customer data needs less sign-off than an exciting one that moves money. |
| Model evaluation happens once, before launch. | Models drift. Evaluation and drift thresholds are a recurring Definition of Done item, not a one-off gate. |
| Vendors are a procurement concern. | The model and vendor supply chain is a delivery risk. If the vendor changes the model under you, your team owns the fallout. |
| Incident response for AI is the same as any outage. | AI incidents often need a human-override switch and a way to explain what the model did, not just why the service went down. |
4.Core delivery pillars
Four disciplines that keep AI features shippable without slowing every Sprint to a crawl.
Write the three tiers down before the first feature ships: low, medium, high blast radius, each with a named approver. Log the approval in the Sprint Backlog, not in a meeting nobody can find later.
Track model version, vendor terms and change notices the same way you track a library version. A silent model update is a breaking change until proven otherwise.
An AI feature is not Done because it passed a demo. It is Done when its evaluation metrics clear an agreed threshold and drift monitoring is switched on.
Every high blast radius feature needs a documented way to pause the model without redeploying the whole service. Practise using it before you need it.
5.Governance metrics that matter
Numbers that show whether governance is working, tracked alongside delivery metrics rather than instead of them.
Time from feature proposed to sign-off received. Long lead times push teams to skip the process.
How many drift breaches were caught by monitoring versus by a customer complaint.
Of the notices received, how many triggered a re-evaluation before the next release.
How long it takes from decision to pause a model to the model actually being paused.
6.Situations you will be asked to handle
The module puts you inside 3 decisions rather than asking you to recognise the right answer on a list. Each one is a situation practitioners meet, with several defensible options and consequences that follow from the one you pick. The scenarios below are the shape of the judgment the subject demands.
- Lesson 29.4: Game - what a security review will actually accept
- Lesson 29.7: Game - match the control to the failure it prevents
- Lesson 29.11: Decision lab - the model you depend on is being retired
7.Common mistakes and why they fail
Running AI governance as a separate committee outside Sprint cadence
Requiring a review board approval on its own schedule, disconnected from the Sprint, which creates a queue that either becomes a permanent bottleneck or gets quietly bypassed under delivery pressure.
Trusting a passing eval suite as proof the feature is safe
Treating automated evaluation results as sufficient evidence, without checking whether the evals actually probe the failure modes that matter or only the ones that were easy to automate.
Assuming a third-party or open-weight model transfers no governance responsibility
Building on an external model without understanding its own training data provenance, known limitations, or update cadence, then discovering those gaps only after an incident.
Treating an AI incident like a normal outage
Applying standard incident response playbooks built for clean failures to a subtle degradation problem that may require retraining or re-prompting work with no quick rollback available.
8.Questions worth asking before you commit time to this
How do you keep AI governance from becoming a Sprint bottleneck?
Fold governance checks into the same refinement and Definition of Done process the team already uses, so approval evidence is produced as a natural part of doing the work, rather than routing every AI feature through a separate committee with its own schedule.
Is a passing eval suite enough to call an AI feature safe to ship?
Not on its own. Evals need to be checked against the specific failure modes that actually matter for the feature, since it is easy to build a suite that tests convenient scenarios while missing the harmful edge case that eventually reaches production.
What governance responsibility does a team inherit from using a third-party model?
The team still needs to understand the model's training data provenance, known limitations, and update cadence, and to document how those factors were considered, rather than assuming the provider's own diligence transfers automatically.
Why does incident response differ for AI failures compared to typical outages?
AI failures are often subtle degradation rather than a clean outage, and the fix frequently requires retraining or re-prompting work that does not resolve in minutes, unlike a deployment rollback. Response plans need to account for that longer, less certain recovery path.
9.What to remember
- A blast radius model for approvals that ships in weeks, not committees
- Supply chain and evaluation checks that belong in the Definition of Done
- A situational matrix for the calls a governance review actually tests
10.Where this sits in the Scrumling course
Run the day-to-day governance of AI features inside normal Sprint work: approvals, policy, supply chain, evals and incident response.
About 69 minutes of lessons and decision scenarios.
- Lesson 29.1: Who approves an AI feature, and at what blast radius
- Lesson 29.2: An AI usage policy engineers actually follow
- Lesson 29.3: The model and vendor supply chain
- Lesson 29.5: Evaluation as a Definition of Done item
- Lesson 29.6: Prompt and system prompt change control
- Lesson 29.8: Data handling boundaries in prompts
- Lesson 29.9: Incident response and human oversight that has actually been exercised
- Lesson 29.10: Cost and rate limits as a real Sprint constraint
- Lesson 29.4: Game - what a security review will actually accept
- Lesson 29.7: Game - match the control to the failure it prevents
- Lesson 29.11: Decision lab - the model you depend on is being retired
Assessment: AI Governance for Delivery Teams quiz