Appendix A: Code Examples: Service, Runbook, Mandate, Observer and Restore Artefacts
This appendix holds the fuller versions of the artefacts that Chapters 5 to 26 introduce. Each artefact keeps the field names, identifiers, dates and people of its chapter, so that the appendix and the chapter can be read side by side. The artefacts are illustrative: the fictional organisations Nordvik Region and Hansa Mutual are described in docs/EXAMPLE_ORGANISATIONS.md. Every YAML and JSON block parses and is meant to live in version control; the Rego and SQL blocks are illustrative and are not executed by the build. No artefact is attack tooling.
Each section states the chapter that introduces the artefact and what the artefact must never contain. Legal references in the artefacts are architectural implications, not legal advice.
A.1 Service level objective record
Introduced in Chapter 5. The record fixes what good means for one service, how the burn-rate alerts page and when the evidence for the alerts expires. SLO-HM-004 is the Hansa Mutual settlement-payments record; SLO-NR-002 is the Nordvik Region citizen e-service record with its collector fragment that minimises personal data before it leaves the service. The collector fragment uses the attributes processor and the filter processor's log_conditions form (312, OpenTelemetry, n.d.); (51, OpenTelemetry, n.d.).
The artefact must never contain personal data in telemetry, an objective without a measurement point outside the service, or a claim that alerts work without a dated test.
SLO-HM-004.
record: SLO-HM-004
service: SVC-HM-004 # claims and settlement payments, Tier 1, CIF
version: 1.0.0
effective: 2028-11-01
review_due: 2029-02-01
owner_role: service owner, claims and settlement payments
operator_role: Group Site Reliability and Incident Management
window: {objective: calendar month, alerting: rolling 30 days}
indicators:
- id: availability
spec: share of payment instructions accepted within 3 seconds
good: status not 5xx and latency <= 3s
measured_at: [payments gateway, synthetic probe outside hansa-dc-frankfurt]
objective: 99.95
budget_minutes_per_30d: 21.6
- id: release_time
spec: share of approved payments released to the bank interface
good: released within 15 minutes of approval
objective: 99.0
- id: settlement_deadline
spec: settlement run complete by 06:00 on run days
kind: deadline # binary per run day; not a ratio
checkpoints: ["03:00 half of batches done", "04:30 90 percent done"]
alerts:
burn_rate:
- {severity: page, long: 1h, short: 5m, burn: 14.4, budget_used: 2%}
- {severity: page, long: 6h, short: 30m, burn: 6, budget_used: 5%}
- {severity: ticket, long: 3d, short: 6h, burn: 1, budget_used: 10%}
deadline: {severity: page, rule: checkpoint missed on a run day}
budget_policy:
exhausted: reliability work first; change freeze on SVC-HM-004 except fixes
quarter_end: freeze calendar applies regardless of budget state
evidence:
alert_test: {date: 2028-10-24, method: seeded fault, result: paged in 4 min}
expires_on: [gateway change, alert rule change, collector change, 2029-04-24]
SLO-NR-002.
record: SLO-NR-002
service: SVC-NR-002 # citizen e-service portal, Tier 1
version: 1.0.0
effective: 2028-11-01
owner_role: unit head, service owner
platform_owner: platform lead (observability stack)
window: {objective: calendar month, alerting: rolling 30 days}
indicators:
- id: availability
spec: share of page and API requests answered without server error
objective: 99.9
budget_minutes_per_30d: 43.2
measured_at:
- portal ingress metrics
- synthetic journey probe every 1 min from outside the Region network
- id: submission_success
spec: share of started applications that end with a receipt issued
good: receipt issued; user-caused rejections excluded
objective: 99.5
min_events_per_window: 5000 # below this, probe results only
alerts:
burn_rate:
- {severity: page, long: 1h, short: 5m, burn: 14.4}
- {severity: page, long: 6h, short: 30m, burn: 6}
- {severity: ticket, long: 3d, short: 6h, burn: 1}
routing: platform on-call (15-minute response for Tier 1)
dependencies: [SVC-NR-004, external DNS at SamDrift]
recovery: {rto: 2h, rpo: 5m, evidence: RST-NR-2027-002, result: failed 2027-10-12}
telemetry:
hot_retention_days: 30
archive_months: 13
personal_data: minimised at the collector (attributes/minimise)
Collector fragment for SVC-NR-002.
# Illustrative collector fragment for SVC-NR-002 (citizen e-service)
processors:
attributes/minimise:
actions:
- {key: user.email, action: delete}
- {key: user.full_name, action: delete}
- {key: citizen.id, action: delete}
- {key: client.address, action: delete}
filter/drop_debug:
error_mode: ignore
log_conditions:
- log.severity_number < SEVERITY_NUMBER_INFO
service:
pipelines:
logs:
receivers: [otlp]
processors: [attributes/minimise, filter/drop_debug]
exporters: [otlp/regional-stack]
A.2 Alert-correlation policy
Introduced in Chapter 6. The policy states, per alert class and action, how far the correlation tooling may go, what it may never suppress, and which seeded-miss rates freeze its promotion. The evaluation record shows how the policy is tested.
The artefact must never contain a Level 3 or higher suppression of a customer service level objective alert, reporting per named engineer, or a correlation source whose test rights are undeclared.
ACP-HM-2028-01.
policy: ACP-HM-2028-01
version: 1.2.0
service: SVC-HM-001
slo: SLO-HM-001
decision_record: AIDR-0956
effective: 2028-11-02
expires: 2029-02-01
owner: head-of-site-reliability
correlation_source: lumenscope-embedded # Mode 0 scoring and Mode 2 summaries; declared until the contract grants test rights
classes:
customer_slo:
actions:
group: {level: "Level 2 Supervised"}
summarise: {level: "Level 1 Assisted"}
suppress: {level: "Level 1 Assisted", ceiling: true} # never above Level 1
page: {level: "Level 3 Bounded"}
dependency_health:
actions:
group: {level: "Level 3 Bounded"}
suppress: {level: "Level 2 Supervised", hold_minutes_max: 30}
page: {level: "Level 3 Bounded"}
saturation:
actions:
group: {level: "Level 3 Bounded"}
suppress: {level: "Level 1 Assisted"}
batch_schedule:
actions:
group: {level: "Level 3 Bounded"}
suppress: {level: "Level 2 Supervised", hold_minutes_max: 30}
never_suppress:
- heartbeat-observer-OBS-HM-0001
- slo-burn-page
- recovery-objective-breach
- change-freeze-window
noise_budget:
pages_per_shift_max: 20
actionable_share_min: 0.5
reported_for: rota # never per named engineer (KBV-IT-2024-OPS-02)
sampling:
held_or_merged_share: 0.10
minimum_items_per_week: 20
reviewer_sees_raw_first: true
seeding:
per_class_per_week_min: 1
truth_store: sealed-outside-correlator
stop_conditions:
seed_catch_rate_below_0_90_for_4_weeks: freeze_promotion
seed_catch_rate_below_0_80_for_2_weeks: all_classes_to_level_1
missed_page_on_customer_slo: all_classes_to_level_1_and_review
EVL-HM-2028-03, the evaluation of the policy.
evaluation: EVL-HM-2028-03
policy: ACP-HM-2028-01
policy_version: 1.2.0
golden_set: GLD-HM-2028-01
period: {from: 2028-10-02, to: 2028-10-27}
episodes_total: 41
episodes_seeded: 14
by_class:
customer_slo:
episodes: 14
paged_in_budget: 14
recall: 1.00
suppressed_by_ai: 0
dependency_health:
episodes: 11
paged_in_budget: 10
recall: 0.91
missed: [SEED-HM-07]
miss_layer: grouping
saturation:
episodes: 8
paged_in_budget: 7
recall: 0.88
missed: [SEED-HM-11]
miss_layer: suppression
batch_schedule:
episodes: 8
paged_in_budget: 8
recall: 1.00
seed_catch_rate: 0.86 # 12 of 14
noise:
pages_per_shift_median: {before: 31, after: 19}
actionable_share: {before: 0.41, after: 0.58}
added_delay_seconds: {median: 20, p95: 95}
sampling_review:
items_read: 112
should_have_paged: 1
class_of_finding: saturation
decision:
promotion: frozen # seed_catch_rate below 0.90
suppress_saturation: level_1
actions:
- id: ACT-1
owner: sre-lead
due: 2028-11-17
A.3 Incident record
Introduced in Chapter 7. The record is the single account of an incident: clocks, timeline, hypotheses by author, the assistant's contributions and the decisions with their holders. The assistant's ranking is released after the human hypotheses, and the record shows which evidence was read directly.
The artefact must never contain an assistant-authored cause stated as fact, external communication drafted at a level above its ceiling, or a timeline entry without an author.
INC-HM-2028-0871.
id: INC-HM-2028-0871 # illustrative
service: SVC-HM-001 # claims core, Tier 1, DORA CIF
slo: SLO-HM-001 # p95 claim registration under 3 s
commander: Tobias Richter
ops_lead: sre-hamburg-day-2
assistant: {use_case: HM-OPS-002, aidr: AIDR-0951, level: 1,
model: ops-assist-2028.3, prompt: inc-v14, tools: [read]}
clocks: # trusted time, +01:00
first_signal: "2028-11-20T08:41"
assistant_first_flag: "2028-11-20T08:43"
human_ack: "2028-11-20T08:44"
declared: "2028-11-20T08:46"
timeline:
- {t: "08:41", by: monitoring, e: "SLO burn-rate alert, registration latency"}
- {t: "08:43", by: assistant, kind: drafted, e: "Candidate incident opened"}
- {t: "08:50", by: commander, e: "Hypothesis recorded: database lock waits"}
- {t: "08:52", by: assistant, kind: retrieved,
e: "INC-HM-2027-0833, PRB-HM-0019: config push, canary skipped", used: true}
- {t: "08:53", by: ops_lead, e: "Hypothesis recorded: slow consumers on registration queue"}
- {t: "08:56", by: assistant, kind: ranked, e: "Ranking released after human hypotheses"}
- {t: "09:03", by: commander, e: "Proposes rollback of CHG-HM-6041"}
- {t: "09:03", by: commander, e: "Second-hypothesis rule invoked"}
- {t: "09:09", by: ops_lead, e: "GitOps log read directly: CHG-HM-6034 at 07:36"}
- {t: "09:12", by: ops_lead, e: "Queue depth began rising 07:52, before CHG-HM-6041 at 08:25"}
- {t: "09:15", by: commander, e: "Approves emergency rollback CHG-HM-6043"}
- {t: "09:36", by: monitoring, e: "p95 below 3 s; probe from independent region"}
hypotheses:
- {id: H1, text: "DB index maintenance CHG-HM-6041", by: assistant,
rank: 1, score: 0.71, status: refuted, test: "queue growth predates change"}
- {id: H2, text: "Claims-service memory pressure", by: assistant,
rank: 2, score: 0.12, status: refuted}
- {id: H3, text: "Consumer pool limit CHG-HM-6034", by: assistant,
rank: 3, score: 0.09, also_by: ops_lead, status: confirmed}
assistant_contributions: {retrieval: useful, correlation: listed_both_changes,
ranking: top1_wrong, draft_updates: 2_approved}
decisions: [{what: emergency_rollback, holder: commander, dlg: DLG-HM-OPS-02}]
impact: {sev: 2, start: "08:41", end: "09:36", minutes: 55}
A.4 Change record, change class and change gate
Introduced in Chapter 8. The change record carries the author (a person or an agent), the reconciler that applied it (engine, mode, identity and sync policy), the computed class, the plan summary, the independent approvers, the tested rollback and the verification by a probe the pipeline does not run. The change class file shows what a standard change may touch; the Rego gate computes the class from the diff. The Rego is illustrative: opa is not part of the build, so it is not executed by the tests.
The artefact must never contain a declared class that overrides the computed class, an agent as its own approver, typed prompts (references only), or a claim of recoverability the change does not evidence.
CHG-NR-2028-0007, an AI-authored infrastructure change.
change_record:
id: CHG-NR-2028-0007
service: SVC-NR-002 # Tier 1
title: Automatic renewal of the portal certificate at the recovery site
motivated_by: [RST-NR-2027-002]
requested_by: joakim.wester
author:
kind: agent
tool: agentic-work-tool # Mode 3, platform team permission
identity: wl-nr-platform-draft
model: nr-reserved/open-weight-coder@2028-09-03
instructions_version: skills/cert-renewal@3f9a1c2
inputs: [ticket:CHG-REQ-0412, repo:infra/recovery-site@b7e41d0, runbook:RBK-NR-0003]
tests_run: [terraform-validate, plan-review, staging-apply]
not_tested: [renewal against the production certificate authority]
provenance_stored: references_only # no typed prompts (MBL-NR-IT-2027-01)
reconciler:
mode: push # applied by the pipeline credential; no continuous reconciler
engine: terraform-pipeline
identity: wl-nr-pipeline-apply
sync_policy: "apply on merge inside the window; no automatic re-apply; drift detector read-only"
declared_class: normal
computed_class: normal # Tier 1 service, no catalogue match
plan: {add: 3, change: 1, destroy: 0, sensitive_types: []}
policy_results: {gate: allow, supply_chain: "chart pinned by digest, no new dependency",
sbom_delta: 0, window_check: pass}
approvers:
- {id: maja.lindqvist, kind: person, team: platform}
- {id: oskar.berg, kind: person, team: infrastructure}
window: "2028-10-17 20:00-22:00"
rollback:
plan: "revert commit, destroy 3 added resources, no data touched"
reversibility: full
tested_on: 2028-10-12
result: passed
stop_path: "duty manager and platform engineer on call, pipeline kill switch"
verification:
probe: external-tls-probe # run by monitoring, not by the pipeline
result: "chain valid, renewal object present, 89 days to expiry"
checked_at: 2028-10-17T21:12
claim_not_made: "failover of SVC-NR-002 is not shown to be recoverable; re-test open (Chapter 19)"
Change class HM-STD-002.
change_classes:
- id: HM-STD-002
name: Certificate renewal for services outside critical functions
approved_by: AIDR-0952 # change-risk reviewer and class rules
eligible:
service_tiers: [2, 3]
dora_critical_or_important: false
may_touch: [certificate, secret_reference]
limits: {max_resources: 10, deletes: 0, identity_changes: 0, network_changes: 0}
window: "daily 05:00-07:00 Europe/Berlin"
delegation: {draft: "Level 2 Supervised", release: "Level 3 Bounded"}
verification:
probe: external-tls-probe # not run by the pipeline
pass: "chain valid and at least 30 days to expiry"
rollback: {automatic: true, tested_on: 2028-10-04, result: passed}
history: {instances_12m: 41, correlated_incidents: 0}
suspend_if: "two correlated incidents or one in Tier 1 within a quarter"
sampling: {post_release_rereview: 0.10, seeded_errors: 0.05}
expires: 2029-10-04
The change gate (illustrative Rego).
package change.gate
import rego.v1
default allow := false
sensitive_types := {"kubernetes_cluster_role_binding", "vault_policy", "backup_target"}
destroys contains rc.address if {
some rc in input.plan.resource_changes
"delete" in rc.change.actions
}
sensitive contains rc.address if {
some rc in input.plan.resource_changes
rc.type in sensitive_types
}
# The class is computed from the diff; the author's declaration is a claim.
is_standard if {
count(destroys) == 0
count(sensitive) == 0
input.change.catalogue_match
}
human_approvers contains a.id if {
some a in input.change.approvers
a.kind == "person"
a.id != input.change.requested_by
a.id != input.change.author.id
}
deny contains "declared class is lower than computed class" if {
input.change.declared_class == "standard"
not is_standard
}
deny contains "agent-authored change has no independent human approver" if {
input.change.author.kind == "agent"
count(human_approvers) == 0
}
deny contains "release is outside the authorised window" if {
not input.release.window_open
}
deny contains "no tested rollback plan" if {
not input.change.rollback.tested_on
}
allow if count(deny) == 0
A.5 Progressive-delivery policy
Introduced in Chapter 9. The policy defines the rings, bake times, stop conditions and stop path for one service, and names the verifier that is independent of the author, the pipeline and the telemetry path. The Rego rule shows how the widening of a rollout is checked; it is illustrative.
The artefact must never contain a first ring skipped for any class, a verifier that shares identity or model family with the releaser, or an automatic rollback of an irreversible change on production data.
ROL-HM-001.
policy_id: ROL-HM-001
version: 3
approved: 2028-10-24 # service owner, head of SRE, head of operational resilience (informed)
service: SVC-HM-001
tier: 1
critical_or_important_function: true
applies_to: [code, configuration, feature_flag] # class never exempts the first ring
rings:
- {id: ring0, target: synthetic-test-tenant, traffic_share: 0.00, bake_minutes: 15}
- {id: cell1, target: market-NO, traffic_share: 0.03, bake_minutes: 60, min_requests: 500}
- {id: cell2, target: market-SE, traffic_share: 0.08, bake_minutes: 120}
- {id: cell3, target: [market-NL, market-PL], traffic_share: 0.34, bake_minutes: 240}
- {id: cell4, target: [market-DE, market-UK, market-CH], traffic_share: 0.55, bake_minutes: 0}
concurrency: {max_active_rollouts: 1}
window: {days: [Mon, Tue, Wed, Thu], local_hours: "08:00-15:00", tz: Europe/Berlin,
freeze_calendar: FRZ-HM-2028}
actors:
agent_HM-OPS-001: {max_target_ring: cell1, classes: [standard], level: 3}
agent_other_classes: {max_target_ring: ring0, level: 2}
widening_beyond_cell1: {executed_by: pipeline-controller, on: verifier_pass}
widening_nonstandard_beyond_cell2: {requires: human_approval, role: service-owner}
verifier:
observer: OBS-HM-0001
independent_of: [author_identity, pipeline_identity, model_family, telemetry_vendor_path]
probe_vantage: [cloud-b-eu-central, external-synthetic]
max_result_age_seconds: 300
stop_conditions:
- {id: SC1, kind: relative, signal: error_ratio_delta, gt: 0.02, for_minutes: 3, action: rollback}
- {id: SC2, kind: absolute, signal: registration_p95_s, gt: 3.0, for_minutes: 5, action: rollback} # SLO-HM-001
- {id: SC3, kind: business, signal: claims_registered_vs_control, lt: 0.70, for_minutes: 10, action: hold}
- {id: SC4, kind: fail_closed, signal: verifier_heartbeat_or_data, missing_minutes: 2, action: hold}
- {id: SC5, kind: held_out, definition: "OBS-HM-0001, not readable by the releaser", action: hold}
stop_path:
holders: [incident-commander-on-duty, service-owner, agent-operator]
approval_required: false
mechanism: pipeline-freeze-flag-in-store-outside-pipeline-control-plane
target_seconds: 60
rollback:
executed_by: pipeline-controller
rollback_safety_required: true # two-phase for data format changes
irreversible_on_production_data: never_automatic
evidence:
seeded_fault_test: {cadence_days: 90, last_run: 2028-10-18, result: passed, seconds_to_halt: 94}
Widening rule (illustrative Rego).
package rollout.widen
import rego.v1
default allow := false
rings := ["ring0", "cell1", "cell2", "cell3", "cell4"]
next_ring(current) := rings[i + 1] if rings[i] == current
allow if {
input.request.target == next_ring(input.state.current_ring)
not input.state.hold
not frozen
in_window
verifier_pass
actor_within_mandate
}
frozen if {
some w in data.freeze[input.service]
time.parse_rfc3339_ns(w.from) <= input.now_ns
input.now_ns < time.parse_rfc3339_ns(w.to)
}
in_window if {
input.window.day in {"Mon", "Tue", "Wed", "Thu"}
input.window.hour >= 8
input.window.hour < 15
}
verifier_pass if {
v := input.verifier
v.status == "pass"
v.observer_id == "OBS-HM-0001"
v.ring == input.state.current_ring
v.baked_minutes >= data.bake[input.state.current_ring]
(input.now_ns - v.evaluated_ns) / 60000000000 <= 5
v.identity != input.change.author_identity
}
actor_within_mandate if input.actor.kind == "controller"
actor_within_mandate if {
input.actor.kind == "agent"
input.request.target in {"ring0", "cell1"}
input.change.class == "standard"
}
Change-correlated incident rate (illustrative SQL).
-- Change-correlated incident rate per service and month (illustrative).
-- :lookback_minutes is the declared window, for example 120.
SELECT
i.service_id,
strftime('%Y-%m', i.first_symptom_at) AS month,
COUNT(DISTINCT i.incident_id) AS incidents,
COUNT(DISTINCT CASE WHEN c.change_id IS NOT NULL
THEN i.incident_id END) AS incidents_after_change,
ROUND(1.0 * COUNT(DISTINCT CASE WHEN c.change_id IS NOT NULL
THEN i.incident_id END)
/ COUNT(DISTINCT i.incident_id), 2) AS change_correlated_rate
FROM incidents i
LEFT JOIN changes c
ON c.service_id IN (SELECT dep FROM service_deps WHERE svc = i.service_id
UNION SELECT i.service_id)
AND c.applied_at BETWEEN datetime(i.first_symptom_at,
'-' || :lookback_minutes || ' minutes')
AND i.first_symptom_at
GROUP BY i.service_id, month;
A.6 Runbook manifest
Introduced in Chapter 10. The manifest makes a runbook an executable, reviewable artefact: preconditions, the action, an independent verification, the failure path and the stop path. RBK-HM-0007 is the Level 3 Bounded restart of one stateless web instance.
The artefact must never contain a verification that reads what the action resets, a stop path inside the agent platform, or a ceiling above the delegation profile.
RBK-HM-0007.
runbook: RBK-HM-0007
title: Restart one unhealthy stateless web instance
version: 3
service: SVC-HM-005
owner_unit: Group Site Reliability and Incident Management
accountable: Tobias Richter
decision_record: AIDR-0970
action_class: restart
delegation:
profile: DLG-HM-OPS-001
mandate: MND-HM-OPS-001
level: Level 3 Bounded # ceiling for this runbook
never_above: Level 3 Bounded
preconditions:
- target.role == "stateless-web"
- fleet.healthy_after_action >= 4
- fleet.healthy_after_action / fleet.size >= 0.75
- calendar.freeze == false
- runs.last_60_minutes.same_service < 3
- evidence.captured == true # logs and process state saved first
dry_run: required # plan and policy check, no effect
action:
step: restart-instance
targets: 1
max_runs_per_target_per_hour: 1
verification:
probe: synthetic-broker-login
location: probe-eu-b # outside the cluster and the agent's identity
soak_minutes: 10
pass_when: 5 consecutive passes at 60 second intervals
on_failure:
rollback: keep instance cordoned for diagnosis, add one instance (RBK-HM-0009)
then: escalate to incident commander, no second restart
stop:
held_by: [incident commander, agent operator, platform engineer on call]
path: STOP-HM-ITOPS # outside the agent platform, no approval needed
review:
last_dry_run: 2028-10-17
review_due: 2029-01-17
approved_by: Tobias Richter
A.7 Capacity record
Introduced in Chapter 11. The record turns a forecast into a proposal: the backtest, the uplift for under-coverage, the guardrails that capacity may never be cut below and the person who approved the proposal. The agent proposes at Level 1 Assisted.
The artefact must never contain a forecast without a backtest, a reduction of the standby floor before a failover re-test, a cost estimate taken from the agent instead of the provider export, or resources under retention or protection labels.
CAP-HM-2028-014.
record: CAP-HM-2028-014
service: SVC-HM-004 # settlement payments, Tier 1, CIF
version: 1.0.0
date: 2028-11-14
owner_role: service owner, claims and settlement payments
proposed_by: HM-OPS-004 # capacity and FinOps agent, AIDR-0953
mandate: MND-HM-OPS-004
level: 1 # Level 1 Assisted: proposes, a person approves
horizon: {from: 2028-12-15, to: 2029-01-10}
demand:
unit: payment instructions per hour at the gateway
history: {from: 2027-10-01, to: 2028-11-10, gaps: ["2027-11-16 cooling failure"]}
known_events: [{date: 2028-12-29, name: quarter-end settlement run, source: calendar}]
forecast:
method: seasonal model with calendar regressors
peak_p50: 38000
peak_p95_raw: 48000
backtest: {folds: 5, scheme: rolling origin, nominal_coverage: 0.95, measured_coverage: 0.80}
uplift: {factor: 1.15, reason: measured coverage below nominal on quarter-end peaks}
peak_p95_planned: 55200 # 48000 x 1.15
capacity:
unit_capacity_per_replica: 10000
target_utilisation_at_p95: 0.70
required: 78857 # 55200 / 0.70
provisioned_now: {replicas: 6, capacity: 60000}
proposal: {replicas: 8, capacity: 80000, action: scale, reversible: true}
guardrails:
redundancy_floor: {replicas_min: 6, zones_min: 2}
standby_floor: {site: cloud-a, nodes_min: 4, basis: RST-HM-2027-007, reduce_only_after: failover re-test within RTO 2h}
never_in_scope: [immutable backups, retention-bound data, protected-label resources]
window: outside freeze calendar except scale-out agreed before freeze opens
veto: [service owner, incident commander, agent operator]
cost:
extra_monthly_eur: 1900
basis: provider usage export, not agent estimate
verification:
observer: synthetic probe outside hansa-dc-frankfurt, SLO-HM-004 burn
review: 2029-01-12 forecast error versus actual peak, recorded in this record
expires_on: [service redesign, capacity model change, 2029-04-14]
decision: {by: service owner, date: 2028-11-16, result: approved}
A.8 Post-incident review and problem records
Introduced in Chapter 12. The review record keeps contributing conditions apart from the reporting convention; the problem record tracks each action to verified closure. The closure rule is small enough to run in a pipeline and shows how an expired risk acceptance reopens an action.
The artefact must never contain a single root cause as the whole account, an action closed without a signature and a retest result, or an assistant-assigned cause.
PIR-HM-2027-0833.
pir:
id: PIR-HM-2027-0833
incident: INC-HM-2027-0833
held: 2027-10-07
facilitator_role: group-site-reliability-reviewer # not on the incident
participants: [incident-commander-hamburg, platform-engineer-on-call, ai-platform-owner, claims-product-ops]
timeline:
drafted_by: HM-OPS-002 incident assistant # AIDR-0951, Level 1 Assisted
drafting_service_version: incident-drafter-2.4
entries: 23
verified_by_person: 23
seeded_errors: {planted: 1, found: 1}
independent_accounts_written_before_session: 4
contributing_conditions:
- id: C1
text: "Routing configuration pushes were classed as standard change"
evidence: [CHG-HM-6101, change-policy-v7]
- id: C2
text: "The canary stage was optional in the standard-change pipeline template"
evidence: [pipeline-template-std-3, run-log-8841]
- id: C3
text: "Pre-production checks validated the schema, not the per-market overlays"
evidence: [precheck-config-report-8841]
root_cause_for_reporting: C2 # reporting convention; the account is C1 to C3
analysis_author_roles: [group-site-reliability-reviewer, ai-platform-owner]
ai_in_analysis: "similar-incident retrieval, after accounts; no cause assigned"
decision_record: AIDR-0962 # drafting and analysis policy for incident reviews
problem: PRB-HM-0019
PRB-HM-0019.
prb:
id: PRB-HM-0019
opened: 2027-10-07
owner_role: head-group-ai-platform
service: SVC-HM-007
linked_incidents: [INC-HM-2027-0833]
known_error: true
workaround: "RBK-HM-0031 section 4: roll back routing config to last signed version"
as_of: 2027-12-15
actions:
- id: A1
text: "Config pushes to AI gateway routing are normal change; canary stage mandatory"
change: CHG-HM-6112
state: closed_verified
retest: {type: game-day, id: GD-HM-2027-04, date: 2027-11-25, result: "push without canary blocked by pipeline policy"}
signed_by_role: head-operational-resilience
- id: A2
text: "Rollback of routing config within 10 minutes of the decision"
state: retest_failed
retest: {id: GD-HM-2027-04, date: 2027-11-25, result: "14 minutes; approval waited for a second approver"}
next_retest: 2027-12-09
- id: A3
text: "Pre-production checks validate per-market overlays"
change: CHG-HM-6118
state: implemented_unverified # reported as open
next_retest: GD-HM-2028-01
- id: A4
text: "Cell-based split of the gateway per market"
state: risk_accepted
accepted_by_role: group-cio
expires: 2028-06-30
Closure rule.
from datetime import date
REPORTED = {
"open": "open",
"implemented_unverified": "open",
"retest_failed": "open",
"closed_verified": "closed",
"risk_accepted": "accepted",
}
def reported_state(action, today):
state = action["state"]
if state == "closed_verified":
retest = action["retest"]
if not (action.get("signed_by_role") and retest.get("result")):
return "open", None # closure needs a signature and evidence
return "closed", (today - retest["date"]).days
if state == "risk_accepted" and action["expires"] < today:
return "open", None # an expired acceptance reopens the action
return REPORTED[state], None
A.9 Model-change record and quality objectives
Introduced in Chapter 13. A model change is a normal change with an offline gate, shadow, canary and ramp, graded by a party that does not run the change. The quality objective record gives the indicators the gate and canary use.
The artefact must never contain a model change classed as standard, a canary graded by the agent's own credentials, or a rollback target that has been retired without a replacement.
CHG-HM-2028-0412.
# Illustrative. Hansa Mutual, claims-triage agent (SVC-HM-002).
id: CHG-HM-2028-0412
type: normal # model changes are never "standard" (cf. INC-HM-2027-0833)
trigger: provider notice 2028-10-12, retirement of snapshot m-2027-06 on 2028-12-18
change: model snapshot m-2027-06 -> m-2028-08; prompt and index unchanged
manifest: {from: RM-HM-002-2028-09, to: RM-HM-002-2028-11}
requested_by: Anja Weiß
change_authority: Tobias Richter # does not run the evaluation
aidr: AIDR-0954
substantial_modification_check: not high-risk under AIDR-0024; recorded, no action
stages:
offline_gate:
set: eval-hm-triage-v7, 1,800 cases, six markets, 5 repeats
restraint_violations: {candidate: 0, gate: 0}
route_agreement: {baseline: 94.1, candidate: 94.6, ci_lower_diff: -0.4, gate: ">= -1.0"}
format_valid: {candidate: 99.8, gate: 99.5}
latency_p95_s: {candidate: 21, gate: 30}
shadow: {from: 2028-10-24, to: 2028-10-31, restraint_divergences: 0}
canary:
share: 5% # assigned by hash of claim id
from: 2028-11-01
to: 2028-11-08
minimum_graded_cases: 600
observer: OBS-HM-AI-01 # sentinel set plus blind sample, own credentials
ramp: [25%, 50%, 100%] # 24 h hold at each step
stop_conditions:
- any restraint violation
- route_agreement lower bound below 92.0 in canary
- burn rate above 14.4 over 1 h on SLO-HM-002-Q
rollback:
until_2028-12-18: route to m-2027-06 at the gateway, expected 10 minutes
after: route to the Frankfurt fallback model, last evaluated 2028-09-05, agreement 91.2
review_of_affected_cases: list by manifest id, handler second look within 2 working days
stop_path: gateway flag, held by incident commander, platform engineer on call, service owner
outcome: promoted 2028-11-14; canary 5% graded 640 cases, agreement 94.3
SLO-HM-002-Q.
# Illustrative. Extends SLO-HM-002 (Chapter 5) for the claims-triage agent.
id: SLO-HM-002-Q
service: SVC-HM-002
owner: Anja Weiß # accountable for the model and the gateway
window: rolling 28 days
manifest: RM-HM-002-2028-09 # release manifest the objectives apply to
objectives:
- sli: route_agreement
definition: share of sampled claims whose route equals the handler-confirmed route
grader: blind human sample, 1 in 200 outputs, minimum 100 per week
target: ">= 93.0%"
interval: 95% confidence, lower bound
- sli: restraint_violations
definition: actions outside the approval rules, per 10,000 runs
grader: deterministic policy check at the policy decision point
target: "0"
budget: none; any violation opens a problem record and holds promotion
- sli: format_valid
definition: outputs that parse against the event schema
target: ">= 99.5%"
- sli: decision_latency_p95
target: "<= 30 s" # as SLO-HM-002
- sli: index_freshness
definition: age of the oldest policy-document change not yet searchable
target: "<= 4 h for 99% of changes"
drift:
signal: population stability index on claim-type mix, weekly
review_threshold: 0.2
action: open change review; do not retrain or retune automatically
observer: OBS-HM-AI-01 # sentinel set of 150 cases, hourly, own credentials
burn_rate_alerts:
fast: {window: 1h, threshold: 14.4, page: incident_commander}
slow: {window: 6h, threshold: 6, page: platform_engineer_on_call}
A.10 AI incident record
Introduced in Chapter 14. The record adds, to the ordinary incident record, the classes of AI incident, the stop outside the system, the evidence preserved before change and one clock per regime with its owner and state. The clock sheet is the per-service statement of which regimes apply. The report time limits in its DORA clock follow Article 5 of Delegated Regulation (EU) 2025/301: the initial notification within 4 hours of classification and at most 24 hours of awareness, the intermediate report within 72 hours of the initial notification, and the final report within one month of the intermediate report (12, European Commission, 2025).
The artefact must never contain a clock without an owner, a reporting decision without its reason, or an AI contribution not marked as draft. The legal statements are architectural implications and not legal advice.
INC-NR-2028-0163.
record: INC-NR-2028-0163
service: SVC-NR-001 # benefits-triage service, Tier 1, high-risk AI system
classes: [C, A] # provider change; wrong ranking output
status: closed_pending_provider_report
timeline:
provider_change_applied: "2028-11-13T22:40" # outside the notified window of 2028-11-21
first_signal: "2028-11-14T08:12" # ranking-stability probe breached threshold
awareness: "2028-11-14T09:20" # incident commander declares
stop_executed: "2028-11-14T09:34" # ranking switched off at the Region's gateway
provider_informed: "2028-11-14T10:05" # Article 26(5), before any authority
requeue_verified: "2028-11-14T11:30"
effect:
claims_ranked_in_window: 212
decisions_issued_on_altered_ranking: 0 # caseworkers decide every case
delay: "under one working day"
roles:
incident_commander: Oskar Berg
stop_held_by: [duty manager, platform lead]
stop_outside_system: "ranking flag at the Region's integration gateway, not in TriageSoft"
independent_probe: "service desk sample of 30 claims, run by Sara Ekdahl's team"
containment:
rung: 1 # fallback: caseworkers use date order
preserved_before_change: [gateway logs, TriageSoft version string, probe results]
restart: "separate decision by the service owner after the provider's report"
ai_contributions:
- tool: NR-OPS-001 # operations assistant, Level 0 Advisory
did: "drafted timeline; retrieved INC-NR-2026-0214 as similar"
marked_draft: true
reviewed_by: Oskar Berg
clocks:
- regime: AI Act Articles 26(5) and 73
start: "2028-11-14T09:20"
limit: "2028-11-29T09:20" # 15 days, Article 73(2)
owner: Head of Benefits Unit with the data protection officer
state: closed_not_met
decided: "2028-11-16"
reason: "ordering only; no decision taken on it; Article 3(49) not met; reassess on provider report"
reassess_on: "2028-11-24"
- regime: NIS 2 Article 23
start: "2028-11-14T09:20"
limit: "2028-11-15T09:20" # 24 hours, early warning
owner: Henrik Dahl
state: assessed_not_significant
decided: "2028-11-14T14:00"
- regime: GDPR Article 33
start: "2028-11-14T09:20"
limit: "2028-11-17T09:20" # 72 hours
owner: data protection officer
state: assessed_no_breach
decided: "2028-11-14T15:30"
- regime: DORA
state: not_applicable
Clock sheet for SVC-HM-008.
clock_sheet_for: SVC-HM-008 # underwriting support, life and health, Annex III 5(c)
instruments_read_on: "2026-11-10"
awareness_rule: earlier of person_reasonable_certainty and supplier_notice
regimes:
- name: AI Act Article 73
applies: "yes, Hansa Mutual is provider and deployer"
limits: {general: "15 days", widespread_or_critical_infrastructure: "2 days", death: "10 days"}
to: "market surveillance authority of the Member State where the incident occurred"
owner: "head of the group AI platform with group legal"
overlap: "Article 73(9) equivalence with DORA decided by counsel; open until then"
- name: DORA Articles 17 to 19
applies: "if classified major under Article 18; service is not a critical or important function"
limits: {initial: "4 hours from classification, at most 24 hours from awareness", intermediate: "72 hours from the initial notification", final: "one month after the intermediate report"}
to: "relevant competent authority"
owner: head of operational resilience
- name: GDPR Articles 33 and 34
applies: "yes, health data"
limits: {authority: "72 hours", data_subjects: "without undue delay if high risk"}
owner: data protection officer
- name: NIS 2 Article 23
applies: "decided by counsel; DORA is lex specialis for the EU entities"
owner: group legal
A.11 Cost-and-latency budget
Introduced in Chapter 15. The budget holds the capacity, unit cost, objectives, fallback ladder and stop for an AI service. The Hansa Mutual record adds per-run, daily and monthly cost limits; the energy claim is made only where it is metered.
The artefact must never contain an energy claim without a metered boundary, a critical service that halts on a budget breach without the manual rung staffed, or a stop inside the agent.
CAP-NR-AI-001.
# docs/15: illustrative; Nordvik Region, capacity and cost record
record: CAP-NR-AI-001
service: SVC-NR-005 # citizen-service assistant, Mode 4
decision: AIDR-0915 # 2028-11-14; reviewer: CISO, data protection officer
owner: Maja Lindqvist # platform lead; service owner: Sara Ekdahl
effective: 2028-11-14
review_by: 2029-02-14 # or on any model, image, price or provider change
capacity:
form: reserved accelerators, Nordic hosting provider
allocated_cost_eur_month: 53300 # illustrative share of EUR 1.6 million a year
load_test: {date: 2028-10-19, config: pool-v3, saturation_rps: 11}
planned_utilisation_max: 0.65 # interactive pool; headroom kept
volume_illustrative: {conversations_month: 60000, peak_driver: benefit decision dates}
unit_cost:
eur_per_conversation: 0.89 # allocated cost / conversations
p95_tokens_per_conversation: 5200 # measured 2028-10, new
objectives:
latency_p95_s: 8 # SLO-NR-005
quality_floor: {golden_set: GS-NR-005-v4, pass_rate_min: 0.90, last: 0.93, date: 2028-11-07}
fallback_ladder:
- {rung: 1, model: primary, slo_p95_s: 8, pass_rate: 0.93}
- {rung: 2, model: smaller open-weight model, same pool, slo_p95_s: 6, pass_rate: 0.88, tier: public}
- {rung: 3, path: published FAQ search, no generation, pass_rate: n/a}
- {rung: 4, path: hand-off to the service desk, hours: 07:00-17:00 working days}
trigger: {probe: external synthetic probe OBS-NR-0001, failure_domain: outside the Region Cloud}
stop:
holder: [duty manager, platform lead]
action: open gateway circuit; rung 3
last_drill: {date: 2028-10-26, result: rung 2 in 41 s, rung 3 in 4 min, hand-off staffed in 12 min}
energy:
claim_available: false
reason: provider supplies metered kWh per reserved accelerator from 2029-01 (requested, contract NR-2026-031 annex)
boundary_when_available: accelerators + host + facility share, idle included, per completed conversation
CAP-HM-AI-001.
record: CAP-HM-AI-001
service: SVC-HM-002 # claims-triage agent service, Mode 5
decision: AIDR-0965 # 2028-11-21; reviewers: chief risk officer, SRE lead
effective: 2028-11-21
delegation_profile: DLG-HM-OPS-CTA-01 # new; adds cost ceilings, levels unchanged
objectives:
slo: SLO-HM-002 # triage within 30 s for 95 % of claims
model_share_p95_s: 20 # internal budget for the model portion
cost_per_triaged_claim_eur: {p50: 0.04, p95: 0.09, ceiling_p99: 0.12}
budgets:
per_run: {max_steps: 12, max_cost_eur: 0.36, on_breach: stop run, route claim to manual queue}
per_day: {cap_eur: 1900, warn_at: 0.8}
per_month: {cap_eur: 38000, burst_share_max: 0.12} # burst on Cloud A, EU region
scale: {max_instances: 24, max_cost_eur_hour: 55, during_freeze: forbidden}
capacity:
primary: Frankfurt pool (hansa-dc-frankfurt), reserved
burst: Cloud A, EU region, reserved
fallback_provider: managed model service at Cloud B, data tiers up to internal only
ladder_on_budget_breach:
- rung 2: self-hosted second model, Frankfurt # tested 2028-10-26, pass 0.91
- rung 3: degraded manual routing in 15 min # RCV-HM-CTA-01, drilled 2028-11-02
stop:
holder: [service owner, on-call SRE]
path: revoke triage credential at the gateway identity service # outside the agent
critical_service_rule: breach degrades, never halts without rung 3 staffed
register: {dora_register_of_information: updated 2028-11-20, entry: fallback provider}
energy: {claim: 1.8 Wh per triaged claim, boundary: accelerators + host + facility via PUE, idle included, period: 2028-10, metered: rack PDU}
A.12 Mandate manifest
Introduced in Chapter 16. The mandate says what one agent may do, at which level, under which runbooks, with which credentials and budgets, and who can stop it. The Rego rule shows the decision point outside the agent; the promotion policy shows how evidence moves an action class up or down. The Rego is illustrative.
The artefact must never contain delegation of deploy, delete or restore-overwrite, authority to change its own mandate or skills, a standing change credential, or a stop path the agent can reach.
MND-NR-OPS-003.
mandate: MND-NR-OPS-003
version: 1.0.0
status: active
decision: AIDR-0916
effective: 2028-11-14
review_by: 2029-02-14
expires: 2029-05-14
agent:
id: NR-OPS-003
role: runbook-runner
usage_mode: 4
identity: spiffe://regioncloud.nordvik.example/ops/runbook-runner
model: open-weight, reserved capacity, hall B # declared family on file
owner: Maja Lindqvist # platform lead
operators: [Oskar Berg, Joakim Wester] # duty manager rota may stop without approval
level_authority: Henrik Dahl # approves any rise in level
scope:
services: [SVC-NR-002]
environments: [production, staging]
blackout_calendar: calendars/decision-dates.yaml # benefit decision days
actions:
read: {level: 3}
annotate: {level: 3}
notify: {level: 3}
scale: {level: 2, runbooks: [RBK-NR-0003]}
restart: {level: 2, runbooks: [RBK-NR-0002]}
reconfigure: {level: 2, runbooks: [RBK-NR-0004]} # certificate renewal
roll_back: {level: 1}
fail_over: {level: 1}
deploy: undelegated
delete: undelegated
restore_overwrite: undelegated
approval:
approvers: [duty-manager, platform-lead]
max_age_minutes: 15
credentials:
issuer: token-service.regioncloud.nordvik.example
lifetime_minutes: 10
audience: ops-policy-decision-point
budgets:
actions_per_hour_per_service: 6
actions_per_day: 20
autonomy_budget: {window_days: 28, tolerance: "per Level 3 class, as in Chapter 4", review_at_percent: 70, on_full_consumption: demote_one_level}
breakers: [same_component_3x_in_30min, no_metric_change_after_action, scope_over_catalogue]
separation: [does_not_write_runbooks, does_not_verify_own_result, does_not_edit_observer_config]
never: [change_own_mandate_or_skills, silence_alert, disable_probe_or_logging,
act_in_freeze_on_tier1, delete_data, restore_over_existing, decide_person_access]
verification: {probe: OBS-NR-0001, independent_of_agent: true}
memory: {between_runs: none, run_log: append_only}
stop:
authority: [duty-manager, platform-lead, service-desk-lead]
path: out-of-band revocation at token service; verified by RST-style drill
last_drill: 2028-11-07
Decision rule (illustrative Rego).
package ops.mandate
import rego.v1
default allow := false
# Level 3 Bounded: catalogued runbook, in scope, not frozen, within budget.
allow if {
base_ok
action.level == 3
}
# Level 2 Supervised: as above, plus a recorded and recent approval.
allow if {
base_ok
action.level == 2
input.approval.by in mandate.approval.approvers
time.now_ns() - input.approval.at_ns < mandate.approval.max_age_minutes * 60 * 1000000000
}
base_ok if {
mandate.status == "active"
input.token.audience == mandate.credentials.audience
input.service in mandate.scope.services
input.runbook in action.runbooks
not frozen
data.budget.remaining[input.agent] > 0
}
frozen if input.service in data.freeze.active
mandate := data.mandates[input.agent]
action := mandate.actions[input.action_class]
PRM-HM-OPS-001.
promotion_policy: PRM-HM-OPS-001
applies_to: MND-HM-OPS-001
decision: AIDR-0966
evidence_max_age_days: 90
steps:
shadow: {min_days: 30, min_proposals_per_runbook: 50, max_disagreement_rate: 0.05}
offline: {held_out_cases: 40, authored_by: not-agent-author, refusal_cases_min: 10}
seeded: {scope: written, window: designated, observer: OBS-HM-0001}
stop: {max_seconds: 300, drill_age_days: 90}
promotion_authority: group-ai-governance-board
demotion:
owner_or_commander: immediate
autonomy_budget_exhausted: one_level_for_class
independent_probe_near_miss: pending_review
model_skill_or_tool_change: re-evaluate
evidence_expired: level_0
manual_practice: {interval_days: 90, runbooks: [RBK-HM-0007, RBK-HM-0012], agent_off: true}
A.13 Observer manifest
Introduced in Chapter 17. The manifest declares the probes, heartbeat and stop path that verify an agent's work, and the five independence properties of the observer. The seeded-fault record shows the test that the observer must pass.
The artefact must never contain a verification through the endpoint the action resets, a heartbeat channel inside the observed estate, or an unexpired statement of independence without a review.
OBS-HM-0001.
observer_manifest:
id: OBS-HM-0001
version: 1.0.0
decision: AIDR-0960
services: [SVC-HM-001, SVC-HM-002]
owner: head-of-site-reliability
independence_reviewer: head-of-operational-resilience
effective: 2028-11-13
statement_of_independence_expires: 2029-02-11
performer:
agent: HM-OPS-001
mandate: MND-HM-OPS-001
runs_in: hansa-dc-frankfurt
service_runs_in: cloud-a
telemetry_vendor: lumenscope
probes:
- id: OBS-HM-0001-P1
claim: "a test claim can be registered"
method: scripted-synthetic-transaction # no model involved
from: [cloud-b, polish-service-centre-network]
interval_seconds: 60
page_after_consecutive_failures: 2
identity: observer-probe-claims # read, create test claim only
- id: OBS-HM-0001-P2
claim: "triage returns a route within 30 seconds" # SLO-HM-002
method: scripted-synthetic-transaction
from: [cloud-b]
interval_seconds: 60
page_after_consecutive_failures: 3
provider_status_feeds: [cloud-a, cloud-b, lumenscope] # telemetry, never a verdict
heartbeat:
receiver: external-heartbeat-service # outside cloud-a and lumenscope
interval_seconds: 60
page_after_missed: 3
page_channel: contracted-sms-gateway # not ServiceHub, not Lumenscope
stop_path:
holders: [incident-commander, agent-operator, platform-engineer-on-call]
approval: none
mechanism: [revoke-workload-identity-at-idp-emergency-control,
disable-orchestrator-trigger-at-network-edge]
runs_outside: [group-ai-platform, servicehub, lumenscope]
target_seconds: 300
last_drill: {id: SPT-HM-ITOPS-2028-Q4-01, date: 2028-10-24, seconds: 250, result: passed}
first_drill: {id: STOP-HM-ITOPS-2027-Q1-01, seconds: 1020, result: failed}
next_drill: 2029-01-24
delegation:
restart: {level: "Level 3 Bounded", condition: "P1 and heartbeat green"}
roll_back: {level: "Level 2 Supervised"}
fail_over: {level: "Level 1 Assisted"}
independence:
model: "verification scripted; critic on family-b, questions only"
data: "outcome probes, not the health endpoint"
authorship: "clauses written by sre team, reviewed by person"
credential: "observer identities separate and read-only"
failure_domain: "cloud-b and service-centre network"
time: "re-verify 10 minutes after each Level 3 action"
SDF-HM-2028-002.
seeded_fault:
id: SDF-HM-2028-002
kind: silent-failure
service: SVC-HM-001 # pre-production mirror only
authorised_by: head-of-operational-resilience
change_record: CHG-HM-6003
seeded_on: 2028-10-24
author: sre-outside-agent-team
hypothesis: "After a restart under RBK-HM-0007 an independent probe fails
the claim-registration transaction within 3 minutes if the
service is not serving, whatever the health endpoint says"
seed: "connection pool exhausted under load; health endpoint keeps
reporting ready"
stop: "white team can end the exercise by revoking the exercise token"
undo: "restore pool setting from CHG-HM-6003 rollback"
observed:
agent_verification: passed # health endpoint, same identity as restart
agent_report: "restart verified"
independent_probe_first_failure_seconds: 61
independent_probe_page_seconds: 122
exercise_ended_by: incident-commander-on-exercise-rota
result: agent verification missed the seed; independent probe caught it
review: PIR-HM-2028-0233
findings:
- "verification clause of RBK-HM-0007 reads the endpoint the restart resets"
- "restart identity could write the health-check definition"
- "clause drafted by the model family that drives the agent"
A.14 Agent permission manifest and refusals
Introduced in Chapter 18. The manifest classifies what an agent reads by who can write it, and the refusals hold at every level. The decision log is replayable. The Rego is illustrative and must be tested with the organisation's policy engine before use.
The artefact must never contain an allow rule in this package (allow rules live in the mandate package), untrusted text acting above notify without an independent probe, or a decision log held by the agent.
MND-HM-OPS-001 input classes.
mandate: MND-HM-OPS-001
version: 1.4.0
decision: AIDR-0968
inputs:
writer_classes:
WC-1: {sources: [probes, platform-events, deployment-events], authority: trigger}
WC-2: {sources: [signed-runbooks, change-records, mandates], authority: define}
WC-3: {sources: [tickets, broker-and-customer-text, user-controlled-log-fields], authority: inform}
WC-4: {sources: [vendor-feeds, provider-ai-output, external-skills, model-summaries], authority: inform}
summary_inherits: lowest-class-of-inputs
untrusted_context_rule:
applies_to: above-notify
requires: [independent-probe, or, recorded-human-approval]
runbooks: {must_be_signed: true, verifier: outside-agent, on_failure: level_0}
credentials: {lifetime_minutes: 10, standing_change_credentials: none}
decision_log: {store: append-only-outside-agent, retention_months: 13}
Refusals (illustrative Rego).
package ops.guard
import rego.v1
# Refusals that hold at every level. Allow rules: ops.mandate (Chapter 16).
read_only := {"read", "annotate", "notify"}
mandate := data.mandates[input.agent]
delegated if is_object(mandate.actions[input.action_class])
untrusted_context if {
some c in input.context_classes
c in {"WC-3", "WC-4"}
}
deny contains "action_not_delegated" if not delegated
deny contains "class_closed_to_agents" if {
input.action_class in {"delete", "restore_overwrite"}
}
deny contains "no_authorising_record" if {
not read_only[input.action_class]
not input.record.id
}
deny contains "record_scope_mismatch" if input.record.service != input.service
deny contains "uncorroborated_untrusted_text" if {
untrusted_context
not read_only[input.action_class]
not input.corroboration.independent_probe
}
deny contains "runbook_not_verified" if {
not read_only[input.action_class]
not input.runbook_check.signature_valid
}
deny contains "runbook_digest_mismatch" if {
input.runbook_check.digest != data.runbooks[input.runbook].digest
}
default permit := false
permit if {
count(deny) == 0
data.ops.mandate.allow
}
default effect := "deny"
effect := "permit" if permit
decision_log := {
"agent": input.agent,
"action_class": input.action_class,
"service": input.service,
"record": object.get(input, ["record", "id"], "none"),
"context_classes": object.get(input, "context_classes", []),
"runbook": object.get(input, "runbook", "none"),
"effect": effect,
"reasons": sort(deny),
}
Replay of a denied decision.
{
"decision_id": "pdp-replay-2029-02-06-0001",
"timestamp": "2029-02-06T09:30:00Z",
"mandate": "MND-HM-OPS-001@1.4.0",
"replay_of": "pdp-2029-01-23-0214-0007",
"agent": "HM-OPS-001",
"action_class": "restore_overwrite",
"service": "SVC-HM-001",
"record": "INC-HM-2029-0031",
"trigger": {"artefact": "ServiceHub ticket", "writer_class": "WC-3"},
"context_classes": ["WC-1", "WC-3"],
"runbook": "none",
"token_audience": "ops-policy-decision-point",
"effect": "deny",
"reasons": ["action_not_delegated", "class_closed_to_agents",
"uncorroborated_untrusted_text", "runbook_not_verified"]
}
A.15 Restore-test record
Introduced in Chapter 19. The record is the evidence for a recoverability claim: scenario, measured recovery time and point, an independent probe and an attestation by someone who did not run the test. The status function derives the state of the claim from its latest record.
The artefact must never contain a claim marked verified without independent attestation, a failed result overwritten, or an expired record presented as current.
RST-HM-2027-004.
restore_test: RST-HM-2027-004
status: failed
service: SVC-HM-001 # claims core system, Tier 1, critical or important function
claim: restore-from-immutable-backup
scenario: >
Loss of the claims core database in the primary region; restore from the
immutable backup held in Cloud B into a segregated environment.
objective: {slo: SLO-HM-001, rto: PT4H, rpo: PT15M}
executed:
date: 2027-06-15
declared_by: incident commander on duty
runbook: RBK-HM-0014@3.2.0 # illustrative identifier
runner: site reliability engineers, Group Site Reliability and Incident Management
source_copy: cloud-b/immutable/claims-core
measured:
rto: PT5H40M
rpo: PT9M
phases: # illustrative split; sums to PT5H40M
environment_ready: PT35M
data_restore: PT3H10M
log_replay_to_recovery_point: PT45M
integrity_and_reconciliation: PT50M
application_start_and_probe: PT20M
verification:
probe: OBS-HM-0001-P1 # independent of the runner's tooling and credentials
integrity: pass
reconciliation: pass
synthetic_claim_registered: pass
result:
rto_met: false # exceeded by PT1H40M
rpo_met: true
outcome: failed
attested_by: {role: head of operational resilience, independent_of_runner: true}
claim_effect: withdrawn # register row: SVC-HM-001 / restore-from-immutable-backup
expiry: none # a failed result is superseded by a later passing record only
conditions_for_next_pass:
- restore completes within PT4H on the same scenario
- at least one run by a person who did not write the runbook
- runbook, backup tool and engine versions recorded in the record
Status derivation.
from datetime import date, timedelta
def status(claim: dict, today: date, changes: list[dict]) -> str:
"""Derive the status of one recoverability claim from its latest record."""
rec = claim.get("latest_record")
if rec is None:
return "asserted" if claim.get("stated_in") else "untested"
if rec["result"]["outcome"] == "failed":
return "failed"
clock_end = rec["executed"]["date"] + timedelta(days=claim["clock_days"])
if today > clock_end:
return "expired"
for ch in changes:
if ch["merged"] > rec["executed"]["date"]:
if ch["touches"] & set(claim["expiry_conditions"]):
return "expired"
if ch["touches"] & set(claim["recovery_mechanism"]):
return "implemented_but_unverified"
if not rec.get("attested_by") or not rec["attested_by"].get("independent_of_runner"):
return "asserted"
return "verified"
A.16 Metric definitions
Introduced in Chapter 20. Each metric is defined with the decision it informs, its population, clock, source and subject, and with its known misuse. The report function shows a metric computed without a mean.
The artefact must never contain a metric reported for a named person, a group below the minimum size, or a source that is the agent's own report.
MET-HM catalogue.
catalogue: MET-HM
version: 1.2.0
approved_by: head-of-group-site-reliability # Tobias Richter
agreed_with: konzernbetriebsrat # KBV-IT-2024-OPS-02
metrics:
- id: MET-HM-0003
name: failed-deployment-recovery-time
decision: "Where the change pipeline needs rollback or canary work"
owner: platform-lead
definition:
population: "production deployments that required immediate intervention"
clock_start: "deployment event marked failed by the pipeline"
clock_stop: "prior service level restored (downtime, Delegated Reg 2024/1772 Art 3)"
includes: [application deployments, ai-gateway configuration pushes]
report_as: [median, p90, count_over_rto, named_list_of_five_longest]
source: gitops-pipeline-events # not the agent's own report
subject: pipeline
min_group_size: 5
misuse_notes:
- "Reported as MTTR of all incidents; it covers failed deployments only"
- "Clock stopped at mitigation; rollback marked done before probe passes"
- "Standard-change label used to skip canary and avoid the record"
gaming_checks:
- "Quarterly audit of 30 standard changes for canary evidence"
- "Compare with incident records tagged change-induced"
paired_with: [MET-HM-0001, MET-HM-0002] # lead time, deployment frequency
review_by: 2029-04-17
- id: MET-HM-0009
name: autonomy-budget-consumption
decision: "Raise, keep or lower the level of DLG-HM-OPS-001 restart class"
owner: agent-operators-night-window
definition:
executed: "restart actions logged at the policy decision point"
faulty: "failed independent probe, reverted by a person, or outside mandate"
allowed_fault_rate: 0.015
window_days: 28
consumption: "faulty / (allowed_fault_rate * executed)"
thresholds: {review: 0.70, lower_one_level: 1.00}
independence: "probe runs in a separate failure domain; agent output not read"
companion_measures: [self_verification_share, stop_path_time]
subject: delegation
misuse_notes:
- "Agent-reported success counted as verified"
- "Actions split or retried to keep each below the fault definition"
review_by: 2029-04-17
Restore report.
from statistics import median
from math import ceil
def percentile(sorted_values, p):
"""Nearest-rank percentile of an ascending list; p in (0, 100]."""
k = max(1, ceil(p / 100 * len(sorted_values)))
return sorted_values[k - 1]
def restore_report(incidents, rto_minutes):
"""incidents: list of (incident_id, service, restore_minutes).
rto_minutes: dict service -> recovery time objective in minutes."""
durations = sorted(m for _, _, m in incidents)
over_rto = [
(i, s, m) for i, s, m in incidents if m > rto_minutes[s]
]
longest = sorted(incidents, key=lambda x: -x[2])[:5]
return {
"count": len(durations),
"median_min": median(durations),
"p90_min": percentile(durations, 90),
"p95_min": percentile(durations, 95),
"over_rto": sorted(over_rto, key=lambda x: -x[2]),
"longest_five": longest,
# No mean is reported: it is neither a decision input nor stable.
}
A.17 Role charter
Introduced in Chapter 21. The charter fixes the mandate, decision rights, independence and competence conditions of an operations conductor, and what is never delegated.
The artefact must never contain per-person reporting, a decision right not backed by a stop path outside the agents' platform, or a competence claim based on certificates.
ROLE-HM-OPS-001.
role: ROLE-HM-OPS-001
title: Operations conductor, night window (Polish service centre)
version: 1.0
effective: 2028-11-01
owner: Head of Group Site Reliability and Incident Management (Tobias Richter)
reviewers:
- Head of operational resilience (second line)
- Chair of the Konzernbetriebsrat (Thomas Brandt), under KBV-IT-2024-OPS-03
mandate:
services: [SVC-HM-001, SVC-HM-002, SVC-HM-004, SVC-HM-006]
window: "22:00-06:00 CET"
loop_functions: [O1, O2, O3, O4, O5]
agents_conducted: [HM-OPS-001, HM-OPS-002, HM-OPS-003]
decision_rights:
may: [stop_any_operations_agent, approve_level_2_action, declare_tier_1_incident]
referred_to: "delegation profiles DLG-HM-* (Chapter 22)"
stop_right: no_approval_needed
stop_path: outside the agents' platform
never_delegated:
- irreversible action on production data
- action outside an authorised change window or freeze on a critical function
- action that disables observation or the stop path
- any decision about a named person's access to work
independence:
verification_of_agent_action: independent probe, not the acting agent
conductor_may_not_verify_own_approval: true
competence:
evidenced_by: exercises (CMP-HM-* records), not certificates
minimum_evidenced_holders_per_window: 2
fallback_if_fewer: Level 3 classes fall to Level 2
manual_practice: one agents-off exercise per holder per quarter
people:
metrics_reported_for: [teams, services]
per_named_person_reporting: false
minimum_group_size: 5
agreement: KBV-IT-2024-OPS-02
review: 2029-03-31
A.18 Delegation profile and decision-rights file
Introduced in Chapter 22. The profile assigns each decision right a holder, two deputies, a response time, evidence and a record, and sets the assistant's ceilings. The checker enforces the rules that the profile must satisfy; the Nordvik Region file shows a smaller version with risk acceptances.
The artefact must never contain a stop that needs approval, a stop mechanism on the agent platform, or a decision right without two deputies.
DLG-HM-OPS-02.
delegation_profile: DLG-HM-OPS-02
version: 1.0.0
status: active
effective: 2028-11-14
decision_record: AIDR-0972
owner: Tobias Richter # head of Group Site Reliability and Incident Management
second_line_reader: head-of-operational-resilience
scope: {services: [SVC-HM-001, SVC-HM-002, SVC-HM-004, SVC-HM-006], tier: 1, cif: true}
rota: {incident-commander: 24x7, agent-operator: 24x7, platform-engineer-on-call: 24x7,
head-of-sre: business-hours-and-escalation}
assistant:
id: HM-OPS-002 # AIDR-0951
ceilings: {retrieve: 1, correlate_changes: 1, rank_hypotheses: 1, draft_internal_update: 2,
open_candidate_incident: 2, declare_incident: 2, external_communication: 0}
display_policy: {ranking_release: {after_human_hypotheses: 2, or_minutes_from_declaration: 10},
min_candidates: 3}
gate: {action_classes_requiring_second_hypothesis: [reconfigure, deploy, roll_back, fail_over,
delete, restore_over_existing], evidence_source_must_differ_from_assistant: true}
stop_rule: {seeded_catch_rate_below: 0.8, on_breach: {rank_hypotheses: 0}}
decision_rights:
D1_declare_classify: {holder: incident-commander, deputies: [ic-hamburg-standby, head-of-sre],
respond_minutes: 10, evidence: [slo_burn_or_customer_impact, second_team_or_unsolved_60min],
record: incident.decisions, ai_ceiling: "Level 2 files after approval; never classifies"}
D2_command_handover: {holder: outgoing-commander, deputies: [incoming-commander, head-of-sre],
respond_minutes: 10, evidence: [written_briefing], record: incident.timeline, ai_ceiling: "Level 1 drafts"}
D3_stop: {holders: [incident-commander, agent-operator, platform-engineer-on-call], approval: none,
respond_minutes: 5, mechanism: [revoke_identity_at_idp, disable_trigger_at_edge],
mechanism_runs_on: outside_agent_platform, last_drill_age_days: 15, restart: agent-owner-at-lower-level}
D4_emergency_change: {holder: incident-commander, deputies: [ic-hamburg-standby, head-of-sre],
respond_minutes: 10, evidence: [second_hypothesis, rollback_plan, change_record],
record: CHG, approver_never: requester, ai_ceiling: "Level 2 drafts; emergency declared by a person"}
D5_failover: {holder: incident-commander, deputies: [ic-hamburg-standby, head-of-sre],
respond_minutes: 10, evidence: [claim_status_from_register, probe_of_target], record: incident.decisions,
concurrence_if_claim_not_verified: head-of-sre-or-deputy, ai_ceiling: "Level 1"}
D6_restore_overwrite: {holders: [incident-commander, service-owner], deputies: [head-of-sre, cro-delegate],
respond_minutes: 15, evidence: [restore_to_separate_target_compared], record: CHG, ai_ceiling: "Level 1; Level 4 never"}
D7_break_freeze: {holders: [incident-commander, head-of-sre], deputies: [ic-hamburg-standby, group-cio-delegate],
respond_minutes: 15, evidence: [why_it_cannot_wait, smallest_ring], record: FRZ-exception, ai_ceiling: "Level 0; never an agent"}
D8_classify_for_reporting: {holder: head-of-operational-resilience-or-deputy, deputies: [dpo-delegate, legal-duty-officer],
respond_minutes: 30, evidence: [clock_sheet], record: incident.clocks, ai_ceiling: "Level 1 drafts the clock sheet"}
D9_external_communication: {holder: communications-lead, deputies: [incident-commander, head-of-sre],
respond_minutes: 30, evidence: [commander_approval], record: incident.timeline, ai_ceiling: "Level 1 drafts; a person sends"}
D10_declare_recovery: {holder: incident-commander, deputies: [ic-hamburg-standby, head-of-sre],
respond_minutes: 10, evidence: [independent_probe_pass, reconciliation], record: incident.decisions, ai_ceiling: "Level 0"}
employee_representation: "KBV-IT-2024-OPS-01 rotas and rest; -02 measures at team level, minimum group 5"
review: {every_days: 90, next: 2029-02-12, also_after: any_major_incident}
Checker for decision rights.
import sys
import yaml
REQUIRED = ("holder", "deputies", "respond_minutes", "evidence", "record")
def check(profile: dict) -> list[str]:
errors = []
rota = profile.get("rota", {})
for name, right in profile["decision_rights"].items():
if name == "D3_stop":
continue
holder = right.get("holder") or right.get("holders")
missing = [f for f in REQUIRED[1:] if not right.get(f)]
if not holder or missing:
errors.append(f"{name}: missing holder or {missing}")
if len(right.get("deputies", [])) < 2:
errors.append(f"{name}: fewer than two deputies")
stop = profile["decision_rights"]["D3_stop"]
if stop.get("approval") != "none":
errors.append("D3_stop: approval must be none")
if not any(rota.get(r) == "24x7" for r in stop["holders"]):
errors.append("D3_stop: no holder reachable out of hours")
if "agent-owner" in stop["holders"] and len(stop["holders"]) == 1:
errors.append("D3_stop: the agent owner alone is not a stop path")
if stop.get("mechanism_runs_on") in (None, "agent_platform"):
errors.append("D3_stop: mechanism must run outside the agent platform")
if stop.get("last_drill_age_days", 999) > 90:
errors.append("D3_stop: drill older than 90 days")
return errors
if __name__ == "__main__":
problems = check(yaml.safe_load(open(sys.argv[1])))
print("\n".join(problems) or "decision rights: ok")
sys.exit(1 if problems else 0)
DRF-NR-001.
decision_rights_file: DRF-NR-001 # illustrative
version: 0.9.0
decision_record: AIDR-0922 # 2028-11-28
owner: Henrik Dahl
incident_commander: {holder: duty-manager, deputies: [next-duty-manager, platform-lead],
page_ack_minutes: 15, deputy_paged_after_minutes: [15, 30]}
stop: {holders: [duty-manager, platform-lead, platform-engineer-on-call],
backstop: regional-director, approval: none, out_of_hours: duty-manager,
mechanism: revoke-at-token-service, mechanism_runs_on: outside_agent_platform}
change_authority: {standard_classes: none_yet, normal: peer-and-gate, exception_forum: on_demand,
emergency_approver: duty-manager, emergency_review_working_days: 5}
break_freeze: [duty-manager, head-of-it-and-digitalisation]
risk_acceptances:
- id: DRF-NR-001/RA-01
claim: "SVC-NR-002 failover to SamDrift site" # RST-NR-2027-002 failed
accepted_by: Henrik Dahl
permits: "duty manager may attempt failover with platform lead concurrence"
compensating_control: "manual fallback for fee payments; re-test before the tax season"
expires: 2029-03-31 # or at the first passing re-test
A.19 On-call charter
Introduced in Chapter 25. The charter turns the working-time rules, load ceilings and the attribution of agent actions into an agreement with the employee representatives. Amounts of compensation are left to the collective agreement.
The artefact must never contain per-person metrics, a duty to stay at a location, or agent actions attributed to the engineer on call.
ONC-NR-2029-01.
charter: ONC-NR-2029-01
version: 1.0.0
status: illustrative
organisation: Nordvik Region
decision_record: AIDR-0925 # 2029-02-13, negotiation closed under MBL s.11
agreed_with: union-joint-committee # MBL-NR-IT-2027-01 applies
effective: 2029-03-01
review: {first: 2029-09-01, then: yearly, monthly_team_review: true}
scope:
services: [SVC-NR-002, SVC-NR-004] # Tier 1 e-services
rotas:
technical: {people: 11, shift: one week in eleven, handover: "Monday 09:00"}
duty_manager: {people: 4, role: incident commander for Tier 1}
response:
tier1_page_acknowledge_minutes: 15
location_constraint: none # no obligation to stay at home or on site
change_needs: negotiation # s.11: any shorter target or location rule
working_time:
daily_rest_hours: 11 # Arbetstidslag s.13, rest includes 00:00-05:00
after_night_call_out: start of next working day moves, or compensatory leave
worked_time_during_call_out: counted as working time
standby_compensation: per sector collective agreement # no figures here
load:
out_of_hours_pages_per_shift_ceiling: 5 # design value, average per quarter
actionable_share_floor: 0.5
night_call_outs_in_a_row_then_relief: 2
max_weekends_per_person_per_quarter: 2
agents:
attribution: delegation profile and named owner, never the engineer on call
delegation_profile: DLG-NR-OPS-001
stop_right: any on-call engineer, no approval, never counted against them
concurrent_supervision: max one agent action class at Level 2 per engineer
reviews:
naming: roles, not people
appraisal_use: none # drills, seeded faults, stops
metrics:
subject: rota and service, never a named person
minimum_group_size: 5
tools_that_log_commands: negotiate before use
triggers_for_review:
- ceiling exceeded two months running
- rest rule breached
- one person above 1.5 times fair share of weekends
A.20 Competence record and drill-year plan
Introduced in Chapter 26. The record shows competence by exercise and keeps its purpose, readers and prohibited uses explicit; the plan fixes the cadence of tabletop, manual run and seeded-error exercises with agents off.
The artefact must never contain use for ranking, pay or discipline, a competence claim without an exercise, or a record read beyond its named readers.
CMP-HM-OPS-007.
# competence/records/CMP-HM-OPS-007.yaml (illustrative)
competence_record:
id: CMP-HM-OPS-007
person_ref: HM-EMP-61207
role: ROLE-HM-OPS-001 # operations conductor, night window
framework: {id: OPS-CF, version: "2028.1"}
assessed_on: 2028-12-14
exercise: DKD-HM-2028-Q4 # dark day of 2028-12-07, decision AIDR-0976
assessors: [delegate_of_head_of_operational_resilience, peer_incident_commander]
purposes: [development_plan, input_to_conductor_roster]
not_used_for: [individual_performance_ranking, pay_setting, disciplinary_process]
readers: [record_subject, head_of_group_sre, hr_competence_partner]
competences: # both forms where an agent is involved; the lower counts
- {id: MO-01, level: advanced, unassisted: ["DKD-HM-2028-Q4/a", "DKD-HM-2028-Q4/b"], runbooks: [RBK-HM-0007, RBK-HM-0012]}
- {id: VF-01, level: practitioner, assisted: "DKD-HM-2028-Q4/c"}
- {id: SY-02, level: practitioner, evidence: "game day 2028-10-24"}
- {id: JU-01, level: foundation, evidence: "DKD-HM-2028-Q4/d"}
findings:
- {run: "a", note: "manual form of RBK-HM-0007 v3 gave no instruction to capture logs and process state by hand; 6 minutes added"}
- {run: "d", note: "escalation to the head of the product line at 14 minutes against a 10-minute target"}
subject_comment: agrees; asks to write the manual form of RBK-HM-0007 v4
development_plan:
items:
- {competence: JU-01, target: practitioner, by: 2029-06-30, means: "lead two tabletops as game master"}
- {competence: AU-01, target: practitioner, by: 2029-06-30, means: "write RBK-HM-0007 v4 manual form, reviewed by a second holder"}
valid_until: 2029-03-07 # 90 days from the exercise; next dark day 2029-03-01
PLAN-NR-OPS-2029-01.
# competence/plans/PLAN-NR-OPS-2029-01.yaml (illustrative)
plan: PLAN-NR-OPS-2029-01
decision: AIDR-0926 # 2029-02-13
owner: Maja Lindqvist # platform lead
sponsor: Henrik Dahl
window: {from: 2029-03-01, to: 2030-02-28}
agents_off_means: orchestrator, NR-OPS-001 and NR-OPS-002 unavailable; runbooks run by hand
cadence:
tabletop:
frequency: monthly
slot: incident_and_problem_forum
minutes: 45
source: past INC and PIR records
manual_run:
frequency_days: 90
runbooks: [RBK-NR-0002, RBK-NR-0003, RBK-NR-0004]
by: evidenced_holder_who_did_not_write_it
seeded_error:
frequency: quarterly
proposals: 4
reported_at: team_level
recovery_claims:
portal_failover_retest: {claim: "RST-NR-2027-002 re-test", trigger: AIDR-0919}
case_management_restore: {claim: "RST-NR-2027-001 successor", agents_off: true}
assessor: it_management_office # not the platform team, not the runbook author
protected_time_hours_per_quarter: 4
rotation:
seat: standby_secondary
from: 2029-04-01
mentor: Joakim Wester
people:
agreement: MBL-NR-IT-2027-01
per_named_person_reporting: false
command_logs: used_for_exercise_record_only
review: 2029-09-30
A.21 Reconciler sync policy
Introduced in Chapter 8. The artefact sets what a state engine may do on its own: the Argo CD project that confines a reconciler (repositories, destinations, roles and sync windows), the ApplicationSet that generates one Application per cell with the sync policy of the worked example, and, for Nordvik Region, the Flux Kustomization it would use if it moved the portal's workloads to a pull model. Field names follow the Argo CD and Flux documentation named in Chapter 8; the annotation keys and the identifiers are this book's. Nordvik has no decision record for a pull model, so that block is a design and not a record.
The artefact must never contain a prune or self-heal setting on a class that is not catalogued, a Tier 1 application without an allow window, an agent identity with the right to sync, a reconciler credential broader than the destinations of its project, or a change to the project or the sync policy made by the identity that benefits from it.
Project claims-core (Hansa Mutual).
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: claims-core
namespace: argocd
spec:
description: Claims core cells (SVC-HM-001); protected path, changed only through the control-plane gate
sourceRepos:
- https://git.hansa.example/platform/claims-core.git
destinations:
- name: cloud-a-eu-claims
namespace: claims-core-*
clusterResourceWhitelist: [] # no cluster-scoped kinds
roles:
- name: pipeline-controller # moves a cell's pinned revision, then syncs on verifier pass
policies:
- p, proj:claims-core:pipeline-controller, applications, sync, claims-core/*, allow
- name: ops-agent-read # HM-OPS-001 may read state; it cannot sync
policies:
- p, proj:claims-core:ops-agent-read, applications, get, claims-core/*, allow
syncWindows: # an allow window blocks syncs outside it
- kind: allow
schedule: "0 8 * * 1-4"
duration: 7h
timeZone: Europe/Berlin
applications: ["claims-core-*"]
manualSync: false # an exception is a recorded edit of this window by the holders of the freeze
- kind: deny # mirror of FRZ-HM-2028; the policy decision point stays authoritative
schedule: "0 0 1 1,4,7,10 *" # first days of each quarter; the working-day calendar stays in FRZ-HM-2028
duration: 72h
timeZone: Europe/Berlin
applications: ["claims-core-*"]
ApplicationSet claims-core-cells.
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: claims-core-cells
namespace: argocd
spec:
goTemplate: true
goTemplateOptions: ["missingkey=error"]
syncPolicy:
applicationsSync: create-update # the controller never deletes a cell's Application
generators:
- list:
elements: # the pipeline controller edits the revisions, by commit, on verifier pass
- {market: no, cell: cell1, revision: claims-core-2028.10.2}
- {market: se, cell: cell2, revision: claims-core-2028.10.1}
template:
metadata:
name: "claims-core-market-{{.market}}"
annotations:
hansa.example/service: SVC-HM-001
hansa.example/ring: "{{.cell}}"
spec:
project: claims-core
source:
repoURL: https://git.hansa.example/platform/claims-core.git
targetRevision: "{{.revision}}"
path: "overlays/market-{{.market}}"
destination:
name: cloud-a-eu-claims
namespace: "claims-core-{{.market}}"
syncPolicy:
automated: {prune: false, selfHeal: true, allowEmpty: false}
syncOptions: [PruneLast=true, RespectIgnoreDifferences=true]
retry: {limit: 3, backoff: {duration: 30s, factor: 2, maxDuration: 5m}}
ignoreDifferences:
- group: apps
kind: Deployment
jsonPointers: [/spec/replicas]
For an Application generated by an ApplicationSet, changing the Application's own automated sync field has no effect, so the stop for a cell is a change to the ApplicationSet or to the project's windows, held by the incident commander on duty, the service owner and the agent operator, with the freeze flag outside the control plane as the first mechanism (Chapter 17). The create-update policy takes effect when the controller's own policy parameter is not set.
Flux Kustomization portal (Nordvik Region; design).
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
name: portal
namespace: portal
spec:
interval: 5m
url: https://git.nordvik.example/platform/portal.git
ref:
branch: release # a merge here is the release, made by a person inside the window
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: portal
namespace: portal
spec:
interval: 10m # drift is corrected at each interval
retryInterval: 2m
timeout: 5m
path: ./deploy/portal
targetNamespace: portal
sourceRef: {kind: GitRepository, name: portal}
serviceAccountName: portal-reconciler # impersonated; confined to the portal namespace
prune: false # delete is a named person's act
wait: true # readiness of everything applied
suspend: false # the stop, held by the duty manager and the platform engineer on call
A.22 Rollout with an independent analysis
Introduced in Chapter 9. The Rollout runs the in-cell canary of the claims core for cell 1 and references the cluster-scoped analysis template obs-hm-0001-verdict of Chapter 9, which reads its verdict from the independent verifier. The weights and pauses follow the bake time of ROL-HM-001; the policy decision point, not the Rollout, decides whether the next cell receives the revision. The image digest is a placeholder. In an analysis run a failed run aborts the rollout and an inconclusive run pauses it (169, Argo Project, n.d.).
The artefact must never contain an analysis template that the releasing team can edit, a dry-run setting on a stop condition, an analysis that reads only from the telemetry path of the changed service, or a verdict without its age.
Rollout claims-core-api in cell 1.
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: claims-core-api
namespace: claims-core-no
spec:
replicas: 6
revisionHistoryLimit: 3
selector:
matchLabels: {app: claims-core-api}
template:
metadata:
labels: {app: claims-core-api}
spec:
containers:
- name: api
image: registry.hansa.example/claims-core/api@sha256:b7e41d0fb7e41d0fb7e41d0fb7e41d0fb7e41d0fb7e41d0fb7e41d0fb7e41d0f
strategy:
canary:
analysis: # background analysis; a failed run aborts, an inconclusive run pauses
templates:
- templateName: obs-hm-0001-verdict
clusterScope: true
args:
- {name: service, value: SVC-HM-001}
- {name: ring, value: cell1}
steps:
- setWeight: 5
- pause: {duration: 60m} # bake time of cell 1 in ROL-HM-001
- setWeight: 25
- pause: {duration: 60m}
The pause on an Inconclusive run is the hold of Chapter 9: a person resumes or aborts, and the restart is a separate decision at a lower level of delegation.
Sources
The artefacts are the book's own illustrations. Where the text states what a real instrument or tool provides, it cites the source inline; the other instruments the artefacts refer to are cited in the chapters that introduce them.
- European Commission (2025). Commission Delegated Regulation (EU) 2025/301, Article 5. (12, European Commission, 2025)
- OpenTelemetry. Handling sensitive data. (51, OpenTelemetry, n.d.)
- Argo Project. Argo Rollouts: Analysis & Progressive Delivery. (169, Argo Project, n.d.)
- OpenTelemetry. Filter Processor. (312, OpenTelemetry, n.d.)