Competences and Keeping the Operator's Craft

An operations loop that agents run for most of the working day still depends on people for the hours when it fails. This chapter asks what those people need to know, how the knowledge decays beside reliable automation, how it is kept, where the next operators come from when entry-level tasks are automated, and how competence is evidenced, so that a claim such as "our operators can recover this service" has a test and a date behind it. It concerns all five operations functions by name: O1 Observe, where reading raw telemetry is a skill that summaries erode; O2 Diagnose, where forming and testing a hypothesis is the skill least easily regained once lost; O3 Change, where reviewing an agent's proposal is the new craft; O4 Restore, where manual operation is the last line when the agents or their platform are unavailable; and O5 Sustain, where the craft is kept fit by record, drill and rota. The chapter ends in a debate on whether automation will deskill operators, with a verdict.

The chapter builds on conclusions that the earlier volumes reached for other roles. Architecture as Code (Chapter 18) managed competence as a skills matrix by level (295, Nordqvist, 2025). AI Governance, Chapter 14, assessed competence by practical demonstration and agreed its use with employee representatives (296, Nordqvist, 2026). AI Threat, Chapters 6 and 14, collected the evidence that reliance on AI reduces recall and checking and that automating junior work removes the route by which experts are made (299, Nordqvist, 2026). The Transformed Security Architect, Chapters 15, 21 and 22, designed the keeping of a craft (300, Nordqvist, 2026). AI Blue, Chapters 17, 22 and 23, set the rule that with fewer than two evidenced holders a Level 3 class falls back to Level 2 and treated the apprenticeship problem for analysts (302, Nordqvist, 2026), and AI Purple, Chapter 24, did the same for the red craft (304, Nordqvist, 2026). This chapter applies their instruments to operations.

Career route, drills with agents off, an independent assessor, the competence record, and two evidenced holders or a fall to Level 2

Figure 26.1: Competence, drill and evidence: an independent assessor outside the people and agents assessed writes the competence record, and expired evidence lowers the level.

What an operator needs now

The competences below are the book's own analysis, written as capabilities that can be observed in an exercise and not as traits, in five areas.

Systems thinking is the ability to read an estate as a set of dependencies with blast radii: which services share an identity provider, a DNS service, a recovery site or a model family, and what a failure of one does to the rest. Chapter 3 gave the map, Chapter 17 the observer's failure domain and Chapter 24 the provider seams. The skill is reading a failure across them.

Verification is the ability to check an agent's proposal against the record and against a probe that the agent does not control. A proposal is accepted for a reason that can be written down, and the reason is not that it is fluent (Chapters 4 and 17).

Manual operation is the ability to carry out a catalogued runbook, a restore or a failover by hand, in its manual form, with the agent and its platform unavailable. It has a motor part (the commands) and a cognitive part (knowing which step comes next, recognising that a step failed and why). The evidence below suggests the second part decays first.

Authorship is the ability to write what the loop runs: a runbook manifest with verification and a stop, an agent skill, a mandate manifest, and the test that accompanies each. An operator who cannot write a runbook cannot judge an agent's draft of one (Chapter 16).

Judgement under uncertainty is the ability to declare an incident, escalate, stop an agent or refuse a risk when the evidence is incomplete, and to record the reasons.

A competence framework for operations

A framework lists what the people of a function must be able to do, at which level and with what evidence. Hansa Mutual's group framework OPS-CF, version 2028.1, has the five areas above and thirteen identifiers, at the four levels that Architecture as Code (Chapter 18) and AI Governance, Chapter 14, used: Foundation, Practitioner, Advanced and Expert.

Area Identifiers Evidenced by Functions
SY Systems SY-01 read a service map for dependencies and blast radius; SY-02 read a failure across providers and seams Game day, tabletop O1 Observe, O2 Diagnose
VF Verification VF-01 verify an agent's proposal against the record and an independent probe; VF-02 design a seeded error and the probe that should catch it Seeded-error drill O2 Diagnose, O3 Change
MO Manual operation MO-01 run a catalogued runbook by hand; MO-02 restore or fail over by hand to a measured time; MO-03 recover with the agent platform unavailable Dark day, game day with agents off O4 Restore
AU Authorship AU-01 write a runbook manifest with verification and stop; AU-02 write and review an agent skill or mandate manifest; AU-03 write the test for a runbook Written artefact reviewed by a second person O3 Change, O4 Restore, O5 Sustain
JU Judgement JU-01 declare, escalate or stop under uncertainty; JU-02 accept or refuse a risk with recorded reasons; JU-03 give a blameless account of a failure to a service owner Exercise and review record O2 Diagnose, O5 Sustain

Two rules from AI Blue, Chapter 22, carry over. A competence shown with and without the assistant counts at the lower level, because a skill that exists only beside the assistant is a skill of the pair; and a competence that exists only beside an assistant, such as VF-01, is assessed in the assisted form alone (302, Nordqvist, 2026). MO-01 to MO-03 are assessed unassisted by definition. Nordvik Region has seven platform engineers, no Mode 5 agent and no night team of its own, so a framework of thirteen identifiers would describe an organisation it is not. Its subset OPS-CF-NR, version 2028.1, has eight: SY-01, VF-01, MO-01, MO-02, AU-01, AU-03, JU-01 and JU-02. It leaves SY-02, MO-03 and the agent-facing parts of AU-02 to the review that follows the first scripted runbook that is raised above Level 2.

What the evidence says about skill beside automation

The evidence has a long theory and a short, recent set of measurements. None of it measures operators who supervise remediation agents, and the chapter says so before it draws anything from it.

The theory is old. Bainbridge's analysis of the ironies of automation, written about industrial process control, argues that automation leaves the operator to monitor the automatic system and to take over when it fails, that taking over requires manual control skills and cognitive skills, and that skills deteriorate when they go unused (217, Bainbridge, 1983). Parasuraman and Manzey's review of the empirical literature reports that automation bias occurs in both naive and expert participants, that it affects individuals and teams, and that it is not prevented by training or instruction alone (218, Parasuraman & Manzey, 2010).

The most relevant measurement for operations comes from aviation. Casner and colleagues asked 16 airline pilots to fly routine and non-routine scenarios in a Boeing 747-400 simulator while the level of automation was varied. They found instrument scanning and manual control skills mostly intact, even when pilots reported that they rarely practised them. When pilots were asked to perform the cognitive tasks of manual flight, such as tracking the aircraft's position without a map display, deciding which navigational step comes next and recognising instrument system failures, the authors observed more frequent and significant problems. Their conclusion is that retention of the cognitive skills may depend on how actively pilots remain engaged in supervising the automation, and they note that many airlines permit pilots to turn automation off occasionally to practise (233, Casner et al., 2014). The study is small and concerns pilots; the reading for operations is the book's own: the part of manual recovery most at risk is the diagnostic and sequencing judgement and not the keystrokes, and engaged supervision protects it.

Three recent studies measured the effect of AI assistance on later unaided performance in other domains. At four Polish centres, the adenoma detection rate of experienced endoscopists in colonoscopies performed without AI fell from 28.4 per cent (226 of 795) before regular AI assistance was introduced to 22.4 per cent (145 of 648) afterwards, an absolute fall of 6.0 percentage points. The design is observational, and an independent commentator noted that the number of procedures rose sharply after the tool was introduced (234, Budzyń et al., 2025). In a field experiment with nearly a thousand high school mathematics students, access to a GPT-4 interface in the standard form raised grades on practice problems by 48 per cent; when access was withdrawn, those students performed worse than students who had never had it, with a 17 per cent reduction in grades. A tutor with safeguards for learning raised practice grades by 127 per cent and largely avoided the later drop (235, Bastani et al., 2025). In a preprint of a study of developers learning a new programming library, AI use weakened conceptual understanding, code reading and debugging with no significant efficiency gain on average, and three of six observed interaction patterns, those with cognitive engagement, preserved learning (236, Shen & Tamkin, 2026).

One workplace study points the other way for newer staff. In a study of 5,179 customer-support agents, access to a generative AI assistant raised productivity, measured as issues resolved per hour, by 14 per cent on average and 34 per cent for novice and low-skilled workers, with little effect for experienced workers, and the authors offer suggestive evidence that newer workers gained experience faster (237, Brynjolfsson et al., 2025).

Read together, these results do not say that AI makes people worse at their work. They say that the effect depends on use: use that replaces the attempt is associated with weaker unaided performance, use that keeps the attempt is not, and novices can gain. That reading is the book's interpretation. The labels follow.

  • Observable. Unaided performance can fall after sustained use of an automated aid in some tasks (cognitive tasks in the cockpit, colonoscopy, school mathematics, learning a programming library), and the effect depends on the pattern of use. Novices gain more than experts from an assistant in at least one large workplace study.
  • Expected. That the mechanism applies to the cognitive part of manual recovery in operations as agents take the routine remediations, because the mechanism is the same and Chapters 10 and 16 record the concern. It has not been measured in operators.
  • Speculation. That operators lose the ability to recover services across the profession, or that organisations stop recruiting entry-level operators altogether.

Keeping the craft

A craft is kept by use, and where daily work no longer supplies the use it must be supplied on purpose. The instruments below are used together, and each produces evidence that goes into a competence record.

The manual-practice clause

Chapter 16 put a clause in the mandate of every agent that runs runbooks: at stated intervals, a runbook that the agent normally runs is run by a person, with the agent off, so that the skill stays in the team. The default interval is 90 days, the maximum age of the evidence on which an agent is promoted, so that the evidence for the agent and for the people who can do without it expire together. The mandate of HM-OPS-001 names RBK-HM-0007 (restart of one stateless instance) and RBK-HM-0012 (certificate renewal) with agent_off: true, and Chapter 10 names the measure the clause produces, the age of the last manual recovery. This chapter completes the clause in three respects.

  1. The run is by an evidenced holder, not by the runbook's author. An author knows what the manual form leaves out, and the exercise is meant to discover that. The run is done from the manual form as written.
  2. The run produces a record, not only a result. It states who ran it, the date, the elapsed time from page to verified result, every step that was missing, wrong or ambiguous, and the change that follows. A manual form never run is a document and not a procedure.
  3. A lapse has a consequence. When the last run is older than 90 days the holder's MO evidence expires, and when a window has fewer than two evidenced holders of the conductor role every Level 3 class in it falls to Level 2 (Chapter 21, following AI Blue, Chapter 22). The consequence falls on the delegation and not on the person: nobody is disciplined for a lapsed drill, and the agent's ceiling is lowered until the drill is done.

Dark days and game days with the agents off

A dark day is a scheduled period in which a defined workload is operated without the operations agents, so that the people do what the agents normally do. It differs from a game day (Chapter 19), which injects a fault and measures recovery against an objective; a dark day measures nothing against an objective and exercises the ordinary work, the reading of raw alerts, the manual runs, the escalation. A game day with the agents off combines the two for a recovery claim: the claim "the service can be restored when the remediation agent is unavailable" is about people and runbooks and needs a test of that form (Chapter 19).

The safety rules follow those of Chapter 19 and the Level 4 prohibitions of Chapter 4. A dark day is scoped in writing, names the services and the hours, runs outside any freeze on a critical function and keeps the observer and the stop path in service: only the actors are switched off, never the observation. An abort rule returns the agents to service when a service outside the scope degrades or a Tier 1 incident is declared there. The stop path that turns the agents off is the mandate's stop path, held outside the agent platform, so that a dark day also drills the stop. It ends in an improvement: every missing step becomes a change to a runbook.

Two older practices belong in the same family. Google's account of training site reliability engineers describes the Wheel of Misfortune, a role-playing exercise in which a game master picks a primary and a secondary on-call engineer and describes a scenario from a past outage or a hypothetical failure, and the on-call team says what it would do to mitigate and investigate; the chapter argues that hands-on work is better than reading and recommends shadowing (232, Widdowson, 2017). Google's account of service level objectives describes a deliberate planned outage of an internal lock service, taken when real failures had not brought its availability below target, to flush out dependencies that had come to assume it would never fail (241, Jones et al., 2017). That exercise exposed dependencies and did not train people, but the principle is the same: a dependence on automation that is never interrupted is not seen until it fails. A Wheel of Misfortune from the team's own incident records costs an hour and needs no injected fault.

Seeded-error drills

A manual run keeps the ability to act. A seeded-error drill keeps the ability to check. Proposals from an agent, or from a person playing one, enter the review queue with a known flaw: a restart that would leave one instance of five, a certificate renewal for the wrong service, a rollback to a version that predates a schema change. The drill measures whether the reviewer catches the flaw and gives a reason. The measure is reported at team level, with a minimum group size, never per person, as AI Threat concluded for oversight quality (299, Nordqvist, 2026). The catch rate on a small team is noisy; the drill's main value is the reasons it surfaces. A reviewer who accepts a flawed proposal because it was fluent has shown the automation bias that Article 14(4)(b) of the AI Act asks overseers of high-risk AI systems to remain aware of (1, European Union, 2024).

Practical guide: planning a drill year with agents off

Purpose. To give a team a year of drills that keeps manual operation, verification and judgement evidenced at a cost a small team can carry.

Roles. The head of operations or SRE lead (owner), the platform lead, an incident commander, an independent assessor, the service owners, the employee representatives and the operators.

Steps.

  1. List the runbooks that agents or the orchestrator run and the recovery claims in the resilience register (Chapter 19).
  2. Fix the cadence. As a default, one manual run of each runbook at Level 2 or above every 90 days by an evidenced holder who did not write it; one tabletop a month from a past incident; one dark day a quarter; one game day with the agents off a year for each Tier 1 recovery claim, placed in the quarter before the claim's evidence expires.
  3. Fix the scope of each dark day in writing: services, hours, abort rule, stop path and its holder. Exclude services under a freeze.
  4. Name an assessor who is neither the operator's line manager nor the runbook's author, normally the second line or a peer team.
  5. Prepare four to six flawed proposals a quarter, including one that a fluent summary would hide.
  6. Protect the time: by default four hours per operator per quarter outside the rota, with rest after a night exercise under the works agreement.
  7. Agree the records with the employee representatives before the first exercise: what is logged, who reads it and what it is never used for.
  8. After each exercise, file every missing step as a change to the runbook, and re-run the manual form within 30 days if it changed materially.
  9. Report each quarter at team level: age of the last manual run per runbook, evidenced holders per window, seeded-error catch rate with its group size and changes caused.

Definition of done. Every runbook above Level 1 has a manual run dated within 90 days, every window has two evidenced holders or a recorded fall to Level 2, and the next year's plan is approved. Adapt the intervals to the service's criticality, the size of the team and the works agreement.

Worked example: a drill-year plan at Nordvik Region

Illustrative example. Nordvik Region has no Mode 5 agent, so "agents off" means that the orchestrator, the operations assistant NR-OPS-001 and the scribe NR-OPS-002 are unavailable and the runbooks are run by hand. The plan was approved by the Head of IT and Digitalisation, Henrik Dahl, as AIDR-0926 on 2029-02-13, and it uses the monthly incident and problem forum of Chapter 12 for its tabletops so that it adds no meeting.

# competence/plans/PLAN-NR-OPS-2029-01.yaml (illustrative)
plan: PLAN-NR-OPS-2029-01
decision: AIDR-0926               # 2029-02-13
owner: Maja Lindqvist             # platform lead
sponsor: Henrik Dahl
window: {from: 2029-03-01, to: 2030-02-28}
agents_off_means: orchestrator, NR-OPS-001 and NR-OPS-002 unavailable; runbooks run by hand
cadence:
  tabletop:
    frequency: monthly
    slot: incident_and_problem_forum
    minutes: 45
    source: past INC and PIR records
  manual_run:
    frequency_days: 90
    runbooks: [RBK-NR-0002, RBK-NR-0003, RBK-NR-0004]
    by: evidenced_holder_who_did_not_write_it
  seeded_error:
    frequency: quarterly
    proposals: 4
    reported_at: team_level
  recovery_claims:
    portal_failover_retest: {claim: "RST-NR-2027-002 re-test", trigger: AIDR-0919}
    case_management_restore: {claim: "RST-NR-2027-001 successor", agents_off: true}
assessor: it_management_office   # not the platform team, not the runbook author
protected_time_hours_per_quarter: 4
rotation:
  seat: standby_secondary
  from: 2029-04-01
  mentor: Joakim Wester
people:
  agreement: MBL-NR-IT-2027-01
  per_named_person_reporting: false
  command_logs: used_for_exercise_record_only
review: 2029-09-30

The plan puts Joakim Wester's experience of recovery to work as mentoring and not as a single point of failure, and ties the portal failover re-test that the Region owes under Chapter 19 to a rotation seat, so that the re-test also trains a second person. Its weakness is that the assessor, the IT management office, is independent of the platform team but is not an operations assessor, so the first year's results are treated as a baseline and not as evidence for a claim of competence.

The early-career route

The tier-1 pyramid of security operations was a response to alert volume and a school for the craft (AI Blue, Chapters 17 and 23, F108 and F109) (302, Nordqvist, 2026). Operations had the same pyramid under other names: the service desk, the first-line on-call and the shift that restarted what had stopped. That routine was also where new operators learned what an ordinary day in an estate looks like. The apprenticeship problem has two halves: where the next operators come from if routine closure moves to agents, and how present experts keep a skill their own automation no longer exercises. The previous section treated the second; this one treats the first.

The table gives the book's own analysis of what junior operations work taught, so that a team can ask for each element where it is now learned. It is not drawn from a source that documents how operators entered the craft, and it is offered as a checklist.

What junior work taught Where AI now does the work Where it can be learned instead
What an ordinary day looks like: the alert volume, the usual noise, the weekly rhythm Correlation and noise reduction (Chapter 6) Reading the raw alert stream for a fixed period before opening the correlator's summary (SY-01), as a standing first-week task
How a service fails in practice, by restarting, scaling and watching the result The runbook runner at Level 3 Manual-form runs in staging and in the manual-practice clause, by the apprentice with the evidenced holder (MO-01)
How to write down what happened, and why it matters The scribe drafts timelines Writing the timeline unaided for a past incident, then comparing it with the scribe's draft (JU-03)
How a change fails, by seeing change records and rollbacks The change reviewer scores proposals Reviewing a seeded flawed change record with a mentor (VF-01, VF-02)
How to read an unfamiliar system under time pressure The incident assistant ranks hypotheses The Wheel of Misfortune, in which the apprentice answers first and the assistant's proposal is opened after the attempt (SY-02)

Two conclusions follow, both the book's own. The routine did teach, and the new work teaches only if designed to: reviewing a runbook that an agent drafted teaches little to someone who has never written one. And the loop creates new first rungs. Because runbooks, tests and manifests are code, a junior can write the test for a runbook (AU-03), run a dry run, review a seeded change record or file the change that follows a dark day, with consequences small enough to learn from. Google's account of on-call training makes a related point: newcomers should shadow on-call, because it is unfair to make them wait for their first real page, and may be listed as coauthors on postmortems (232, Widdowson, 2017).

The customer-support study found that novices gained most from an assistant (237, Brynjolfsson et al., 2025), which is a reason to let apprentices use it, and the study of students suggests the condition: a tutor with safeguards avoided most of the harm (235, Bastani et al., 2025). The practical rule is "attempt first, then ask": for the first six months an apprentice states a diagnosis, a timeline or a runbook step before opening the assistant's proposal, and the sequence is noted in the exercise record.

Rotation in both directions is the cheapest instrument. A platform engineer who spends a quarter on the incident commander rota learns what the on-call needs from a runbook; an incident commander on the platform team learns what the agents can be given. The rotation also builds the second evidenced holder for each window, which the rule of Chapter 21 requires before a Level 3 class can stand. A route with no completions after two years is reviewed.

Frameworks, law and records

Mapping to ECSF, SFIA and e-CF

External frameworks make a role legible to a recruiter, an auditor and a supervisor. The mapping below is the book's own and is to be confirmed by an assessor. SFIA 9 describes skills in seven levels of responsibility, and its skills include incident management (USUP), problem management (PBMG), change control (CHMG), release management (RELM), availability management (AVMT), capacity management (CPMG), service level management (SLMO), continuity management (COPL), infrastructure operations (ITOP) and systems installation and removal (HSIN) (220, SFIA Foundation, 2024). The European e-Competence Framework, EN 16234-1, groups 41 competences in five areas (PLAN, BUILD, RUN, ENABLE, MANAGE) with five proficiency levels (67, European Committee for Standardization (CEN), 2019). ENISA's European Cybersecurity Skills Framework lists 12 typical cybersecurity role profiles (219, ENISA, 2022).

OPS-CF area SFIA 9 skills e-CF area ECSF
SY Systems AVMT, CPMG, SLMO RUN Not covered; operations beyond security
VF Verification CHMG, RELM RUN, BUILD Boundary with security operations (AI Blue)
MO Manual operation ITOP, USUP, COPL RUN Boundary: incident response profiles
AU Authorship RELM, HSIN, ITOP BUILD Not covered
JU Judgement USUP, PBMG RUN, MANAGE Boundary: incident response profiles

SFIA and e-CF describe operations in the vocabulary of service management, and neither has a skill for verifying an agent's proposal or keeping manual operation in practice; those are OPS-CF additions that an organisation records as an extension of the nearest skill. ECSF is a cybersecurity framework and meets operations only where an incident is also a security incident, the boundary that AI Blue treats.

What the law asks

The instruments ask for competence in general terms, and none prescribes manual drills. Article 4 of the AI Act, as replaced by Article 1, point (5), of Regulation (EU) 2026/1744 (the Digital Omnibus on AI), requires providers and deployers to take measures to support the development of AI literacy of their staff and others operating AI systems on their behalf, and states that this does not require them to guarantee a specific level of AI literacy of any individual (11, European Union, 2026). Article 26(2) requires deployers of high-risk AI systems to assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support, and Article 14(4) lists what overseers must be enabled to do, including to remain aware of the tendency to over-rely on the system's output (automation bias), to disregard, override or reverse it, and to interrupt it through a stop button or similar procedure (1, European Union, 2024). Whether an operations agent is itself a high-risk AI system is a classification question for Chapter 23; the duty applies to the people who oversee the services that are, such as SVC-NR-001 at Nordvik Region and SVC-HM-008 at Hansa Mutual.

DORA Article 5(4) requires members of the management body of a financial entity to keep up to date with sufficient knowledge and skills to understand and assess ICT risk, including by following specific training on a regular basis, and Article 13(6) requires ICT security awareness programmes and digital operational resilience training as compulsory modules in staff training schemes, for all employees and senior management (2, European Union, 2022). NIS 2 Article 20(2) requires Member States to make members of the management bodies of essential and important entities follow training, and to encourage similar training for employees (3, European Union, 2022). The implication is limited. The law asks for literacy, for competent and empowered oversight and for tested recovery (Chapter 19); none prescribes manual drills. The manual-practice clause is the book's inference from those demands together: a recovery test that only an agent can run evidences the agent, and an overseer who has never done the work by hand has little basis for the override that Article 14(4) expects. This is an organisational reading, not legal advice.

Records as personal data

A competence record CMP-* is a record of judgement exercised: the levels a person has shown, the evidence, the validity period and the development plan, as in AI Blue, Chapter 22 (302, Nordqvist, 2026). What that chapter concluded about records as personal data carries over: exercise evidence only, no ranking, the subject reads all of it, and its use is agreed with employee representatives. GDPR Article 88 lets Member States provide, by law or by collective agreements, more specific rules for employees' personal data in the employment context (6, European Union, 2016), and in both organisations the rule is a collective agreement. Nordvik Region's MBL-NR-IT-2027-01 reports measures about engineers never by name and makes any tool that logs engineers' commands a matter for negotiation before use; Hansa Mutual's KBV-IT-2024-OPS-02 reports metrics for teams and services with a minimum group size of five, and KBV-IT-2024-OPS-03 agrees such tools before use. Command logs from a dark day are evidence of the exercise and are not used to rank anyone. AI Boss, Chapters 13 and 23, treat co-determination in detail (301, Nordqvist, 2026).

Worked example: a competence record for an operations conductor

Illustrative example. On 2028-12-07, from 22:00 to 02:00 CET, Hansa Mutual held its first quarterly dark day, DKD-HM-2028-Q4, under AIDR-0976 (dated 2028-11-20; owner Tobias Richter). The scope was SVC-HM-005 (Tier 1, not a critical or important function for DORA) and the pre-production mirror of SVC-HM-001; services supporting critical or important functions stayed under normal agent cover, and a Tier 1 incident outside the scope would have returned the agents to service. The stop path STOP-HM-ITOPS switched off the runbook runner HM-OPS-001 and the read-only agents HM-OPS-002 and HM-OPS-005. A week later a delegate of the head of operational resilience and a peer incident commander assessed Ewa Kowalczyk, night incident commander and agent operator for the night window, against OPS-CF.

# competence/records/CMP-HM-OPS-007.yaml (illustrative)
competence_record:
  id: CMP-HM-OPS-007
  person_ref: HM-EMP-61207
  role: ROLE-HM-OPS-001              # operations conductor, night window
  framework: {id: OPS-CF, version: "2028.1"}
  assessed_on: 2028-12-14
  exercise: DKD-HM-2028-Q4           # dark day of 2028-12-07, decision AIDR-0976
  assessors: [delegate_of_head_of_operational_resilience, peer_incident_commander]
  purposes: [development_plan, input_to_conductor_roster]
  not_used_for: [individual_performance_ranking, pay_setting, disciplinary_process]
  readers: [record_subject, head_of_group_sre, hr_competence_partner]
  competences:                       # both forms where an agent is involved; the lower counts
    - {id: MO-01, level: advanced, unassisted: ["DKD-HM-2028-Q4/a", "DKD-HM-2028-Q4/b"], runbooks: [RBK-HM-0007, RBK-HM-0012]}
    - {id: VF-01, level: practitioner, assisted: "DKD-HM-2028-Q4/c"}
    - {id: SY-02, level: practitioner, evidence: "game day 2028-10-24"}
    - {id: JU-01, level: foundation,   evidence: "DKD-HM-2028-Q4/d"}
  findings:
    - {run: "a", note: "manual form of RBK-HM-0007 v3 gave no instruction to capture logs and process state by hand; 6 minutes added"}
    - {run: "d", note: "escalation to the head of the product line at 14 minutes against a 10-minute target"}
  subject_comment: agrees; asks to write the manual form of RBK-HM-0007 v4
  development_plan:
    items:
      - {competence: JU-01, target: practitioner, by: 2029-06-30, means: "lead two tabletops as game master"}
      - {competence: AU-01, target: practitioner, by: 2029-06-30, means: "write RBK-HM-0007 v4 manual form, reviewed by a second holder"}
  valid_until: 2029-03-07            # 90 days from the exercise; next dark day 2029-03-01

The record shows what it was designed to show. The manual run of RBK-HM-0007 took 19 minutes from page to five consecutive probe passes, of which ten were the soak that the manifest requires, and exposed what the agent made invisible: the runbook's precondition evidence.captured was satisfied by the agent, and the manual form said nothing about how a person captures it. The finding became version 4 of the runbook. The escalation target was missed by four minutes, a gap that daytime tabletops had not shown, and it sits on the plan. The seeded-error drill of the same night, in which the night-window group of eight faced six flawed proposals, caught four; the figure is reported at team level, is too small to be a rate and is not on this individual record, which states a level and a reference. The record informs the roster of conductors: with Ewa Kowalczyk and a second evidenced holder, ROLE-HM-OPS-001 meets its rule of two evidenced holders per window until 2029-03-07 and not after, unless the dark day of 2029-03-01 renews the evidence.

The debate

The debate concerns whether automation will deskill operators. It does not ask whether automation is good, which Chapter 10 treated for remediation, but whether the people beside it keep the ability to do what it does.

The case for concern

Those who hold the concern begin with Bainbridge. The more reliable an automated system, the less practised the person who must take over when it fails, and the situations that reach that person are the ones the automation could not handle (217, Bainbridge, 1983). Applied to operations the argument is concrete. A remediation agent that restarts, scales and renews certificates removes the repeated, low-risk pages that used to give an operator daily practice, and leaves the rare, hard and novel ones. The operator who meets them has done less of the ordinary work than any predecessor and meets a failure that automation has already failed to handle. ITBench reports that agents resolved 11.4 per cent of its site reliability engineering scenarios at its ICML 2025 publication (225, Jha et al., 2025), so most of the hard work stays with people.

They add the measurements. In the cockpit study the cognitive part of manual flying, knowing what comes next and recognising a failure, showed the problems and not the hand skills (233, Casner et al., 2014); the colonoscopy, mathematics and programming studies found weaker unaided performance, including weaker debugging (234, Budzyń et al., 2025) (235, Bastani et al., 2025) (236, Shen & Tamkin, 2026). Debugging and recognising a failure are the operator's work. Automation bias is not cured by instruction alone (218, Parasuraman & Manzey, 2010), so a policy statement that operators will keep their skills is not evidence that they have.

The last element is the entry route. If the service desk's routine closes by class and the first-line shift disappears, the next generation of incident commanders has no first rung. The claim is not that people will be replaced but that the supply of experienced operators will thin when the present ones have moved on.

The case for calm

Those who hold the calm view start from the history. Each wave of tooling changed the craft without ending it. Operators no longer rack servers, patch by hand or read printed logs, and the craft moved up to the design of systems that do those things. Site reliability engineering began from the observation that repeated manual work is a cost and not a virtue: Google's goal is to keep toil below half of each engineer's time, and its surveys reported an average of about a third (253, Rau, 2017). Time released from toil is spent on engineering that removes causes, which is the craft growing.

They point out that the strongest measurement says less than the concern suggests. The pilot study found hand skills mostly intact even when rarely practised, and concluded that the cognitive skills depend on how actively pilots stay engaged in supervising (233, Casner et al., 2014). The colonoscopy study is observational, with a different workload in the two periods (234, Budzyń et al., 2025); a tutor with safeguards largely avoided the harm in the mathematics experiment (235, Bastani et al., 2025); and three of six patterns of use preserved learning in the programming study (236, Shen & Tamkin, 2026). The calm reading is that the effect belongs to how the aid is used, not to using it, and that organisations control how it is used.

They add that the entry-route evidence points both ways: novices gained most from the assistant in the customer-support study (237, Brynjolfsson et al., 2025), and an assistant that generates scenarios could serve as a practice partner, which is the book's reasoning and has not been measured in operations. Operations has also practised the deliberate removal of automation for a long time, in the planned outage and the Wheel of Misfortune (241, Jones et al., 2017) (232, Widdowson, 2017). The calm conclusion is that the means exist and the question is whether organisations use them.

Where the positions meet

The two sides accept more than the framing suggests. They agree that skill must be exercised on purpose, because the daily work that agents now take will not keep it. They agree that the cognitive part of recovery, the sequencing and the recognition of a failure, matters more than the typing, and that the effect depends on how the aid is used, so that "attempt first, then ask" is a better design than unrestricted use. They agree that the entry route needs design and that nobody has measured the outcome in operators, and that what cannot be measured by comparison can be measured by exercise.

The measures both accept are the age of the last manual run per runbook, the elapsed time of a manual recovery measured at least yearly for each Tier 1 recovery claim, the evidenced holders per window, the share of exercises that led to a runbook change, the seeded-error catch rate with its group size, and the people on the early-career route with the completions. Each can be reported for a team and none measures a person's worth.

What can be done

The CIO or head of operations can ask for the register of evidenced holders per window and the age of the last manual run per runbook, and set the rule that no class stays at Level 3 in a window without two evidenced holders. The SRE or platform lead can run the drill year of the practical guide, start with one dark day a quarter on a service that is not critical, and treat every missing step in a manual form as a defect to fix within 30 days. The incident commander can hold the stop that switches the agents off for a dark day and use it without approval when a service outside the scope degrades. The head of operational resilience or another independent assessor can assess the exercises, hold the records in a form that the subject reads, and add a game day with the agents off to the testing programme for each Tier 1 recovery claim. The management body can ask how many windows fell to Level 2 last quarter for want of evidenced holders. Employee representatives can agree in advance what exercise records contain and who reads them, and ask that drill time count as working time. Operators can ask to write the manual form of the runbooks they depend on.

Verdict

For O4 Restore, that unaided performance of the cognitive part of recovery can decay beside reliable automation is observable in other domains (the pilot, colonoscopy, mathematics and programming studies), on stated conditions: the studies differ in design, one is observational, one is a preprint, and none concerns operators. That the same decay occurs in operators who no longer practise manual restart, restore and failover is expected: the mechanism is the same, and the rule of two evidenced holders is a design response to it, not a finding. For O2 Diagnose and O3 Change, that reviewers of an agent's proposals show automation bias is observable in the review of the literature, and that it persists after instruction is observable; that seeded-error drills reduce it in operations is the book's inference and has not been measured. For O1 Observe, that reading raw telemetry decays when summaries replace it is expected. For O5 Sustain, that a craft register, protected time and a drill year keep the skills in a team is expected on the strength of the airline practice and the safeguards found in the tutor study, and no operations study has tested it.

That the supply of entry-level operators thins as routine closure moves to agents is expected where organisations let it, and it is within their control. That operators lose the ability to recover services across the profession, or that organisations stop recruiting juniors altogether, is speculation. Organisations can influence the interval of the drills, the second holder, the route, the records and the protection of the people who are measured. They cannot change the effect of unrestricted assistance on skill, which the studies show to vary with use, and the design assumes that unrestricted use is the default to be changed.

Across the usage modes

Usage mode What the operator still does Skill at risk What is drilled
Mode 0 Ambient AI Reads suggested categories and priorities in the ITSM tool Triage judgement; knowing what an ordinary queue looks like Reading the raw queue for a fixed period; sampling classifications against the raw ticket
Mode 1 Conversational assistants Asks for commands, explanations and drafts Command recall; reading a log unaided Attempt first, then ask; a monthly tabletop without the assistant
Mode 2 Embedded workplace assistants Reads incident and telemetry summaries inside the observability tool Reading raw telemetry; checking a summary against the data Reading the raw data before the summary (VF-01, SY-01)
Mode 3 Agentic work tools Reviews drafted runbooks and infrastructure code Authorship; judging a draft one could not have written Writing a runbook unaided; reviewing a seeded flawed draft (AU-01, AU-03)
Mode 4 Built AI applications Uses the incident and operations assistants to rank hypotheses Forming and testing a hypothesis Diagnosing a past incident from the raw record before opening the assistant (SY-02)
Mode 5 Autonomous production agents Conducts the agents, holds the stop, recovers when they cannot Manual operation of restart, restore and failover; recovery without the agent platform The manual-practice clause; dark day; game day with the agents off (MO-01 to MO-03)

Nordvik Region and Hansa Mutual

Hansa Mutual holds a quarterly dark day. The first, DKD-HM-2028-Q4 of 2028-12-07 under AIDR-0976, ran four hours in the night window and produced CMP-HM-OPS-007 and version 4 of RBK-HM-0007. The clause of Chapter 16 (every 90 days, agent off, RBK-HM-0007 and RBK-HM-0012) is the evidence cycle, and ROLE-HM-OPS-001 requires two evidenced holders per window and one agents-off exercise per holder per quarter; a window that fails the rule drops the Level 3 classes of HM-OPS-001 to Level 2. The weak point is the restore: the claims core restore in Cloud B took 5 hours 40 minutes against 4 hours in RST-HM-2027-004, and restore-overwrite is delegated to no agent, so it is a manual skill by design. A game day with the agents off for that claim is the exercise whose evidence the register of Chapter 19 needs, and until it passes the claim is not verified, whatever the dark day shows about restarts. Drills are placed in working time under KBV-IT-2024-OPS-01.

Nordvik Region has no Mode 5 agent and no night team of its own, and its craft is concentrated. Joakim Wester is its most experienced person on recovery, writes the runbooks and takes the most weekend on-call. Its plan, PLAN-NR-OPS-2029-01 (AIDR-0926, 2029-02-13), is therefore an apprenticeship rotation more than a dark-day programme: a standby secondary seat on the out-of-hours rota from 2029-04-01 with Joakim Wester as mentor, one of the two vacant posts re-profiled as a first rung, monthly tabletops in the incident and problem forum, manual runs of its three runbooks, and the portal failover re-test under AIDR-0919 as the first rotation exercise. No scripted runbook is raised above Level 2 until the observer of AIDR-0910 is in force and a window has two evidenced holders, and MBL-NR-IT-2027-01 governs what an exercise log may contain. Hansa Mutual protects an evidence cycle against the erosion of skill by its own agent; Nordvik Region protects scarce experience against the loss of the one person who holds it.

Summary

Operators need five groups of competences: systems thinking, verification of an agent's proposals, manual operation, authorship of what the loop runs and judgement under uncertainty. The chapter concerned all five operations functions: O1 Observe, O2 Diagnose, O3 Change, O4 Restore and O5 Sustain. The evidence on skill beside automation is long in theory and short in measurement, and none of it measures operators. It shows in other domains that unaided performance can fall after sustained use of an aid, that the cognitive part of a task is more at risk than the motor part, that the effect depends on how the aid is used, and that novices can gain from it.

A craft is kept on purpose. The manual-practice clause of Chapter 16 has an interval of 90 days, an agent switched off, an evidenced holder who did not write the runbook, a record of every missing step and a consequence that falls on the delegation: when evidence lapses and a window has fewer than two evidenced holders, its Level 3 classes drop to Level 2. Dark days, game days with the agents off, Wheels of Misfortune and seeded-error drills supply the practice, are scoped in writing, keep the observer and the stop path in service and end in an improvement. The early-career route replaces the lost routine with designed first rungs, an attempt-first rule and rotation. Competence is evidenced by records of exercises, mapped to SFIA and e-CF and agreed with employee representatives.

The verdict is that decay of unaided performance is observable in other domains and expected in the cognitive part of manual recovery in operations, that a thinning supply of entry-level operators is expected where organisations let it, and that wholesale loss of the craft is speculation. The reader can choose one runbook that an agent runs, have an evidenced holder who did not write it run the manual form with the agent off this quarter, and file every missing step as a change. Chapter 27 turns to the future of operations, with the author's labelled projections.

Sources

  1. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, 12 July 2024. (1, European Union, 2024)
  2. European Parliament and Council of the European Union. Regulation (EU) 2022/2554 on digital operational resilience for the financial sector (DORA). Official Journal of the European Union, 27 December 2022. (2, European Union, 2022)
  3. European Parliament and Council of the European Union. Directive (EU) 2022/2555 on measures for a high common level of cybersecurity across the Union (NIS 2 Directive). Official Journal of the European Union, 27 December 2022. (3, European Union, 2022)
  4. European Parliament and Council of the European Union. Regulation (EU) 2016/679 (General Data Protection Regulation). Official Journal of the European Union, 4 May 2016. (6, European Union, 2016)
  5. Nordqvist, G. (2025). Architecture as Code. The Transformed Architecture, Book 1. (295, Nordqvist, 2025)
  6. Nordqvist, G. (2026). AI Governance. The Transformed Architecture, Book 2. (296, Nordqvist, 2026)
  7. Nordqvist, G. (2026). AI Threat. The Transformed Architecture, Book 5. (299, Nordqvist, 2026)
  8. Nordqvist, G. (2026). The Transformed Security Architect. The Transformed Architecture, Book 6. (300, Nordqvist, 2026)
  9. Nordqvist, G. (2026). AI Boss. The Transformed Architecture, Book 7. (301, Nordqvist, 2026)
  10. Nordqvist, G. (2026). AI Blue. The Transformed Architecture, Book 8. (302, Nordqvist, 2026)
  11. Nordqvist, G. (2026). AI Purple. The Transformed Architecture, Book 9. (304, Nordqvist, 2026)
  12. Bainbridge, L. (1983). Ironies of Automation. Automatica 19(6), 775-779. (217, Bainbridge, 1983)
  13. Parasuraman, R. & Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors 52(3), 381-410. (218, Parasuraman & Manzey, 2010)
  14. ENISA (2022). European Cybersecurity Skills Framework (ECSF) Role Profiles. (219, ENISA, 2022)
  15. SFIA Foundation (2024). SFIA 9: the global skills and competency framework for the digital world. (220, SFIA Foundation, 2024)
  16. Jones, C., Wilkes, J. & Murphy, N., with Smith, C. (2017). Service Level Objectives. Chapter 4 in Site Reliability Engineering. (241, Jones et al., 2017)
  17. Rau, V. (2017). Eliminating Toil. Chapter 5 in Site Reliability Engineering. (253, Rau, 2017)
  18. Jha, S. et al. (2025). ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Tasks. ICML 2025. (225, Jha et al., 2025)
  19. Widdowson, A. (2017). Accelerating SREs to On-Call and Beyond. Chapter 28 in Site Reliability Engineering. (232, Widdowson, 2017)
  20. Casner, S. M., Geven, R. W., Recker, M. P. & Schooler, J. W. (2014). The Retention of Manual Flying Skills in the Automated Cockpit. Human Factors 56(8). (233, Casner et al., 2014)
  21. Budzyń, K., Romańczyk, M. et al. (2025). Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy. The Lancet Gastroenterology & Hepatology 10. (234, Budzyń et al., 2025)
  22. Bastani, H. et al. (2025). Generative AI without guardrails can harm learning. PNAS 122(26). (235, Bastani et al., 2025)
  23. Shen, J. H. & Tamkin, A. (2026). How AI Impacts Skill Formation. arXiv:2601.20245. (236, Shen & Tamkin, 2026)
  24. Brynjolfsson, E., Li, D. & Raymond, L. R. (2025). Generative AI at Work. The Quarterly Journal of Economics 140(2). (237, Brynjolfsson et al., 2025)
  25. European Parliament and Council of the European Union (2026). Regulation (EU) 2026/1744 (Digital Omnibus on AI). (11, European Union, 2026)
  26. CEN (2019). EN 16234-1:2019 e-Competence Framework (e-CF), Part 1: Framework. (67, European Committee for Standardization (CEN), 2019)