Skip to content

Governance and the Lifecycle

Treat every skill as a stopgap written to be deleted when the model catches up. That framing is the opposite of how most teams treat skills — as permanent assets that only accumulate — and it's the reason libraries rot. Governance decides who's accountable while a skill lives; the lifecycle decides when it enters and, crucially, when it leaves. The exit is the part almost everyone skips.

Ownership: a team, never a person

Every team and shared skill carries an OWNERS file, and the accountable party is a team, not an individual — because individuals change roles and a skill outlives them, while a regression still needs someone on the hook today. The spine skill you've followed since Module 01, incident_triage, is owned exactly this way:

# Weak — accountability evaporates when this person switches teams
owner: alice

# Strong — a team, a tracker, and a response commitment (incident_triage's OWNERS)
team: sre-platform
approvers: [alice, budi]
issue_tracker: JIRA/SRE-SKILLS
support_slo: 5 business days

The SLO isn't decoration: a skill that regresses and sits unfixed for weeks is a liability loaded into everyone's agent. It's also tuned to blast radius — partner_api commits to a tighter 3-business-day SLO than incident_triage's 5, because a broken partner integration blocks external teams who can't route around it. Ownership is what converts "someone should fix that" into "this team fixes it, by this date."

Review scales with blast radius

The review bar tracks how many agents a change can affect — the same personal / team / shared tiers from Module 04, now read as escalating review gates. A personal skill needs no review; a shared-tier change gets platform review plus an eval gate, because a regression there hits every agent in the organization. Root approval rights for the shared tier stay deliberately narrow, while contributors maintaining one skill are added to that skill's OWNERS rather than the root.

The load-bearing rule: every change to SKILL.md or references/ is a behavioral change, not a docs edit, because those files are executable instructions. So a change isn't reviewable on the diff alone — it needs an ablation report proving it didn't regress. Only genuinely non-instruction edits (an OWNERS tweak, a build refactor) may skip the eval, and only with a stated reason.

Official skills and consolidation

An "official" badge is an endorsement backed by a commitment, not a quality star you award yourself. It means the team that owns the underlying tool also publishes the skill (the payments team owns the payments skill, not a well-meaning fork), commits to a support SLO, and can show ablation evidence of net-positive effect. It's a long-term maintenance promise, which is why it's scarce.

The counterweight to sprawl is consolidation: fewer, higher-quality skills beat many overlapping forks, which fragment users and degrade everyone's experience. When you fold a fork back, add eval cases proving the canonical skill now covers what the fork did before deleting it — the eval is how you prove coverage rather than assert it.

The lifecycle, end to end

Create → evaluate → ship → maintain → retire. Module 05 covered evaluate; the bookends matter just as much. Before creating, ask whether the skill should exist at all: if the failure is really a bad tool error message, fix the tool — a deterministic fix beats spending agent context forever. Ship with a final --runs=5 ablation and the EVAL_REPORT link the review gate requires.

Maintenance is where the operator mindset shows. Beyond keeping SKILL.md in sync with the tool, healthy libraries watch online quality signals: a pattern where production trajectories are auto-graded by an LLM-as-judge — a public, widely used evaluation technique — on a handful of quality dimensions (instruction adherence, execution success, and applicability are reasonable choices), then rolled into a single score. The exact rubric is a design choice, not a standard; any monotonic scheme works. One illustrative construction grades each trajectory SUCCESS / PARTIAL / FAILURE and computes 100 × (SUCCESS + 0.5×PARTIAL) / total — so 8 SUCCESS, 2 PARTIAL, and 0 FAILURE across ten trajectories would score 90. Whatever the formula, cluster the low-scoring trajectories by root cause and each cluster becomes a ranked, concrete fix — the same "diagnose before editing" loop from Module 05, now fed by real usage instead of a fixed eval suite.

Retire on evidence

Retirement is a first-class stage, not a failure. Retire a skill when any of these hold, and re-check on every major model upgrade:

  • The model caught up — ablation lift has fallen to ~zero, so the skill is now pure context cost. The lift incident_triage showed in Module 05 isn't permanent; re-run its eval on each model upgrade, and the day a newer model triages the outage unaided, retire it.
  • The tool fixed it — the workaround the skill encoded was fixed at source.
  • Usage died — near-zero trajectories for a quarter with no owner advocating.
  • It was consolidated — a canonical skill absorbed it (prove coverage first).

The process is gentle: mark it deprecated in the description ("Deprecated: use X"), remove it from default configs, wait one release cycle for stragglers, then delete. Keep the EVAL.yaml even after the skill is gone — it becomes a regression test that proves the base model still handles the task, so you'll know if that ever stops being true.

Key takeaways

  • Ownership is a team plus an SLO; review rigor scales with how many agents a change can reach, and every instruction change needs ablation evidence.
  • "Official" is a support commitment, and consolidation beats a field of forks — proven with eval cases, not assertions.
  • Skills are stopgaps: retire on falling lift, upstream fixes, dead usage, or consolidation, and keep the eval as a regression test after deletion.

Drill it

  • ex09_review_a_submission — apply the governance checklist to a deliberately flawed skill PR and write the verdict.
  • ex10_retire_a_skill — read usage and lift data, decide retire / keep / consolidate, and execute the retirement checklist.

(Exercises unlock in the Exercises section; direct links are wired as each one lands.)