Exercise 01: Dissect a Skill¶
Duration: 40 min | Module: 2 — Anatomy of a Skill
Objective¶
Take the bundled incident_triage skill apart layer by layer and learn to read
a skill the way an agent does. By the end you'll have predicted the skill's
trigger behavior from its description alone, mapped each of its eval cases to the
part of the body that makes it pass, and watched the structural validator flag a
weakened description. No files in library/ are modified — this is a reading and
prediction exercise with observable checkpoints.
Prerequisites¶
- Completed Setup (repo cloned,
mkdocs+ Python working) - Skim Module 2 — Anatomy of a Skill; this exercise is its hands-on counterpart.
Part 1: Map the anatomy on disk (8 min)¶
Step 1: List the skill's files¶
Expected output:
library/skills/incident_triage/EVAL.yaml
library/skills/incident_triage/OWNERS
library/skills/incident_triage/SKILL.md
library/skills/incident_triage/references/summary_template.md
library/skills/incident_triage/scripts/seed_fixture.sh
Match each file to its disclosure layer from Module 2: SKILL.md holds layer 1
(the always-loaded frontmatter) and layer 2 (the on-trigger body);
references/ is layer 3 (loaded only when the body points to it); scripts/
holds deterministic helpers; EVAL.yaml and OWNERS are what make the skill
operable — its contract and its accountable owner.
Why the layering matters
Layer 1 is paid for on every request, so it must be tiny; layer 3 can be
large because it's rarely loaded. Most authoring mistakes are a layer
confusion — bulk that belongs in references/ sitting in the body, or
triggers buried where the agent can't see them until it's too late.
Step 2: Run the structural gate¶
Expected output:
That PASS is your baseline. validate_skill.py checks the frontmatter, the
description's triggers, the eval shape, and that every referenced file exists.
You'll make it complain in Part 4.
Part 2: Predict the trigger from the description alone (12 min)¶
The only thing an agent sees before deciding to load a skill is its name and
description. So that's all you get for this Part — no peeking at the body.
Step 1: Read only the frontmatter¶
Expected output:
---
name: incident-triage
description: >-
Triages production incidents: pulls recent alerts, correlates deployments and
error spikes, identifies the likely owning service, and drafts an incident
summary with severity and next steps. Use when investigating an outage, a
latency spike, a surge in errors, a failing health check, or when asked
"what's broken" / "why are bookings failing". Don't use for postmortem
writing (use the postmortem skill) or for filing routine bugs.
---
Step 2: Predict trigger behavior¶
For each user request below, decide from the description alone whether the agent
should load incident_triage (TRIGGER) or not (SKIP). Write down your
six answers before revealing the key.
| # | User request | Your call |
|---|---|---|
| 1 | "Checkout latency just spiked to 5s across every region — what's going on?" | ? |
| 2 | "Write up the postmortem for yesterday's outage." | ? |
| 3 | "Why are bookings failing right now?" | ? |
| 4 | "File a bug: there's a typo in the settings-page footer." | ? |
| 5 | "Error rate is climbing and requests are timing out — is something broken?" | ? |
| 6 | "Can you add a dark-mode toggle to the dashboard?" | ? |
Answer key (open after you've committed to six answers)
- TRIGGER — "latency spike" is a named Use when trigger.
- SKIP — postmortem writing is an explicit Don't use for negative trigger; it points to a sibling skill.
- TRIGGER — "why are bookings failing" is quoted verbatim in the description. This is the clearest possible match.
- SKIP — "filing routine bugs" is an explicit negative trigger.
- TRIGGER — "surge in errors" / "what's broken" both match.
- SKIP — a feature request, not an incident. Nothing in the description reaches it; it should quietly not load.
Why the negative triggers earn their space
Cases 2 and 4 are the interesting ones. Without the Don't use for line, a postmortem or a bug-filing request could plausibly match "incident" and pull this skill in, where it would do the wrong job. The negative fence is what makes the trigger precise, not just broad.
Part 3: Trace progressive disclosure to the contract (12 min)¶
Now open the body and connect it to the eval. A well-built skill's eval cases each target a specific piece of the body; if you can't find the body content a case depends on, that's a gap.
Step 1: List the eval cases¶
Expected output:
8: - name: triage_active_error_spike
27: - name: correlate_across_timezones
43: - name: rollback_recommendation_only
Step 2: Find the gotcha the timezone case depends on¶
Expected output:
Open library/skills/incident_triage/SKILL.md and read that gotcha in full
(lines 84–85), then map each eval case to the body content that makes it pass:
| Eval case | Body content it exercises |
|---|---|
triage_active_error_spike |
The Step 1–4 workflow (alerts → metrics → deploy correlation → severity) |
correlate_across_timezones |
The UTC/local timestamp gotcha (lines 84–85) |
rollback_recommendation_only |
"Check its diff before recommending rollback… schema migrations do not" + "the human decides" |
Why this mapping is the point
The eval is the skill's contract (Module 5). Each case pins a behavior the body is supposed to produce. Reading them together is how you tell a skill that teaches something from one that just describes it — and it's the first thing you'll do when an eval starts failing.
Step 3: Confirm the layer-3 link¶
Expected output:
23:5. Draft the incident summary using the template in `references/summary_template.md`
76:Fill the template in [references/summary_template.md](references/summary_template.md).
The body points to the template rather than inlining it — layer 3 loaded only at the drafting step, exactly as Module 2 describes.
Part 4: Break the description and watch the gate catch it (8 min)¶
You'll weaken the trigger on a throwaway copy — never the real skill — and see the validator flag it. This is the feedback loop you'll use for real in Exercise 03.
Step 1: Copy the skill somewhere disposable¶
Step 2: Replace the description with a vague one-liner¶
python3 - <<'PY'
import pathlib, re
p = pathlib.Path("/tmp/incident_triage_scratch/SKILL.md")
t = p.read_text()
t = re.sub(r"description: >-.*?\n---",
"description: >-\n Helps with incidents and general system health.\n---",
t, count=1, flags=re.DOTALL)
p.write_text(t)
print("\n".join(t.splitlines()[:4]))
PY
Expected output:
Step 3: Re-run the validator on the copy¶
Expected output:
WARN: description has no 'Use when ...' triggers — under-triggering is the #1 skill failure mode
WARN: description has no negative triggers ("Don't use for ...") to disambiguate siblings
/tmp/incident_triage_scratch: PASS (2 warning(s))
The baseline PASS (0 warning(s)) from Part 1 became PASS (2 warning(s)). The
skill still "passes" — a vague description isn't a structural error — but the
validator names exactly the two things that make it under-trigger.
The validator can't catch everything
Structural checks flag missing triggers, not bad ones. A description with a "Use when" line that simply doesn't match how users phrase requests would pass clean. Fixing that kind of vagueness is judgment work — which is the whole of Exercise 03.
Step 4: Clean up¶
Completion Criteria¶
-
find library/skills/incident_triage -type flists all five files and you can name each one's disclosure layer. -
validate_skill.pyon the real skill printsPASS (0 warning(s)). - You committed to six TRIGGER/SKIP predictions before opening the answer key, and can explain the two negative-trigger cases.
- You mapped all three eval cases to the body content they exercise.
- The weakened copy produced
PASS (2 warning(s))naming the missing triggers, and you removed the scratch directory.
Key Takeaways¶
- A skill is three disclosure layers — the agent reads them in order, so match each piece of content to the cheapest layer that can hold it.
- The description alone decides triggering — you predicted six requests from it, because that's all the agent sees before loading.
- Negative triggers make the match precise — the Don't use for line is what keeps neighboring requests (postmortems, bug filing) from pulling the wrong skill.
- Eval cases pin body behavior — reading cases and body together is how you tell a teaching skill from a describing one.
- The validator catches structure, not judgment — it flags missing triggers cheaply, but a vague-yet-present trigger needs a human eye.
Concept background: Module 2 · Next:
ex02_experience_before_theory (unlocks next; the cross-linking pass wires the
direct link once it's built).