Setup¶

Ten minutes of pre-work. By the end you'll have the repo cloned, a Python environment ready, and the three gates — sanitize, validate, and eval — running green. Every exercise depends on these working, so don't skip the verification step.

Prerequisites¶

Tool	Version	Why
Python	3.11+	Runs the validators and the eval harness
git	any recent	Clone the repo, branch per exercise
A SKILL.md-compatible agent CLI	optional	Real eval numbers instead of demo ones

The agent CLI is genuinely optional. The whole workshop runs in demo mode, which simulates agent trajectories so you can see the eval workflow and report format end-to-end without any live infrastructure or external account.

1. Clone the repo¶

git clone https://github.com/catwang42/skill-library-workshop.git
cd skill-library-workshop

2. Create a Python environment¶

A virtual environment keeps the workshop's dependencies off your system Python. The only two dependencies are the docs theme and a YAML parser.

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

requirements.txt pins the toolchain by major version — MkDocs is held below 2.0 on purpose (the upstream 2.0 rewrite drops the plugin system Material depends on), so your build matches CI.

Windows

Activate with .venv\Scripts\activate instead of source. Everything else is identical.

3. Verify the three gates¶

These three commands are the workshop's feedback loops — you'll run them constantly. Confirm each works now, so a failure later means your change broke something, not your setup.

Gate 1 — sanitize (content is clean)¶

bash .claude/hooks/sanitize.sh

sanitize: CLEAN

This scans for banned terms. It runs automatically as a hook and in CI too, but running it yourself is the fastest way to catch a slip.

Gate 2 — validate a skill's structure¶

python3 library/tools/validate_skill.py library/skills/incident_triage

library/skills/incident_triage: PASS (0 warning(s))

validate_skill.py is a structural presubmit: it checks the frontmatter, the description's triggers, the eval shape, and that every referenced file exists.

Gate 3 — run an ablation eval (with vs. without the skill)¶

python3 library/tools/skilleval.py run \
  library/skills/incident_triage/EVAL.yaml --with-vs-without-skill --demo

suite: incident_triage  cases: 3  runs/case: 1  ablation: True

  case: triage_active_error_spike
      $ ./scripts/seed_fixture.sh error_spike   (skipped: demo mode)
      $ ./scripts/seed_fixture.sh --teardown   (skipped: demo mode)
    with-skill:    1/1 passed   without-skill: 1/1 passed
  case: correlate_across_timezones
      $ ./scripts/seed_fixture.sh tz_mismatch   (skipped: demo mode)
      $ ./scripts/seed_fixture.sh --teardown   (skipped: demo mode)
    with-skill:    1/1 passed   without-skill: 0/1 passed
  case: rollback_recommendation_only
      $ ./scripts/seed_fixture.sh error_spike   (skipped: demo mode)
      $ ./scripts/seed_fixture.sh --teardown   (skipped: demo mode)
    with-skill:    1/1 passed   without-skill: 0/1 passed

  with-skill pass rate:    100.0%
  without-skill pass rate:  33.3%
  lift:                    +66.7 pts

  Ship guidance: a skill that shows no meaningful lift does not justify its context cost — fix it or don't ship it.

Three cases, run with and without the skill; the lift line — the gap between the two pass rates — is the delta that the whole workshop turns on.

These numbers are simulated, not measured

--demo mode simulates both the agent's trajectory and the judge's verdict. It exists so you can see the report's shape — cases, pass rates, the lift line — without wiring up a live agent. The +66.7 is illustrative, not a measurement. Real lift numbers arrive only when the harness is connected to a live agent, a tracked Phase 5 task. What's real today is the workflow: run with, run without, compare, decide.

4. (Optional) preview the site locally¶

If you want to read the modules and exercises in your browser rather than on the published site:

mkdocs serve

Then open http://127.0.0.1:8000.

You're ready¶

All three gates green? You're set. Continue to the concept modules, then work the exercises in order — each one runs against the library/ you just cloned.