Skill Scout

Continuous skill acquisition pipeline for AI assistants: discover → evaluate → adopt → package → install.

Skills are SKILL.md files with YAML frontmatter that extend an AI assistant's capabilities. They're distributed as .skill files (zip archives) and installed through the platform's skills UI. This skill provides a systematic framework for finding them in community repositories, vetting them for quality and security, packaging them, and managing a local library.

Directory Layout

Create this structure in your workspace:

skill-library/
  incoming/   — Pulled from sources, awaiting review
  vetted/     — Reviewed and approved, organized by domain:
    finance/         — Investment, trading, due diligence
    dev/             — Development tools, debugging, architecture
    science/         — Research methodology, writing, peer review
    data/            — Analysis, visualization, databases
    process/         — Workflows, estimation, incident response
    communication/   — Writing, stakeholder comms, documentation
    testing/         — TDD, verification, evaluation
  rejected/   — Declined with documented reasons
  archive/    — Retired skills kept for reference (not actively loaded)
  logs/       — Scout run history and decisions
  zips/       — Packaged .skill files ready for installation
  skill-scout/ — This skill (meta-skill)

Domain subdirectories are optional — you can start flat and reorganize later. The inventory script handles both layouts.

Pre-flight Checks

Run BEFORE any Scout or Vet workflow:

1. Environment Awareness Snapshot

Scan your current environment for existing capabilities before scouting. Any new candidate must not duplicate what's already covered.

For claude.ai users, check these installed layers:

• /mnt/skills/public/ — Anthropic public skills (docx, pdf, pptx, xlsx, frontend-design, etc.)
• /mnt/skills/examples/ — Anthropic example skills (skill-creator, mcp-builder, web-artifacts-builder, theme-factory, etc.)
• /mnt/skills/user/ — Your installed user skills

For other platforms, check:

• Platform built-in skills (e.g., Claude Code's built-ins, Cowork's .skills/skills/)
• Installed plugins and their bundled skills

Then run scripts/inventory.py to see your vetted library.

2. Cull Pass Check

Check when you last ran the cull workflow. If more than 90 days, consider culling before adding new skills — dead weight accumulates and the model landscape changes fast.

Workflows

| User Intent | Workflow | |---|---| | Find / check for new skills | → Scout | | Evaluate skills in incoming/ | → Vet | | Prepare a vetted skill for installation | → Package | | Retire outdated skills from vetted/ | → Cull | | Check what's available | → run scripts/inventory.py |

---

Scout (Discovery)

Search sources for new skills. Maintain your source registry in references/sources.md.

1. Run Pre-flight Checks
2. For each tracked source, fetch the current skill listing (README, catalog, or directory listing)
3. Compare candidates against your full pipeline — check vetted/, incoming/, AND rejected/ to avoid re-pulling
4. Redundancy scan for each candidate:
   • Check against ALL vetted skills and all environment layers
   • If >70% purpose overlap with any existing skill → flag as redundant, do NOT pull into incoming
   • Log reason for skipping in scout report
5. Usefulness review for every non-redundant candidate:
   • Identify 1–3 closest existing skills by purpose
   • Ask: Would loading this skill meaningfully improve the AI's output on tasks in its domain?
   • If strictly better than an existing vetted skill → recommend replacing the old one
   • If equivalent coverage → skip, log "not an improvement over [existing]"
   • If genuinely new capability → proceed
6. For each skill that passes both checks:
   • Create incoming/<skill-name>/SKILL.md
   • Add YAML frontmatter: name, description, source (URL), trust_tier (1-3)
   • Preserve any bundled resources the skill depends on
7. Log run: logs/scout-YYYY-MM-DD.md with sources checked, findings, decisions, skips
8. Report: what's new, what looks promising, what needs vetting, what was skipped and why

---

Vet (Evaluate & Decide)

Evaluate skills in incoming/ against the checklist in references/vetting.md.

1. Mandatory file audit first. Run find <skill-directory> -type f and account for every file — bundled files are the primary attack surface.
2. Check every non-markdown file:
   • Scripts (.py, .sh, .js, .ts) → read every line, understand what each does
   • Config files (.json, .yaml) → check for fields like "command":, "exec":, "eval": indicating code execution
   • Binary files → reject unless clearly a legitimate asset (image, font)
3. Redundancy re-check: Confirm the skill still adds value (environment may have changed since scouting)
4. Score against the 5-dimension checklist (Structure, Value, Quality, Security, Adaptability — 1–5 each, 25 total)
5. For complex multi-step skills: If you have access to the skill-creator skill, use its eval loop to test the skill against representative prompts before adopting — qualitative testing beats checklist scoring alone
6. Decision:
   • Score ≥ 20 → Adopt. Copy to correct domain subdirectory in vetted/, remove from incoming/
   • Score 15–19 → Adopt with modifications. Document changes in MODIFICATIONS.md, copy to vetted/<domain>/
   • Score < 15 → Reject. Create REJECTION.md with scores and reasons, move to rejected/
7. Human gate: Skills containing ANY executable code cannot be auto-adopted. Flag with full file listing and wait for human approval.
8. Explain every decision

---

Package (Prepare for Installation)

Convert a vetted skill into an installable .skill file.

A .skill file is a zip archive containing the skill directory. It's what the platform's Skills UI expects for installation.

1. Confirm skill is in vetted/<domain>/<skill-name>/
2. Copy to a writable temp location to avoid permissions issues: /tmp/<skill-name>/
3. Run the packager (see scripts/package_skill.py):

   python3 scripts/package_skill.py /tmp/<skill-name>/ ./zips/
   

   Or use Anthropic's official packager if available:

   python -m scripts.package_skill <path/to/skill-folder> ./zips/
   

4. The resulting <skill-name>.skill file lands in zips/
5. Install through the platform Skills UI — Claude cannot install directly

Batch packaging: To package all vetted skills at once, run package_skill.py without a target argument (if your version supports it) or loop over the vetted/ directory.

---

Load (Session Bootstrap)

Identify and load vetted skills relevant to the current task.

1. Run scripts/inventory.py <skill-library-path> to list available skills
2. Read YAML frontmatter from each skill in vetted/ to get names and descriptions
3. Match skill descriptions against the current task domain
4. For matched skills, read the full SKILL.md body into context
5. Report which skills were loaded and why

Principle: Skills share the context window. Load only what's relevant. Three to five skills per session is typical; more than seven is likely too many.

---

Update (Check for Upstream Changes)

Periodically re-check adopted skills against their source versions.

1. For each skill in vetted/ with a source field in its frontmatter, fetch the current version
2. Compare against the local copy — look for meaningful changes, not just whitespace
3. If updated: pull to incoming/, re-vet with the full checklist (pay extra attention to newly added files or code), then replace the vetted version if it passes
4. Log all update checks in logs/update-YYYY-MM-DD.md

---

Cull (Retire Obsolete Skills)

Periodic review to drop skills that no longer earn their place.

1. Run scripts/inventory.py for the full current inventory
2. For each skill, apply the retirement test:
   • Has a newer/better skill superseded it (in vetted/ or in platform built-ins)?
   • Has the underlying model improved enough to make the skill unnecessary?
   • Is this domain still relevant to your actual work?
3. Propose retirements with reasoning — human approves before anything moves
4. Retired skills: move to archive/ (keep for reference) or rejected/ (permanently dropped) with documented reason
5. Note the cull date so the pre-flight check stays accurate

---

Trust Tiers

| Tier | Source Type | Vetting Required | |------|-----------|-----------------| | 1 | Platform vendor official repos | Structure check only. Adopt unless environment-incompatible. | | 2 | High-reputation, actively maintained (10k+ stars) | Full 25-point checklist. Likely needs modifications for your environment. | | 3 | Individual repos, social media shares, unknown origin | Deep vet. All files read line-by-line. Assume untrusted. |

Star counts are social proof, not security audits. A compromised maintainer can push malicious updates to a trusted repo. Trending skills get more scrutiny, not less.

Rules

1. Never execute scripts from unvetted skills
2. Prefer methodology skills (plain markdown) over executable skills — safer and more portable
3. Skills with executable code require human approval before adoption
4. Log every scout run and every vetting decision — the audit trail matters
5. When modifying community skills, document every change in MODIFICATIONS.md
6. One skill, one purpose — if a community skill does too much, split it or reject it
7. Re-vet adopted skills if their upstream source reports a security incident
8. Place adopted skills in the correct domain subdirectory, not at vetted/ root
9. Skills must be packaged as .skill files before installation — vetted/ is the library, not the install location
10. Always run pre-flight checks before scouting

Reference Files

• references/sources.md — Source registry template with community examples
• references/vetting.md — Full security audit procedure and 25-point scoring checklist
• scripts/inventory.py — Skill inventory scanner
• scripts/package_skill.py — Single and batch .skill packager