Make Your Skill Better

Project 01 made the Skill exist. This project makes the Skill actually useful. The two are very different.

The fastest way to find out what's wrong with a thing you made is to watch a real person try to use it without you helping. They'll do something you didn't expect. They'll stop at a moment that confused them. They'll ask the Skill a question you'd never have thought to ask.

That's the whole project. Watch. Note. Fix three things. Teach the Skill to say "I don't help with that" — gracefully, like care, not like a wall.

Step by step

1. Pick three friends. Make sure they're not all the same kind of person.

   Real engineers test with different people on purpose. For your three: one who knows your domain a little, one who knows nothing about it, one who's a beginner. If you only test with friends who are already experts, you'll only fix expert problems.

2. Watch them use it for 5 minutes each.

   Open Claude with your SKILL.md loaded. Hand the keyboard to your friend. Don't help. Don't explain. Just watch. Have a piece of paper. Write down every moment they: hesitate, look confused, ask you a question they should have asked the Skill, give up.

   Five minutes feels long. It's the right length. Real user testing sessions are 30–60 minutes — you're getting 5 minutes per friend × 3 friends = 15 minutes of real signal.

3. Group what you noticed into 3 fixable things.

   You'll probably have 6–10 notes total. Most of them will overlap. Two friends got confused at the same point — that's one fix, not two. Group them. Pick the three that show up the most.

4. Make the three fixes in your SKILL.md.

   For each: edit the knowledge or style section to address what confused the friend. Don't add new features — just fix the existing ones.

   After each fix, re-run the question that confused them. The Skill should now answer it cleanly.

5. Teach the Skill to say "I don't help with that."

   This is the most underrated upgrade you can make. Write 1–2 sentences explaining what your Skill is not for. Give a redirect to a real other tool. The Skill is now polite, scoped, and trustworthy.

   The format matters. "I don't help with X — try [other tool] instead. Want to come back when you have a question about [your scope]?" Care, not wall.

6. Ship v0.2 to one of the three friends.

   Pick the friend who struggled most in step 2. Give them v0.2. Ask: "Did the parts that confused you last time work better now?" Listen. Note one thing they say. That's your v0.3 backlog.

A complete worked example

Here's the v0.1 → v0.2 evolution of the pokemon-go-mei Skill from project 01. Three friends tested it. Three fixes were made. One refusal pattern got added.

## Mei (8, my actual little sister — domain-friendly)

  - Asked "what's IV?" — the Skill assumes she knows
  - Got bored when the response was 3 sentences
  - Wanted to know why I said "skip" for Weedle, not just that we should
  - Liked the follow-up questions

## Lucas (10, friend, never played Pokemon Go)

  - First question: "What's a Pokemon?" → confused
  - The Skill assumed too much. He gave up after 2 minutes.
  - Said "this only works if you already play"

## Emma (12, beginner — played 1 week)

  - Asked about gym battles. Skill correctly refused.
  - But: said "I don't help with that" felt like the Skill was being mean
  - Asked: should I evolve my Pidgey? Skill answered confidently — good
  - Got stuck on candy math — said "I don't get how 25 + 100 works"

## summary going into the 3 fixes

  the patterns:
    1. The Skill assumes too much (Mei: IV, Lucas: literally everything,
       Emma: candy math)
    2. The "skip" verdicts feel arbitrary without the WHY
    3. The refusal sounded cold to Emma

  3 actual fixes:
    fix #1: add a one-line gloss for any term used (IV, candy total, etc.)
    fix #2: every verdict gets a one-clause WHY ("skip — beedrill is the
                worst attacker in the game")
    fix #3: rewrite the refusal sentence to be warmer

## changes from v0.1 to v0.2

style:
  - Short. Mei is 8 and English is her second language.
  - Always say the verdict first (yes / no / wait).
+ - Always include a one-clause WHY after the verdict.
+ - If you use a Pokemon Go term (IV, CP, candy) for the FIRST
+   time in a conversation, gloss it in 8 words or fewer.
  - End with one short follow-up question.

knowledge:
  evolution_traps:
    - name: weedle
      candy_cost: 12 + 50
-     verdict: "Skip. Beedrill is one of the worst attackers
-               in the game."
+     verdict: "Skip — Beedrill barely hurts anything in the game.
+               Save the candy for a Caterpie instead. (Caterpie
+               and Weedle look similar — Caterpie is the green one.)"
      mei_specific: "She has like 800 weedle candy."

  out_of_scope_redirect:
-   "I only know what to evolve. Ask Pokemon Go GamePress for
-    raid stuff."
+   "Gym battles are super fun and I wish I knew them, but I'm
+    only good at evolutions. The GamePress raid guide knows
+    way more about gyms than I do — try them. Want to come
+    back if you have a Pokemon you're not sure about evolving?"

# new
  glossary_first_use:
    IV:    "how good a Pokemon's stats are, 0–100%"
    CP:    "how strong it is in fights, higher = stronger"
    candy: "the dust that lets you power up or evolve"

Live demo 1: collect friend-test notes

Use this widget while you watch each friend test your Skill. The structured fields force you to capture the right things — what they did, where they hesitated, what they said. Copy the notes into your friend-notes.md when you're done.

Live demo 2: write a refusal that sounds like care

Paste a refusal sentence you might add to your Skill. The widget checks four things: is it short? Does it suggest another tool? Does it invite the user back? Does it apologize too much?

What makes this hard

The hardest part isn't the fixing. It's watching without helping. Your friend will get stuck. You'll want to lean over and explain. Don't. The 5 seconds of awkward silence while they figure it out — or fail to — is the most valuable signal in the whole project. Real engineers learn to sit through it. So can you.

The second hardest part is not adding features. Your friend asked about gym battles. You feel tempted to add a "gyms" section to your Skill. Don't. Tight scope is what makes the Skill trustworthy. Polite refusal is the right move.