{
  "id": 88,
  "title": "Claude Code Skills: How to Build, Test, and Optimize AI Agent Skills",
  "speaker": "Nate Herk",
  "topic": "A comprehensive guide to Claude Code skills — what they are, the two types (capability uplift vs encoded preference), and how to use Anthropic's new Skill Creator to build, evaluate, benchmark, and trigger-tune skills for AI agents.",
  "duration_label": "16:15",
  "theses": [
    {
      "title": "Skills are just text recipes for your AI agent",
      "description": "A skill is nothing more than a set of text instructions — like a prompt or an SOP — that tells Claude how to perform a specific task consistently. Anyone can read and understand a skill file. When you invoke a skill, the agent reads the recipe and follows it, producing reliable output every time."
    },
    {
      "title": "There are two types of skills: Capability Uplift and Encoded Preference",
      "description": "Capability Uplift skills teach Claude something it doesn't do well by default — like front-end design with good fonts, colors, and layouts. Encoded Preference skills encode a specific multi-step workflow personal to you — like a sequential idea mining process. Understanding the difference matters because they have different lifespans and maintenance needs."
    },
    {
      "title": "Capability Uplift skills may fade as models improve",
      "description": "As newer models like Opus 5 arrive, they may natively outperform a skill that was designed for a weaker model. A front-end design skill that helps today might become unnecessary or even counterproductive tomorrow. You should be prepared to retire these skills when the base model surpasses them."
    },
    {
      "title": "Encoded Preference skills are durable because they encode your unique process",
      "description": "Unlike capability uplift skills, encoded preference skills reflect your personal workflows, business logic, and preferences. A new model won't be trained on your specific processes, so these skills remain valuable across model generations. They are your competitive advantage in agent-driven work."
    },
    {
      "title": "Anthropic's Skill Creator skill is a meta-skill for building better skills",
      "description": "Anthropic released an official Skill Creator that packages all their best practices — from a 33-page PDF and internal research — into a single skill. It can create new skills from scratch, modify existing ones, run evaluations, perform benchmarks, and optimize trigger descriptions. It essentially automates the skill development lifecycle."
    },
    {
      "title": "Evals let you measure and improve skill quality automatically",
      "description": "By providing examples of desired outputs, the eval system tests your skill against those examples and identifies weaknesses. This shortcuts the manual feedback loop of using a skill many times and gradually refining it. Evals serve two purposes: catching regressions when models change, and spotting growth when a skill becomes unnecessary."
    },
    {
      "title": "Benchmarks provide quantitative comparison of skill performance",
      "description": "Benchmarks give you pass rate, execution time, and token usage — with and without a skill loaded. This lets you objectively measure the uplift a skill provides and make data-driven decisions about whether to keep, modify, or retire a skill. Run benchmarks whenever a model updates or after you iterate on a skill."
    },
    {
      "title": "Trigger tuning fixes misfires and false triggers across many skills",
      "description": "When you have 10+ skills in a project, the agent may call the wrong skill or miss the right one. Trigger tuning analyzes your skill descriptions, tests various natural language prompts, and rewrites descriptions so skills are invoked more accurately. This eliminates the need to always use slash commands."
    },
    {
      "title": "Skills get better the more you use them within a project",
      "description": "Each time you run a skill and provide feedback — what you liked, what was wrong — the skill creator can incorporate that feedback. Additionally, existing project context like scripts, brand assets, and channel data gets reused automatically, making skills more powerful in context-rich projects."
    },
    {
      "title": "The future of skills is high-level natural language specs",
      "description": "Anthropic themselves state that eventually a natural language description of what the skill should do may be enough, with the model figuring out the rest. Today we still need to provide steps, rules, and format, but the trajectory is toward simply explaining what you want and the agent building the full skill."
    },
    {
      "title": "Skill files use YAML frontmatter and Markdown for structure",
      "description": "A skill's .md file includes YAML metadata at the top — name, description, whether model invocation is enabled, and argument hints. The body contains context, benchmarks, optional focus areas, and step-by-step instructions. The skill can reference external scripts and templates within the project."
    }
  ],
  "exercises": [
    {
      "goal": "Open Claude Code (VS Code, terminal, or desktop app). Run /plugins, click 'manage plugins', search for 'skill-creator', and install the official Anthropic skill creator. Restart Claude Code. Then verify by asking: 'Do you have the skill creator skill? What does it do?' Confirm you get a detailed response about its capabilities.",
      "title": "Install the Skill Creator and verify it works"
    },
    {
      "goal": "Switch to plan mode in Claude Code. Describe a weekly or recurring workflow you do — for example, a weekly report, content roundup, or competitor analysis — in 2-3 sentences of natural language without technical details. Let the Skill Creator ask you clarifying questions and generate a plan. Accept the plan and let it build the skill end-to-end. Observe how it structures the .md file, creates supporting scripts, and runs an eval.",
      "title": "Build your first Encoded Preference skill from a vague prompt"
    },
    {
      "goal": "Take a skill you already use (or the one you just built). Prepare 3-5 examples of ideal output for that skill. Ask Claude to run the skill creator eval process against your skill using those examples. Then run a benchmark with and without the skill loaded. Compare pass rate, time, and token usage. Identify at least one improvement the eval suggests.",
      "title": "Run an eval and benchmark on an existing skill"
    },
    {
      "goal": "If you have 3+ skills in a project, ask the Skill Creator to analyze trigger accuracy. Write down 5 natural language prompts you'd normally use to invoke each skill. Run the trigger tuning process and compare the before/after descriptions. Test your prompts again to see if the correct skill is now invoked. Note any remaining misfires.",
      "title": "Perform trigger tuning on a project with multiple skills"
    },
    {
      "goal": "Run a skill you built, examine the output carefully, and write down 3 specific things that are wrong or missing. Clear your context, switch to plan mode, and provide that honest feedback to Claude. Let it revise the skill and re-run. Compare the outputs. Repeat this cycle at least twice to experience how skills improve through iteration.",
      "title": "Iterate on a skill using feedback loops"
    }
  ],
  "quotes": [
    {
      "text": "It's basically just a recipe. So, that when you ask your agent to make you, for example, a LinkedIn post, it will read the recipe and it will get it right every single time.",
      "author": "Nate Herk"
    },
    {
      "text": "It is literally just text that you could read that an intern could read. Anybody could read and understand what's going on in the skill.",
      "author": "Nate Herk"
    },
    {
      "text": "Over time, a natural language description of what the skill should do may be enough with the model figuring out the rest.",
      "author": "Anthropic (quoted by Nate Herk)"
    },
    {
      "text": "And I really think that this word 'may' should actually have been 'will'.",
      "author": "Nate Herk"
    },
    {
      "text": "If you ask Claude without a front-end design skill to build you a website, it could do it, but it might just look very generic. It might look AI slop as they call it.",
      "author": "Nate Herk"
    },
    {
      "text": "With an encoded preference skill, these will probably stay pretty durable and accurate because the process is very specific usually to you, which Opus 5 won't be trained on most likely.",
      "author": "Nate Herk"
    },
    {
      "text": "You could either take time and learn this or you could just give your agent the skill creator skill and all that information is already in there.",
      "author": "Nate Herk"
    },
    {
      "text": "This is such a good example of why using a project more and using a skill more makes it stronger because some of the pieces that I already had in this project it's able to reuse.",
      "author": "Nate Herk"
    },
    {
      "text": "Most people that are using skills right now are actual just like executives and managers and operators. They're not engineers, which means we're really good at being able to explain what we want, the metrics we need to hit, and why we need that, but maybe not all of those technical nitty-gritty details.",
      "author": "Nate Herk"
    }
  ],
  "full_markdown": "# Claude Code Skills: How to Build, Test, and Optimize AI Agent Skills\n\n> Speaker: Nate Herk | Duration: 16:15\n\n## Key Ideas\n\n1. **Skills are just text recipes for your AI agent** — A skill is a set of text instructions that tells Claude how to perform a specific task consistently. It's like a prompt or SOP that anyone can read and understand. When invoked, the agent follows the recipe and produces reliable output every time.\n\n2. **Two types of skills: Capability Uplift vs Encoded Preference** — Capability Uplift skills teach Claude something it doesn't do well by default (e.g., front-end design with good fonts and layouts). Encoded Preference skills encode your specific multi-step workflow (e.g., idea mining across YouTube, X, and web). The distinction matters because they age differently.\n\n3. **Capability Uplift skills may become obsolete** — As newer, more capable models release, they may natively outperform skills designed for weaker models. Be ready to retire these when the base model surpasses them.\n\n4. **Encoded Preference skills are durable** — Because they encode your unique processes, business logic, and preferences, they remain valuable across model generations. A new model won't be trained on your specific workflows.\n\n5. **Anthropic's Skill Creator is a meta-skill** — It packages all of Anthropic's best practices into one skill that can create, modify, evaluate, benchmark, and optimize other skills automatically.\n\n6. **Evals measure and improve skill quality** — Provide examples of ideal output, and the eval system tests your skill against them. Evals catch regressions (skill works worse with new model) and spot growth (skill becomes unnecessary).\n\n7. **Benchmarks give quantitative metrics** — Pass rate, execution time, and token usage with and without a skill loaded. Run these whenever a model updates or after iterating on a skill.\n\n8. **Trigger tuning fixes misfires** — When you have many skills, the agent may call the wrong one. Trigger tuning rewrites skill descriptions so they're invoked more accurately from natural language.\n\n9. **Skills improve with use and feedback** — Each feedback cycle makes a skill stronger. Projects with rich context (existing scripts, brand assets, channel data) amplify skill effectiveness.\n\n10. **The future is natural language skill specs** — Anthropic expects that eventually a plain description of what you want will be enough, with the model figuring out steps, rules, and format on its own.\n\n11. **Skill files use YAML + Markdown** — A skill .md file has YAML frontmatter (name, description, invocation settings, argument hints) and Markdown body with context, instructions, and references to scripts.\n\n## Transcript\n\n### What is a Skill?\n\nSkills are text-based recipes that instruct an AI agent on how to complete a specific task consistently. They are stored as Markdown files and can be used in Claude Code, Claude Desktop, or VS Code. A skill might tell Claude how to write internal communications, mine video ideas, or generate reports. The key insight is simplicity: skills are just text that anyone — even a non-technical person — can read and understand.\n\n### Two Types of Skills\n\nThere are two fundamentally different types of skills:\n\n**Capability Uplift Skills** are prompts that teach Claude to do something better than its default behavior. For example, a front-end design skill provides knowledge about good fonts, color schemes, layouts, and background elements. Without it, Claude might produce generic, \"AI slop\" websites. With it, results look significantly more professional.\n\n**Encoded Preference Skills** are sequential workflows specific to you. For example, an \"idea mining\" skill first checks YouTube comments, then looks at niche videos, then analyzes AI trends on X and the web. It spins up two parallel agents (YouTube and research), sends their outputs to a main agent for scoring and cross-referencing, and finally generates video ideas. This process is unique to the user's business.\n\nThe critical difference: capability uplift skills may fade as models improve (Opus 5 might be better at design without a skill than Opus 4.6 with one), while encoded preference skills stay durable because they encode your personal process.\n\n### Anthropic's Skill Creator\n\nAnthropic released an official Skill Creator skill that consolidates all their best practices — including content from a 33-page PDF on skill fundamentals, planning, testing, iteration, distribution, and troubleshooting. It can:\n- Create new skills from scratch\n- Modify and improve existing skills\n- Measure skill performance via evals\n- Run benchmarks\n- Optimize skill descriptions for trigger accuracy\n\n### Evals\n\nEvals let your agent evaluate skill quality and make improvements. You provide examples of ideal output, the system tests prompts against your skill, and compares results. This shortcuts the manual feedback loop.\n\nTwo reasons to use evals:\n1. **Catch regressions** — A model update might cause a skill to perform worse. Evals provide early warning signals.\n2. **Spot growth** — A new model might perform better without the skill entirely. Evals help you decide when to retire a skill.\n\nBenchmarks extend this with quantitative metrics: pass rate, execution time, and token usage — with and without the skill loaded.\n\n### Trigger Tuning\n\nWith 10+ skills in a project, you may experience false triggers or misfires. Trigger tuning analyzes skill descriptions, tests various natural language prompts, and rewrites descriptions for more accurate invocation. Testing showed significant improvement in both test and train scores after trigger tuning, though results aren't yet perfect.\n\n### Building a Skill Live: YouTube Weekly Roundup\n\nNate demonstrated building a \"YouTube Weekly Roundup\" skill from a vague natural language prompt. The process:\n\n1. **Vague prompt**: \"Create a skill that looks at my videos from the past week, analyzes comments, views, engagement, and gives me a PDF report with insights, strengths, weaknesses, threats, opportunities.\"\n2. **Clarifying questions**: The Skill Creator asked about time window (rolling 7 days), report sections, and PDF styling (use brand assets).\n3. **Plan generation**: Without any technical specification, the system created a full implementation plan including scripts, templates, and an eval step.\n4. **First iteration**: Produced a PDF with correct structure but inaccurate data and missing sections.\n5. **Feedback and second iteration**: After honest feedback about data quality, the system revised its data fetching approach, reused existing project scripts, and produced a significantly better report with accurate stats, SWOT analysis, comment analysis, competitor context, and AI trend tracking.\n\nThe final skill file included YAML frontmatter, referenced external scripts (fetch YouTube data, prepare data, render report), and could be invoked simply by saying \"do a YouTube weekly roundup.\"\n\n### Final Output\n\nThe polished report included: video statistics, executive summary with key takeaways, per-video breakdown with metrics, SWOT analysis, top comments and audience signals (including video requests and pain points), competitor context with channel comparisons, and trending AI topics. Built in approximately 20 minutes.\n\n## Practical Exercises\n\n### Exercise 1: Install the Skill Creator\nOpen Claude Code. Run `/plugins`, manage plugins, search for `skill-creator`, and install the official Anthropic version. Restart Claude Code and verify by asking: \"Do you have the skill creator skill? What does it do?\"\n\n### Exercise 2: Build an Encoded Preference Skill from a Vague Prompt\nSwitch to plan mode. Describe a recurring workflow in 2-3 sentences of natural language — no technical details. Let the Skill Creator ask clarifying questions, generate a plan, build the skill, and run evals. Observe the full lifecycle.\n\n### Exercise 3: Run Evals and Benchmarks\nPrepare 3-5 examples of ideal output for a skill. Ask Claude to run the eval process. Then benchmark with and without the skill loaded. Compare pass rate, time, and token usage.\n\n### Exercise 4: Trigger Tune Multiple Skills\nWith 3+ skills in a project, run trigger tuning. Write 5 natural language prompts per skill. Compare before/after descriptions and test invocation accuracy.\n\n### Exercise 5: Iterate with Feedback Loops\nRun a skill, identify 3 specific issues in the output, provide honest feedback in plan mode, and let Claude revise. Repeat at least twice to see improvement.\n\n## Best Quotes\n\n> «It's basically just a recipe. So, that when you ask your agent to make you, for example, a LinkedIn post, it will read the recipe and it will get it right every single time.» — Nate Herk\n\n> «It is literally just text that you could read that an intern could read. Anybody could read and understand what's going on in the skill.» — Nate Herk\n\n> «Over time, a natural language description of what the skill should do may be enough with the model figuring out the rest.» — Anthropic (quoted by Nate Herk)\n\n> «And I really think that this word 'may' should actually have been 'will'.» — Nate Herk\n\n> «If you ask Claude without a front-end design skill to build you a website, it could do it, but it might just look very generic. It might look AI slop as they call it.» — Nate Herk\n\n> «With an encoded preference skill, these will probably stay pretty durable and accurate because the process is very specific usually to you, which Opus 5 won't be trained on most likely.» — Nate Herk\n\n> «You could either take time and learn this or you could just give your agent the skill creator skill and all that information is already in there.» — Nate Herk\n\n> «This is such a good example of why using a project more and using a skill more makes it stronger because some of the pieces that I already had in this project it's able to reuse.» — Nate Herk\n\n> «Most people that are using skills right now are actual just like executives and managers and operators. They're not engineers, which means we're really good at being able to explain what we want, the metrics we need to hit, and why we need that, but maybe not all of those technical nitty-gritty details.» — Nate Herk\n",
  "youtube_url": "https://www.youtube.com/watch?v=RAZVk5NPNtE",
  "url": "https://ekstraktznaniy.ru/workbook/88"
}