Skip to content
alyn

Alyn Bench

Does this skill work? We measure it.

Every skill runs on real tasks with Haiku, Sonnet and Opus, with and without the skill, three times each. Graded automatically. The question that matters: can a cheaper model with the right skill do the job?

The matrix, per task

ModelWithout skillWith skillRepeats
Claude Haiku✓✓3×
Claude Sonnet✓✓3×
Claude Opus✓✓3×

4 tasks per skill: 3 real tasks plus 1 negative control the skill should not touch. Runs in Claude Code headless mode in a fresh folder each time.

Six numbers per skill. One grade.

Lift

Pass rate with the skill minus without, on the same model and the same tasks.

Downshift

Does Haiku with the skill beat Opus without it? Positive means you can drop a model tier.

Cost per success

Total API cost divided by tasks solved. Includes the extra context the skill adds.

Trigger rate

How often the agent loads the skill on its own, and how rarely on tasks where it does not belong.

Consistency

Share of tasks solved in all three repeats, not just once by luck.

Safety

Bundled scripts scanned for network calls and destructive commands. Any finding blocks Verified.

Alyn Verified

score = 45% lift
      + 25% credit savings
      + 15% trigger reliability
      + 15% consistency

A+ ≥ 85 · A ≥ 75 · B ≥ 60 · C ≥ 45

Credit saver badge

Haiku with the skill beats Opus without it.

That is the skill that pays for itself. Independent studies show it on average; Alyn Bench shows it per skill.

First 30 skills in the queue.

The pilot with 5 skills runs first. Results appear here and on each skill page.

UnslopDesign Taste FrontendImpeccableHallmarkFrontend DesignWeb Design GuidelinesHumanizerAvoid AI WritingCopywritingPDFDocxXlsxPptxTest Driven DevelopmentSystematic DebuggingBrainstormingWriting PlansCode ReviewSecurity AuditSEOAI SEOGEOCreate PRDRemotion Best PracticesMCP BuilderVercel React Best PracticesSupabase Postgres Best PracticesCavemanKarpathy GuidelinesObsidian Markdown

Blue dot: pilot.

Questions

Get your skill benchmarked
What is Alyn Bench?

An open benchmark that runs each skill on real tasks in Claude Code with Haiku, Sonnet and Opus, with and without the skill, three times each, and grades the output automatically.

What does Alyn Verified mean?

The skill measurably improves results (lift), is consistent across repeats, triggers when it should and passes the safety scan. Grades go from A+ to C.

What is a Credit saver?

A skill that lets Claude Haiku beat Claude Opus without the skill on the same tasks. Independent studies already show this effect on average (SkillsBench, Tessl); Alyn Bench measures it per skill.

Can I benchmark my own skill?

Soon: npx alyn bench ./my-skill runs the same matrix with your own API key. Until then, submit your skill and tell us which tasks it is for.