Skip to content
Join free
oaustegard

Building Property Tests skill

Finds correctness bugs by checking code against a simple reference model over generated inputs and operation sequences, then shrinks each failure to a regression test and a fix. Use for "property test this", "model-based testing", "find bugs in this data structure", or a Hypothesis state machine.

$npx alyn-cli add building-property-tests
View this skill on GitHuboaustegard/claude-skills/building-property-tests/SKILL.md
Pack stars
403
Pack installs
3.9k
Alyn Score
66/99
Updated
20 h ago
License
MIT

oaustegard

Building Property Tests

What is the Building Property Tests skill?

Building Property Tests is a testing skill by oaustegard for Claude Code, Cursor, Codex and other AI coding agents. Finds correctness bugs by checking code against a simple reference model over generated inputs and operation sequences, then shrinks each failure to a regression test and a fix. Use for "property test this", "model-based testing", "find bugs in this data structure", or a Hypothesis state machine.

Stop and switch to path-agreement properties only when the model grows as complex as the code. Stop and rewrite the contract (step 1) after three rounds where every failure was a harness bug.

When to use it

Use for "property test this", "model-based testing", "find bugs in this data structure", or a Hypothesis state machine.

What the SKILL.md covers

  • Not for
  • Setup
  • Procedure
  • Rules with exceptions
  • Failure modes
  • Verification
  • Credit
Author
oaustegard
Pack
oaustegard/claude-skills (30 skills)
Category
Testing, #140 of 306
Alyn Score
66 / 99
Files
8, includes scripts, includes references
License
MIT
Last updated
20 h ago
Works with
Claude Code, Claude, Cursor, Codex, Gemini CLI, Copilot

Building Property Tests compared to similar skills

SkillAlyn ScoreGitHub starsUpdated
Building Property Tests · oaustegard6640320 h ago
QA · Garry Tan80142k9 h ago
Prototype · vinvcn625.9k17 h ago
Test Import Skill · alibaba511.5k10 h ago

Install Building Property Tests

Pick your agent. Every command is ready to paste.

$npx alyn-cli add building-property-tests

Detects your agents (Claude Code, Cursor, Codex and 70+ more) and installs to the right folder. Add -g to install globally.

$npx skills add oaustegard/claude-skills --skill building-property-tests

Or use the open skills CLI directly.

01

Install it

Run the command above, or upload the folder in Claude.

02

Ask naturally

Describe the task. Your agent loads building-property-tests when the request matches its description.

03

Name it if needed

If it does not trigger, say “use the building-property-tests skill”. In Codex, type $building-property-tests.

The skill itself

SKILL.md

FolderGitHub

Building property tests

Not for

Situation Use
Input that crashes, indexes out of bounds or overflows hunting-failing-inputs
Whether one existing check can go red gating
Tests that copy a registry; declaring invariants declaring-invariants
Performance claims benchmarking
Code with no simpler description of correct behaviour than itself example tests, or reasoning-semiformally
Races, async ordering stress test or race detector

Stop and switch to path-agreement properties only when the model grows as complex as the code. Stop and rewrite the contract (step 1) after three rounds where every failure was a harness bug.

Setup

uv pip install --system --break-system-packages hypothesis pytest
uv pip install --system --break-system-packages -e /path/to/target
cd /tmp && python3 -c "import target; print(target.__file__)"   # None = namespace package; reinstall
git -C /path/to/target rev-parse HEAD                             # record the commit

Rust: proptest + proptest-state-machine. JS/TS: fast-check with fc.commands/fc.modelRun.

Procedure

  1. Write one contract per target, citing its source (README.md:23, core.py:1261). Pick targets where the code is complex and a correct model is simple; references/targets.md lists property kinds.

  2. Write the model with plain loops, importing nothing from the target but the subject. List in the harness docstring anything the model shares with production (tables, matrices, tokenizers) as a blind spot.

  3. Write generators from assets/stateful_template.py:

    • small pools (3–4 keys, few values), unequal overwrite values;
    • edge values as their own strategy (empty, n<k, k=n, duplicates, ties, unaligned sizes);
    • the sizes and modes the existing suite does not use — grep its tests first;
    • domains drawn from the registry, never a copied list;
    • event("<class>") on every interaction a property needs;
    • rare conjunctions built inside one rule.
  4. Get it green on code you believe correct, then check coverage:

    python3 scripts/hypothesis_stats.py --min <class>=5 -- pytest -q test_props.py
    

    Exit 1 names classes below floor or never fired. Fix the generator; do not raise max_examples.

  5. Plant bugs. Own code: python3 /mnt/skills/user/gating/scripts/mutate.py --target src/x.py -- pytest -q test_props.py. Installed library: a MUTANT=<name> monkeypatch branch (dropped invalidation, off-by-one, missing guard). Count a mutant as caught only on AssertionError. A survivor means generation, propagation or assertion is missing — fix and rerun.

  6. Triage each failure. Reproduce it in a plain script outside the harness, then read both subject and model.

    Class Action
    Violates the written contract regression test, then fix
    No document pins the expectation xfail(strict=True) test; ask the owner
    Documented limitation note it
    Harness bug fix the harness

    Vary each suspected condition alone, probe sibling paths with the same invariant, then put the finding behind SKIP_KNOWN and keep searching.

  7. Write one deterministic regression per finding (plain pytest, named after the contract). Run it on the unmodified commit and read the failure text: it must fail for the stated reason.

  8. Fix in a worktree and run all four:

    git worktree add ../target-fix -b fixes
    PYTHONPATH=../target-fix pytest -q test_regressions.py          # pass
    PYTHONPATH=../target-fix SKIP_KNOWN=0 pytest -q test_props.py   # pass — catches fixes the regression accepts
    PYTHONPATH=../target-fix pytest -q ../target-fix/tests          # pass
    PYTHONPATH=/path/to/target pytest -q test_regressions.py        # still fails
    

    A fix that changes persisted output (stored codes, file layout, hashes) goes to the owner as a decision.

  9. Declare the two or three properties the codebase depends on as invariant: tests with a refuted: line, following declaring-invariants, then run its claims.py and totality_lint.py.

  10. Place artifacts. Target repo: one regression file mapping each test to its finding, the fixes, a changelog entry. Discovery harness, mutant results and event counts: an experiments repo, with a script that reruns the regressions on old and new commits.

For several independent areas, run steps 1–7 in parallel agents: references/delegating.md.

Rules with exceptions

Rule Exception
The model imports nothing from the target a constant table checked separately against a published anchor; list it as a blind spot
Compare paths with a tolerance where docs say "byte-identical" or "same bits", use exact equality
Fix every confirmed finding not when the fix changes persisted output

Failure modes

Signal Action
Green on the first full run run steps 4–5 before believing it
A mutant "caught" by an exception from the mutant itself require AssertionError; guard the mutant
Fails on some runs, not others build the conjunction in one rule, or sweep fixed seeds; record the detection rate
Fails only at large float magnitudes if two production paths agree, it is float32 rounding against your float64 model; scale the tolerance
Shrunk case doesn't reproduce it needed an env switch (SKIP_KNOWN=0, MUTANT); record the env with every case
FailedHealthCheck filter_too_much derive the value with st.data()/flatmap instead of filtering
Changes have no effect stale editable install; print target.__file__ in the session

Verification

pytest -q test_regressions.py                          # old commit: all fail
PYTHONPATH=<fixed> pytest -q test_regressions.py       # fix: all pass
PYTHONPATH=<fixed> SKIP_KNOWN=0 pytest -q test_props.py
python3 scripts/hypothesis_stats.py --min <class>=<floor> -- pytest -q test_props.py
MUTANT=<each> pytest -q test_props.py                  # each fails by AssertionError

Bad success: a regression that passes on the old commit; a fix that silently changes persisted output; a green suite whose relied-on events never fired.

Credit

Adapted from Inanna Malick's proptest-praxis (github.com/inanna-malick/agent-skills, MIT, Copyright (c) 2026 Inanna Malick) and her post "Property Testing with Agent Swarms" (recursion.wtf).

Tips for Building Property Tests

  • • Start a fresh session after installing so your agent picks up the new skill.
  • • This skill ships scripts. Read them before the first run, and run it in a project without secrets first.
  • • It bundles reference files. The agent reads them only when needed, so they cost no context until then.
  • • It is part of a pack with 30 skills. Install only the ones you need with --skill to keep triggers precise.
  • • If two skills overlap, mention the one you want by name.

Questions about Building Property Tests

What does the Building Property Tests skill do?

Finds correctness bugs by checking code against a simple reference model over generated inputs and operation sequences, then shrinks each failure to a regression test and a fix. Use for "property test this", "model-based testing", "find bugs in this data structure", or a Hypothesis state machine.

How do I install Building Property Tests?

Run "npx alyn-cli add building-property-tests" in your project. The Alyn CLI installs it for Claude Code, Cursor, Codex and 70+ other agents. In Claude on the web, upload the skill folder as a ZIP under Customize > Skills.

Does Building Property Tests work in Cursor and Codex?

Yes. Building Property Tests uses the open SKILL.md format, so it works in Claude Code, Claude, Cursor, OpenAI Codex, Gemini CLI, GitHub Copilot and other agents that support agent skills.

When should I use the Building Property Tests skill?

Use for "property test this", "model-based testing", "find bugs in this data structure", or a Hypothesis state machine.

What is inside the Building Property Tests skill?

The SKILL.md covers Not for, Setup, Procedure, Rules with exceptions, Failure modes, Verification, and the skill folder has 8 files including scripts.

What is a good alternative to Building Property Tests?

QA by Garry Tan is the highest-ranked similar skill from another creator, with an Alyn Score of 80 compared to 66 for Building Property Tests.

Is Building Property Tests free?

Building Property Tests is open source on GitHub under the MIT license. It is free to install; check the license before commercial use.

Who made Building Property Tests?

Building Property Tests is published by oaustegard in the oaustegard/claude-skills repository, together with 29 other skills.

Similar skills. From other creators.