rasuvaeff/property-testing

Property-based testing plugin for the Testo testing framework (PHP 8.3+). Generate random inputs, find falsifying cases, shrink them to a minimal counterexample.

Install

composer require --dev rasuvaeff/property-testing

Runtime deps: ext-mbstring, ext-random, testo/testo ^0.10.25 || ^1.0.

Rules

• Attribute args are constant expressions in PHP, so generators CANNOT be passed inline to #[Property]. Always name a method returning array<string, ArbitraryInterface> keyed by parameter name.
• Declare generators/examples methods public static (public if the body needs $this). They are invoked via reflection only, so Rector's dead-code set deletes private ones (RemoveUnusedPrivateMethodRector); public is safe and never becomes a test (non-void return).
• The plugin self-registers via #[FallbackInterceptor] on the #[Property] attribute — no registration in testo.php is needed.
• Random uses an object-scoped MT19937 engine (\Random\Randomizer), independent of PHP's global mt_rand state, so seeded runs are reproducible regardless of other random calls in the process.
• Do NOT use generated values for cryptography (MT19937 is a PRNG, not a CSPRNG).
• A generator NEVER yields an out-of-domain value: Gen::filter() and sized collections throw GenerationExhausted when they cannot satisfy their predicate/minimum within their attempt budget. Construct dependent values with Gen::flatMap()/Gen::draw() instead of filtering broadly.
• Property::$runs means successful checks. Assume::that(false) discards an attempt without consuming a run. Retries stop at maxDiscards (default runs * 10) and fail with a structured GaveUpException.
• Shrinking is integrated (since 2.0): generate(Random) returns a Shrinkable (value + lazy tree of smaller candidates). There is NO shrink(mixed) method. Greedy per-parameter descent; best-effort minimal, not provably minimal.
• For value spaces Gen does not cover, implement ArbitraryInterface directly: generate(Random) draws via the injected Random and returns a Shrinkable built with Shrinkable::of($value, $lazyChildren) / Shrinkable::leaf($value). Order candidates most aggressive first, keep every branch finite, and never yield a candidate equal to its parent value.

Public API

Attribute: #[Property]

use Rasuvaeff\PropertyTesting\Property;

#[Property(runs: 100, seed: null, generators: null, maxShrinks: null, examples: null, maxDiscards: null, timeoutMs: null, budgetMs: null)]

• runs (int, default 100): number of successful random inputs to check. Must be >= 1.
• seed (?int, default null): fixed seed for reproducibility. Omit for a random seed (reported in the failure message).
• generators (?string, default null): method name returning array<string, ArbitraryInterface>. Defaults to <testMethod>Generators.
• maxShrinks (?int, default null): cap on accepted shrink steps. Null = no cap. 0 = disable shrinking (report the original counterexample). Must be >= 0.
• examples (?string, default null): method name returning iterable<array<mixed>> of fixed positional argument tuples, each run BEFORE the random inputs and NOT shrunk. Defaults to <testMethod>Examples when that method exists. A failing example -> ExampleViolationException.
• maxDiscards (?int, default null): discard budget; null resolves to runs * 10. Exceeding it -> GaveUpException. Must be >= 0.
• timeoutMs (?int, default null): wall-clock deadline for a SINGLE run (random or example). Overrun -> DeadlineExceededException naming the input (NOT shrunk — timing noise makes timed shrinking non-deterministic). Measured after the run returns; a hung body cannot be interrupted. An assertion failure in the same run wins. Must be >= 1.
• budgetMs (?int, default null): wall-clock budget for the whole random phase. Overrun before runs checks complete -> TimeBudgetExceededException with completed/required counts. Must be >= 1.

Env: PROPERTY_RUNS (override run count), PROPERTY_SEED (seed when the attribute omits it), PROPERTY_VERBOSE (log each run's args), PROPERTY_DB (directory enabling the regression corpus, below).

Regression corpus (PROPERTY_DB=<dir>): every falsified property records its failure; recorded failures replay BEFORE the random phase (unless the attribute pins a seed). A still-failing entry is reported at once; one that passes or is discarded via Assume::that() is pruned. Two entry kinds:

• values — the minimised input stored as data (null/scalars/arrays/enum cases/ byte strings). Replays as ONE run, reported as RegressionViolationException (getArguments(), getSeed()). Survives a shifted generation sequence; dropped when the property's parameter names no longer match.
• seed — the fallback when a value is not representable as data (objects, closures, in-body Gen::draw() values). Re-runs the whole random phase with that seed, reported as PropertyViolationException. Dropped when the package's generation-sequence epoch changes. One JSON file per property (<sha1(id)>.json), max 8 values + 2 seed entries, oldest evicted. Gitignore the directory.

The runner generates runs random argument sets, invokes the test through Testo's pipeline, and on the first failure shrinks the counterexample to a minimal one. Failure carries a PropertyViolationException whose message shows the seed, the original counterexample, and the shrunk one.

Static facade: Gen

use Rasuvaeff\PropertyTesting\Gen;

Gen::int()                              // IntArbitrary, PHP_INT_MIN..PHP_INT_MAX, shrinks to 0
Gen::intBetween(int $min, int $max)     // IntArbitrary, [$min, $max], shrinks to 0 (clamped)
Gen::intPositive()                      // IntArbitrary, 1..PHP_INT_MAX
Gen::float()                            // FloatArbitrary, [0.0, 1.0), shrinks to 0.0
Gen::floatBetween(float $min, float $max)     // FloatArbitrary, [$min, $max], shrinks to 0.0 (clamped)
Gen::bool()                             // BoolArbitrary, true -> false on shrink
Gen::string()                           // StringArbitrary, Unicode, len 0..100, shrinks to '' then chars->a
Gen::stringAscii()                      // StringArbitrary, printable ASCII, len 0..100
Gen::stringOf(int $minLen, int $maxLen) // StringArbitrary, Unicode, bounded length
Gen::stringFrom(string $alphabet, int $minLen = 0, int $maxLen = 100) // CharsetStringArbitrary: chars from a fixed alphabet (multibyte OK); shrinks to '' then chars toward alphabet[0]
Gen::bytes(int $minLen = 0, int $maxLen = 100) // BytesArbitrary: raw bytes 0..255; shrinks to '' then bytes toward "\x00"
Gen::arrayOf(ArbitraryInterface $el, int $min = 0, int $max = 100)    // ArrayArbitrary, lists
Gen::nonEmptyArrayOf(ArbitraryInterface $el, int $max = 100)  // ArrayArbitrary, size >= 1
Gen::uniqueArrayOf(ArbitraryInterface $el, int $min = 0, int $max = 100) // UniqueArrayArbitrary: pairwise-distinct lists; may settle for fewer than drawn when the element space runs dry (never below min — throws)
Gen::dictOf(ArbitraryInterface $key, ArbitraryInterface $value, int $min = 0, int $max = 100) // DictionaryArbitrary, distinct keys int|string; may settle for fewer than drawn when the key space runs dry (never below min — throws GenerationExhausted)
Gen::record(array $shape)               // RecordArbitrary, fixed-shape map ['field' => ArbitraryInterface, ...]
Gen::oneOf(mixed ...$values)           // OneOfArbitrary, one value; shrinks toward EARLIER-listed distinct values (put simpler first)
Gen::elements(array $values)           // OneOfArbitrary, one value from an array (array form of oneOf)
Gen::enum(SomeEnum::class)             // OneOfArbitrary over enum cases; shrinks toward earlier-DECLARED cases
Gen::constant(mixed $value)            // ConstantArbitrary, always $value; does NOT shrink
Gen::char()                            // StringArbitrary, one printable ASCII char
Gen::uuid()                            // UuidArbitrary, RFC 4122 v4 UUID strings; does NOT shrink
Gen::datetime(?DateTimeImmutable $min, ?DateTimeImmutable $max)  // DateTimeArbitrary, UTC, shrinks toward epoch
Gen::floatSpecial()                    // OneOfArbitrary over NAN, INF, -INF, -0.0, PHP_FLOAT_EPSILON/MIN/MAX (opt-in; float()/floatBetween() stay finite)
Gen::intRange(int $min, int $max)      // FlatMappedArbitrary: ordered pairs [lo, hi], min <= lo <= hi <= max; both bounds shrink
Gen::recursive(ArbitraryInterface $leaf, Closure $wrap, int $maxDepth = 3) // bounded recursion: $wrap(ArbitraryInterface): ArbitraryInterface lifts the previous level; 50/50 leaf-vs-branch at each level
Gen::nullable(ArbitraryInterface $inner)      // NullableArbitrary, null or inner value
Gen::map(ArbitraryInterface $inner, Closure $fn)   // MappedArbitrary; shrinks through the inner tree, re-applying $fn (pure!)
Gen::flatMap(ArbitraryInterface $inner, Closure $fn)  // FlatMappedArbitrary; $fn(mixed): ArbitraryInterface — dependent generators; shrinks source (dependent regenerated from seed) then dependent value
Gen::filter(ArbitraryInterface $inner, Closure $predicate) // FilteredArbitrary, max 100 retries then throws GenerationExhausted (never yields an out-of-domain value); tree pruned to predicate-satisfying candidates
Gen::tuple(ArbitraryInterface ...$elements)        // TupleArbitrary, fixed-arity tuple (list), shrinks each position
Gen::frequency(iterable $pairs)                    // FrequencyArbitrary, weighted [int $weight, ArbitraryInterface] pairs; shrinks within the generating branch
Gen::commands(mixed $initialModel, array $commandGenerators, int $minLen = 0, int $maxLen = 100) // CommandSequenceArbitrary for stateful/model-based testing; $commandGenerators = list<ArbitraryInterface> each producing a Command; a sequence shorter than $minLen (no applicable command) throws GenerationExhausted; see "Stateful" below
Gen::ipv4()                            // dotted-quad IPv4 strings; each octet shrinks to 0
Gen::email()                           // local@label.tld addresses; shrinks to shortest local/label + first TLD
Gen::url()                             // http(s)://host.tld[/path]; shrinks to http://a.com
Gen::json(int $maxDepth = 3)           // JSON-encodable value (null/bool/int/float/string/list/object)
Gen::jsonString(int $maxDepth = 3)     // json_encode() text of Gen::json()
Gen::regex(string $pattern, int $maxRepeat = 8)          // strings matching a regex SUBSET, compiled to combinators (shrinks through them)
Gen::stringMatching(string $pattern, int $maxRepeat = 8) // alias of regex()
// regex subset: literals, ., [..] classes (ranges, negation, \d\w\s + negations), \d\w\s\D\W\S\t\n\r, quantifiers * + ? {n} {n,} {n,m}, alternation |, groups (..)/(?:..). A single leading ^ / trailing $ is a no-op. Anchors elsewhere, backreferences, lookaround, named/inline groups, flags -> throw naming the construct.
Gen::draw(ArbitraryInterface $arb)     // mixed: in-body dependent draw — ONLY inside a property body (throws elsewhere); see "In-body draws" below
Gen::sample(ArbitraryInterface $arb, int $count = 10, int $seed = 0)  // list<mixed>: eager debug helper, returns plain values not an arbitrary
Gen::sampleShrinks(ArbitraryInterface $arb, int $seed = 0, int $limit = 10) // array{value: mixed, shrinks: list<mixed>}: one value + its first shrink candidates (debug custom arbitraries)

Dependent generators — a list plus a valid index (no discarded runs):

Gen::flatMap(
    Gen::nonEmptyArrayOf(Gen::int()),
    static fn(array $items): ArbitraryInterface => Gen::tuple(
        Gen::constant($items),
        Gen::intBetween(0, count($items) - 1),
    ),
);

In-body draws: Gen::draw()

For several dependent values where nested flatMap gets awkward — the domain may depend on parameters, previous draws, or intermediate results:

#[Property(runs: 200)]
public function sliceIsContained(array $xs): void
{
    $from = Gen::draw(Gen::intBetween(0, count($xs)));
    $to = Gen::draw(Gen::intBetween($from, count($xs))); // depends on $from
    // ... assertions ...
}

• Drawn values shrink together with the parameters (replay tape: each draw is recorded and shrunk through its own tree; the body is re-run with the tape replayed by position).
• Counterexamples report draws as draw#1, draw#2, ... next to the named parameters; PROPERTY_VERBOSE logs them per run.
• A replayed draw is NOT re-validated against the new arbitrary when a shrunk parameter changes control flow (fast-check gen() model) — assert what the body requires, do not rely on the range after shrinking.
• With draws present, accepted shrink steps are capped at 1000 (maxShrinks wins when set) to guarantee termination.
• Prefer flatMap for a single dependent value.

Shrink trees: Shrinkable

use Rasuvaeff\PropertyTesting\Shrinkable;

Shrinkable::leaf(mixed $value): Shrinkable                    // terminal node, no candidates
Shrinkable::of(mixed $value, Closure(): iterable<Shrinkable> $shrinks): Shrinkable  // lazy candidates, aggressive first
$shrinkable->value                                            // the generated value
$shrinkable->shrinks(): iterable<Shrinkable>                  // smaller candidates, each with its own subtree
$shrinkable->map(Closure(mixed): mixed $fn): Shrinkable       // transform the whole tree lazily

Numeric generators are boundary-biased: ~1 draw in 5 returns an in-range edge value (0/±1/min/max for ints, 0.0/min for floats) instead of a uniform one. Shrinking is unaffected.

Distribution: Classify (static, @api)

use Rasuvaeff\PropertyTesting\Classify;

Classify::label(string $label): void              // tally $label for the current run
Classify::when(bool $condition, string $label): void  // tally only when $condition
Classify::cover(bool $condition, string $label, float $minPercent): void
// like when(), but REQUIRES the label in >= $minPercent of passing runs;
// otherwise the property FAILS with CoverageViolationException even though
// every run passed. Discarded runs are excluded from the denominator.

Call inside a property body. After a fully passing property the runner prints the share of runs that hit each label (a label counts once per run). Confirms a property is not passing vacuously; cover() turns that into a hard CI gate.

Environment overrides

• PROPERTY_RUNS (positive int): overrides every property's run count.
• PROPERTY_SEED (int): seed for any property whose attribute omits seed; an explicit attribute seed still wins.
• PROPERTY_VERBOSE (any value except ''/'0'): logs every run's generated arguments to stdout, plus one line per ACCEPTED shrink step on failure (shrink step 3: x=63 -> 51) — inspect what a replayed seed feeds the test and how the shrinker descends.

Discarding runs: Assume::that(bool $condition): void

use Rasuvaeff\PropertyTesting\Assume;

Assume::that($cap >= $baseSeconds);

Throws AssumptionSkipped when false; the runner treats the attempt as discarded (neither failure nor success) and retries until runs successful checks finish. Warns when >90% of attempts are discarded. Exceeding maxDiscards fails with a GaveUpException exposing required/successful/discarded/attempt counts.

Inspecting failures

The failing TestResult carries a Rasuvaeff\PropertyTesting\PropertyViolationException with getCounterExample(): CounterExample exposing: seed, runsBeforeFailure, originalArguments, shrunkArguments, shrinkSteps, failure, skips, shrinkTrials (candidates tried, accepted + rejected). The exception message includes a Changed: line diffing original vs shrunk arguments (unchanged ones omitted; a draw dropped by tape truncation renders as (absent)). CounterExample::toArray() / toJson() return normalized machine-readable data; toExamplesCode(string $method = 'propertyExamples') emits runnable PHP for scalar/array/enum arguments and rejects unsupported objects. The Testo fallback interceptor is PropertyInterceptor.

Stateful / model-based testing

Test sequences of operations against a simplified model; the failing sequence is shrunk by dropping and simplifying steps. Namespace: Rasuvaeff\PropertyTesting\StateMachine.

Implement Command extends \Stringable:

interface Command extends \Stringable {
    public function preCondition(mixed $model): bool;   // may run in this model state? (gates generation + replay-skip)
    public function nextState(mixed $model): mixed;      // pure model transition; returns NEW model
    public function run(mixed $model, mixed $system): mixed;   // execute against the SUT; return observed result
    public function postCondition(mixed $model, mixed $result): bool;  // check result vs PRE-state model; false/throw = falsify
    public function __toString(): string;                // label for the counterexample trace
}

• Gen::commands($initialModel, $commandGenerators, $minLen = 0, $maxLen = 100) builds valid-by-construction sequences: each step appends a command whose preCondition holds in the running model, then advances via nextState. Value is a CommandSequence (\Stringable; ->initialModel, ->commands).
• StateMachine::check(CommandSequence $sequence, Closure $system): void — call in the property body. $system is a factory returning a FRESH system per run. It replays each command (skipping any whose precondition a shrink invalidated), asserts each postCondition, and throws PostconditionViolation on failure.
• Shrinking drops command blocks (down to single commands, isolating a middle step) then simplifies each command's parameters through its own tree.

#[Property(runs: 200)]
public function stackMatchesModel(CommandSequence $sequence): void {
    StateMachine::check($sequence, static fn(): Stack => new Stack());
}
/** @return array<string, ArbitraryInterface> */
public static function stackMatchesModelGenerators(): array {
    return ['sequence' => Gen::commands([], [
        Gen::map(Gen::intBetween(0, 99), static fn(int $v): Command => new Push($v)),
        Gen::constant(new Pop()),
    ])];
}

Example

use Rasuvaeff\PropertyTesting\Gen;
use Rasuvaeff\PropertyTesting\Property;
use Testo\Assert;
use Testo\Test;

#[Test]
final class SortPropertyTest
{
    #[Property(runs: 200)]
    public function sortedThenSortedIsIdempotent(array $xs): void
    {
        $once = $this->sort($xs);
        $twice = $this->sort($once);

        Assert::same($twice, $once);
    }

    /** @return array<string, \Rasuvaeff\PropertyTesting\ArbitraryInterface> */
    public static function sortedThenSortedIsIdempotentGenerators(): array
    {
        return ['xs' => Gen::arrayOf(Gen::intBetween(-100, 100))];
    }

    private function sort(array $xs): array
    {
        $copy = $xs;
        sort($copy);

        return $copy;
    }
}