Property Testing with Propane

Propane is Riot’s property-testing library.

It integrates directly with Std.Test by returning ordinary Std.Test.test_cases, which means you do not have to choose between:

writing tests in the Riot test runner
using generators and shrinking

You get both.

The simplest property

Open Std and Propane, then define a property directly:

open Std
open Propane

let tests = [
  property "reversing twice yields the original list"
    Arbitrary.(list int)
    (fun xs -> List.reverse (List.reverse xs) = xs)
]

let () =
  Actors.run
    ~args:Env.args ()
    ~main:(fun ~args ->
      Test.Cli.main
        ~name:"list laws"
        ~tests
        ~args)

A few important things are happening here:

property returns a Std.Test.test_case
Arbitrary.(list int) tells Propane how to generate, shrink, and print values
the suite still runs through Test.Cli.main, so riot test can discover it normally

That integration point is the reason Propane belongs in the Test Runner docs, not in some isolated “advanced library” corner.

Why property testing is useful

Example-based tests are good when you know the exact inputs that matter.

Property tests are better when what you really care about is a rule:

a round-trip law
an identity law
a commutativity or associativity law
an invariant that should hold across many inputs

Instead of hand-writing twenty examples, you define the rule once and let the generator produce many cases.

Arbitraries are the center of gravity

Propane uses Arbitrary.t to describe values completely:

how to generate them
how to shrink them
how to print them
optionally how to measure their size

The built-in arbitraries already cover a lot:

Arbitrary.int
Arbitrary.bool
Arbitrary.float
Arbitrary.string
Arbitrary.list
Arbitrary.array
Arbitrary.vector
Arbitrary.hashmap
Arbitrary.option
Arbitrary.result
Arbitrary.pair
Arbitrary.triple

That means many useful properties are only a couple of lines away.

More realistic examples

Round-trip law:

open Std
open Propane

let tests = [
  property "string concatenation preserves length"
    Arbitrary.(pair string string)
    (fun (left, right) ->
      String.length (left ^ right) = String.length left + String.length right);
]

Collection behavior:

open Std
open Propane

let tests = [
  property "hashmap get after insert returns the inserted value"
    Arbitrary.(triple string int (hashmap string int))
    (fun (key, value, map) ->
      Collections.HashMap.insert map key value |> ignore;
      Collections.HashMap.get map key = Some value);
]

These are exactly the kinds of tests that are annoying to write as many hand-authored examples but very natural as properties.

Preconditions with `assume`

Not every generated input is valid for every rule. Propane gives you assume and implies so you can express preconditions.

Example:

open Std
open Propane

let tests = [
  property "division and modulo satisfy the usual relation"
    Arbitrary.(pair int int)
    (fun (a, b) ->
      assume (b != 0);
      (a / b) * b + (a mod b) = a);
]

This says:

generate pairs of ints
discard the meaningless cases where b = 0
test the invariant on the remaining cases

You can also write the same idea with implication:

open Std
open Propane

let tests = [
  property "division law"
    Arbitrary.(pair int int)
    (fun (a, b) ->
      implies (b != 0) ((a / b) * b + (a mod b) = a));
]

Use assume when you want to say “skip this sample entirely if the precondition does not hold.” Use implies when the property reads more clearly as a conditional statement.

Custom failures

Sometimes false is not enough. You want a meaningful reason in the failure output.

That is what fail is for:

open Std
open Propane

let tests = [
  property "rendered output stays under the limit"
    Arbitrary.string
    (fun s ->
      let rendered = my_render s in
      if String.length rendered > 1024 then
        fail "rendered output exceeded 1024 bytes"
      else
        true);
]

This is especially useful when the failure depends on domain semantics rather than only the raw value.

Shrinking is why Propane is worth using

A property test that only produces huge failing inputs is not that useful.

Propane shrinks failing inputs toward a smaller counter-example. That is the real ergonomic win. A good shrinker turns:

“something somewhere failed on a giant input”

into:

“here is the smallest input that still breaks the property”

That difference is the reason property testing becomes practical instead of noisy.

Writing custom arbitraries

Once the built-in arbitraries stop being enough, create your own.

Example:

open Std
open Propane

type color =
  | Red
  | Green
  | Blue

let color_arb =
  Arbitrary.make
    ~print:(function
      | Red -> "Red"
      | Green -> "Green"
      | Blue -> "Blue")
    Generator.(one_of [ return Red; return Green; return Blue ])

let tests = [
  property "color printer never returns an empty string"
    color_arb
    (fun color -> String.length (match color with Red -> "Red" | Green -> "Green" | Blue -> "Blue") > 0);
]

For more control, custom arbitraries can also provide:

a shrinker
a size metric

That is where domain-specific property testing gets really good.

Propane versus `Std.Test.property`

This distinction matters.

Std.Test.property lets you mark a test as a property-style case with an example count:

Test.property "some repeated test" ~examples:1000 (fun _ctx -> Ok ())

That is useful when you are controlling the repeated execution or sampling logic.

Propane.property is different:

it generates values for you
it shrinks failures
it prints counter-examples
it still returns a Std.Test.test_case

If you want genuine property-based testing, use Propane.

If you only want a property-flavored label or custom repeated sampling inside the callback, Std.Test.property may still be enough.

Where Propane fits in the overall test stack

Use:

Std.Test.case for straightforward example-based tests
Propane.property for invariants across many generated inputs
Std.Test.FixtureRunner for file-backed test families
Std.Test.Snapshot when the output artifact itself is what you want to review

These are not competing systems. They are complementary parts of the same test story.