Std.Bench

Std.Bench is Riot’s benchmark-writing surface. It mirrors the spirit of Std.Test: small building blocks, consistent runner wiring, and output that can feed both humans and tools.

The smallest useful benchmark

open Std

let benchmarks = Bench.[
  case "vector push" (fun () ->
    let v = Vector.create () in
    Vector.push v 42);
]

let () =
  Actors.run
    ~args:Env.args ()
    ~main:(fun ~args ->
      Bench.Cli.main
        ~name:"vector"
        ~benchmarks
        ~args)

This is the benchmark equivalent of a small Std.Test suite:

• define benchmark cases
• hand them to the shared CLI entrypoint
• let riot bench discover and run them later

The core constructors

The main constructors are:

• Bench.case
• Bench.skip
• Bench.with_config
• Bench.compare
• Bench.compare_with_config
• Bench.make_case
• Bench.make_case_with_config

Use case when you want one named benchmark:

Bench.case "hashmap insert" (fun () ->
  let map = HashMap.create () in
  HashMap.insert map "key" "value")

Use with_config when you need to override warmup and iteration counts:

Bench.with_config
  ~config:{ iterations = 1000; warmup = 50 }
  "fast operation"
  (fun () ->
    ignore (1 + 1))

What the config means

Each benchmark case uses a bench_config:

type bench_config = {
  iterations: int;
  warmup: int;
}

• warmup is how many runs happen before timing starts
• iterations is how many measured runs are used for statistics

The defaults are intentionally modest:

• 100 measured iterations
• 10 warmup iterations

Those defaults are good enough for many package-level comparisons, but performance-critical cases often deserve a custom config.

Comparison benchmarks

Comparisons are one of the most useful parts of Std.Bench.

Instead of asking “how long did this run take in isolation?”, they let you ask:

• which implementation is faster?
• how much faster?
• which case is the baseline winner?

Example:

open Std

let benchmarks = Bench.[
  compare "insert 10k items" [
    make_case "HashMap" (fun () ->
      let map = HashMap.create () in
      for i = 0 to 9_999 do
        HashMap.insert map (Int.to_string i) i
      done);
    make_case "Swisstable" (fun () ->
      let map = Swisstable.create () in
      for i = 0 to 9_999 do
        Swisstable.insert map (Int.to_string i) i
      done);
  ];
]

Comparison output reports:

• per-case statistics
• the fastest case
• relative speed ratios

That makes the results much more actionable than a pile of isolated timings.

Writing good benchmarks

Good Riot benchmarks usually follow a few rules:

• benchmark one thing at a time
• keep setup cost intentional
• avoid measuring unrelated allocations or I/O unless that is the point
• use meaningful case names
• compare real alternatives, not toy ones

If a benchmark does not help a future reader choose or validate something, it probably needs tighter design.

Custom runners

For more control, Bench.Runner.run_benchmarks is available directly:

open Std

let () =
  Actors.run
    ~args:Env.args ()
    ~main:(fun ~args:_ ->
      let config = Bench.Runner.{
        reporter = (module Bench.Reporter.Default);
        suite_info = { name = "My Benchmarks" };
      } in
      let _summary = Bench.Runner.run_benchmarks ~config benchmarks in
      Ok ())

This is useful when:

• you want a custom binary shape
• you need direct access to the summary
• you want to control the reporter explicitly

But for most packages, Bench.Cli.main is the right default.