hyperfine
A command-line benchmarking tool (inspired by bench).
Demo: Benchmarking fd and find:
Features
- Statistical analysis across multiple runs
- Support for arbitrary shell commands
- Constant feedback about the benchmark progress and current estimates
- Warmup runs can be executed before the actual benchmark
Usage
Basic benchmark
To run a benchmark, you can simply call hyperfine <command>... where the argument(s) can be any shell command. For example:
> hyperfine 'sleep 0.3'
Hyperfine will automatically determine the number of runs to perform for each command. By default, it will perform at least 10 benchmarking runs. To change this, you can use the -m/--min-runs option:
> hyperfine --min-runs 5 'sleep 0.2' 'sleep 3.2'
I/O-heavy programs
If the program execution time is limited by disk I/O, the benchmarking results can be heavily influence by disk caches and whether they are cold or warm.
If you want to run the benchmark on a warm cache, you can use the -w/--warmup option to perform a certain amount of program executions before the actual benchmark:
> hyperfine --warmup 3 'grep -R TODO *'
Conversely, if you want to run the benchmark for a cold cache, you can use the -S/--setup option to run a special command before each benchmarking run. For example, to clear disk caches on Linux, you can run
sync; echo 3 | sudo tee /proc/sys/vm/drop_caches
To use this with Hyperfine, call sudo echo to temporarily gain sudo permissions and then call:
hyperfine -S 'sync; echo 3 | sudo tee /proc/sys/vm/drop_caches' 'grep -R TODO *'
Installation
Hyperfine can be installed via cargo:
cargo install hyperfine
Arch Linux
On Arch Linux, hyperfine can be installed from the AUR:
yaourt -S hyperfine
Origin of the name
The name hyperfine was chosen in reference to the hyperfine levels of caesium 133 which play a crucial role in the definition of our base unit of time — the second.