* docs: document JSON schema v2 breaking changes
* docs: simplify JSON example introduction
* Update `--export-json` to schema version 2
Updated the JSON format for `--export-json` to schema version 2, which now includes metadata, per-run measurements, and statistical summaries. Consumers must update their code to accommodate the new structure.
* Revise breaking changes in CHANGELOG
Updated breaking changes section to clarify command execution and JSON export format.
Add a --setup option for the use-case of wanting to do one-off setup
before a set of benchmarks, not the once-per-test setup that --prepare
does. This is useful for the cases noted in the updated documentation.
Per the feedback on https://github.com/sharkdp/hyperfine/pull/448 this
new "--setup" option will steal the "-s" short option from
"--style" (initially this was called "--build" and used the
non-conflicting "-b").
Potential future work:
I'd prefer if this and --cleanup took N number of commands, so you
could do e.g.:
hyperfine -L rev master,next \
-b 'git worktree add /tmp/test-{r}' \
-b 'make -C /tmp/test-{r} all' \
'make -C /tmp/test-{r} test' \
-c 'git worktree remove /tmp/test-{r}'
I.e. a shortcut around not providing these with &&, which makes things
a bit more readable.
But the --cleanup option doesn't do that, so let's just go with what
it's doing for consistency, so for this you'll now need to do:
hyperfine -L rev master,next \
-b 'git worktree add /tmp/test-{r} &&
make -C /tmp/test-{r} all' \
'make -C /tmp/test-{r} test' \
-c 'git worktree remove /tmp/test-{r}'
Shows an error message if the argument to options like '--warmup <N>'
can not be parsed:
> hyperfine --warmup xxx "sleep 1"
Error: Could not read numeric argument to '--warmup': invalid digit found in string
closes#337
David Peter