This otherwise respects all other configuration
(NO_COLOR, TERM=dumb, &c.), but it's very common for the environment to
be lacking TERM, which in that case usually means a pro-verbial
TERM=cmd.exe, which is colourful
Fixes: 049d1138c9 ("Don't output colours
by default if TERM=dumb or NO_COLOR=1")
This adds a new "raw executor" (next to the "shell executor") that
allows hyperfine to execute commands directly without any intermediate
shell.
The command line is split into tokens (using the `shell-words` crate),
and according to POSIX rules. The first token is taken as the executable,
and the rest as arguments.
The new executor is enabled by default. In order to select the shell
executor, users will have to pass `--shell=default`.
This allows us to reduce measurement noise and to benchmark very quick
commands. It also decreases the time to run benchmarks, as we don't need
the calibration phase.
Also, it allows one to make sure that the executed command is not
implemented as a shell builtin. For example `hyperfine true`
and `hyperfine --shell=default true` return different times due
to the fact that `bash` executes `true` as a NOP.
Co-authored: Ciprian Dorin Craciun <ciprian@volution.ro>
Add a --setup option for the use-case of wanting to do one-off setup
before a set of benchmarks, not the once-per-test setup that --prepare
does. This is useful for the cases noted in the updated documentation.
Per the feedback on https://github.com/sharkdp/hyperfine/pull/448 this
new "--setup" option will steal the "-s" short option from
"--style" (initially this was called "--build" and used the
non-conflicting "-b").
Potential future work:
I'd prefer if this and --cleanup took N number of commands, so you
could do e.g.:
hyperfine -L rev master,next \
-b 'git worktree add /tmp/test-{r}' \
-b 'make -C /tmp/test-{r} all' \
'make -C /tmp/test-{r} test' \
-c 'git worktree remove /tmp/test-{r}'
I.e. a shortcut around not providing these with &&, which makes things
a bit more readable.
But the --cleanup option doesn't do that, so let's just go with what
it's doing for consistency, so for this you'll now need to do:
hyperfine -L rev master,next \
-b 'git worktree add /tmp/test-{r} &&
make -C /tmp/test-{r} all' \
'make -C /tmp/test-{r} test' \
-c 'git worktree remove /tmp/test-{r}'
Benjamin Rombaut