William Chargin
f6464a2943
Support matrices of independent parameter lists
...
This patch permits the `-L`/`--parameter-list` argument to appear
multiple times. Each additional parameter increases the dimensionality
of the parameter matrix. One benchmark will be run for each combination
of parameters (i.e., the benchmark space is the Cartesian product of the
parameter spaces).
For now, `--parameter-list` and `--parameter-scan` are still mutually
exclusive. If desired, a follow-up change could similarly permit
multiple occurrences of `--parameter-scan` and also permit use of both
listed and scanned parameters together. (After all, `--parameter-scan`
can be thought of as syntactic sugar for a parameter list.)
This implementation is a little profligate with memory usage for
`BenchmarkResult`s: each result contains a separate copy of all
parameter names (as `String`s). This could be done away with by
threading lifetimes through the `BenchmarkResult`s, or by ref-counting
the names.
Because `BenchmarkResult`s now have maps with arbitrary key sets, we can
no longer use `serde` to serialize them to CSV. But a CSV writer is easy
to write by hand, so we just do that.
Fixes #253 .
Test Plan:
Some unit tests included. As an integration test, try:
```
cargo run --release -- --runs 10 \
-L foo foo1,foo2 -L bar bar9,bar8 \
'echo {foo} {bar}' \
'printf "%s\n" {foo} {bar}' \
--export-csv /tmp/out
```
with all the export formats.
wchargin-branch: param-list-matrix
wchargin-source: 11051ab222c9de0fbc6ac6a080fbca48e05996c2
2020-10-13 14:00:46 +02:00
sharkdp
46e7e4152a
Initial support for randomized environment variables
2020-05-25 16:03:58 +02:00
Alain Martin
d5e20cc760
Fix case of "AsciiDoc"
...
This matches how it is written on the README.
2020-05-22 22:43:49 +02:00
Alain Martin
b97ecf01c2
Fix typo seperated -> separated
2020-05-22 22:43:49 +02:00
sharkdp
559e387ffa
Use ASCII spinner for Windows
...
closes #229
2020-01-04 11:42:11 +01:00
sharkdp
8b0dd37962
Fix warning on Windows
2020-01-04 11:12:49 +01:00
Andrii Radyk
9a064b7a8f
remove deprecated Error::description
2019-12-19 21:55:16 +01:00
sharkdp
954b62003f
Change behavior for empty input
2019-10-20 13:45:41 +02:00
sharkdp
3e217f4233
Add simple tokenizer for parameter list
2019-10-20 13:45:41 +02:00
sharkdp
1de39a325d
--parameter-list also conflicts with --parameter-step-size
2019-10-20 13:45:41 +02:00
sharkdp
bd0a98d5b8
Update --help text
2019-10-20 13:45:41 +02:00
sharkdp
d7dc232e2f
Use 'enum' to represent numeric/textual parameter values
2019-10-20 13:45:41 +02:00
Hasan Ibraheem
4a5d08ddc2
Implement basic --parameter-list with naive list parsing
2019-10-20 13:45:41 +02:00
Hasan Ibraheem
538691e999
Use String instead of NumericType for paramater in Command
2019-10-20 13:45:41 +02:00
knidarkness
b3084cbb58
update output condition check to use .map
2019-10-15 17:49:48 +02:00
knidarkness
34eb150fef
updates as requested in the review
2019-10-15 17:49:48 +02:00
knidarkness
5bc94416ad
Fixed error with borrowing
2019-10-15 17:49:48 +02:00
knidarkness
08ad79a729
Removed output for warnings
2019-10-15 17:49:48 +02:00
knidarkness
5bdfdbf1cb
Added 'none' as a possible value for "style" option
...
This commit allows to remove all non-errornous output of the tool.
2019-10-15 17:49:48 +02:00
Stefan Seemayer
6a8dd1962c
Also run prepare command during warmup
...
Fixes #182
2019-10-14 22:12:31 +02:00
sharkdp
94dade7cf0
Improve --help text
2019-10-13 16:24:57 +02:00
sharkdp
4cb86f4b9c
Do not show relative stddev for fastest entry
2019-10-13 16:24:51 +02:00
sharkdp
df2e6716d7
Add relative speed to markdown export, closes #225
2019-10-13 16:24:51 +02:00
sharkdp
92e59c37c1
Use 'if' instead of 'match'
2019-10-13 15:11:39 +02:00
sharkdp
ea2d622cc1
Remove space
2019-10-13 15:11:39 +02:00
sharkdp
33591d1433
Update error message
2019-10-13 15:11:39 +02:00
Saurav Sharma
d1cd30b2a3
refactor code
...
Signed-off-by: Saurav Sharma <appdroiddeveloper@gmail.com >
2019-10-13 15:11:39 +02:00
Saurav Sharma
bc82bd5f23
raise error if preparation_command length is not equal to command or 1
...
Signed-off-by: Saurav Sharma <appdroiddeveloper@gmail.com >
2019-10-13 15:11:39 +02:00
Saurav Sharma
0c0713da30
Add multiple --prepare support
...
Signed-off-by: Saurav Sharma <appdroiddeveloper@gmail.com >
2019-10-13 15:11:39 +02:00
sharkdp
3d41613f43
Fix markdown export tests
2019-09-16 21:34:07 +02:00
sharkdp
8a7d04347d
Higher precision for rel. speed in markdown output
...
closes #208
2019-09-16 21:27:54 +02:00
petr-tik
40b6774324
Correct type in command line arg
2019-09-13 19:37:06 +02:00
Piyush Rungta
8244bc11b1
Support case when only step size is float
...
After this change, invokations like
--parameter-scan t 0 1 --parameter-step-size 0.1
become valid
2019-09-01 14:31:35 +02:00
Piyush Rungta
c73636c478
Provide short option for --parameter-step-size
...
Rename STEP to DELTA
2019-09-01 14:31:35 +02:00
Piyush Rungta
ef523e1fb0
Improve StepRequired error message
2019-09-01 14:31:35 +02:00
Piyush Rungta
b10b93f473
Add support for floats in --parameter-scan
2019-09-01 14:31:35 +02:00
Piyush Rungta
737403df35
Run cargo fmt
2019-09-01 14:31:35 +02:00
sharkdp
d6e1298ac7
Whitespace fix in --help text
2019-08-24 08:56:59 +02:00
Piyush Rungta
3e3f4d9a26
Add argument to take parameter range step size
2019-08-24 08:56:59 +02:00
philipp
e867366197
Use byte literal instead of as u8
2019-07-17 21:11:44 +02:00
philipp
16210a1d13
Pass Copyable enums by value
2019-07-17 21:11:44 +02:00
philipp
78d249d656
Use slices instead of borrowed string
2019-07-17 21:11:44 +02:00
philipp
c0ada7d044
Use slices instead of borrowed vectors
2019-07-17 21:11:44 +02:00
philipp
f3f8c9ca98
Avoid redundant ref
2019-07-17 21:11:44 +02:00
philipp
76119e6664
Use f64::from instead of as f64
2019-07-17 21:11:44 +02:00
philipp
a0bd0248ce
Avoid redundant explicit borrow
2019-07-17 21:11:44 +02:00
philipp
5849446315
Avoid long literal without separator
2019-07-17 21:11:44 +02:00
philipp
cfd298ef14
Avoid redundant return
2019-07-17 21:11:44 +02:00
philipp
5e4e383dca
Avoid redundant field names
2019-07-17 21:11:44 +02:00
philipp
3f79985473
Factor common operations into closure
2019-07-17 21:11:44 +02:00
William Chargin