Now, time functions accept array inputs even if they don't have all the
elements, 0 will be assumed if a value is not present.
Also, jv2tm now properly clamps large number values to a signed 32-bit
integer and rejects nan.
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65885
There was a incorrect else, that caused jq to not ensure that the
argument to strftime/1 is a string when the input is a number; this ends
up calling jv_string_value on a non-string value, which does not work,
and causes an assert failure.
Also fix same bug in strflocaltime/1.
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67403
Previously, the tonumber/0 filter parses the input as JSON values, but
this is less-performant on large non-number strings. Parsing the input
string as number literal fixes the performance issue. Also, this fix
changes the filter to reject numbers with white spaces.
C99 only allows string literals long at most 4095 characters.
jq_builtins was a lot longer than that.
I rewrote all the optional libm error stubs in C so the value of
jq_builtins is not build dependent.
I replaced the command that generates builtin.inc with a POSIX compliant
od|sed command that encodes builtin.jq as a comma delimited list of
octal numbers (that can be embedded in C using a {} literal).
I also added -Woverlength-strings to AM_CFLAGS to verify that the
problem is fixed.
Fixes#1481
Previously, ltrimstr/rtrimstr would just let the input pass through for
non-string inputs or arguments.
That was happening because, they were leaking the errors returned by
startswith/endswith treating them as if they were jv_false().
The leak was resolved by #2977 for 1.7.1
This patch rewrites ltrimstr and rtrimstr in jq, and makes them not
ignore startswith and endswith errors anymore.
ltrimstr/rtrimstr was ignoring and leaking the error returned by
f_startswith()/f_endswith().
This also means that they just let the input pass through for non-string
inputs or arguments.
Only fix the leak for now; in the next release, #2969 will make them
rethrow the error returned by startswith/endswith.
Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64946
* Build windows 64bit binary using UCRT64
Is the default and recommended msystem setting. Will produce
binaries that are compatible with windows 10 and later.
Also run tests for 32bit build.
Related to #2831
* Use jq -b in tests/shtest
* Add Windows strptime
* Make Windows-optional tests not run on Windows again
---------
Co-authored-by: Nicolas Williams <nico@cryptonector.com>
This patch exports all the binary operator builtins functions from
builtin.c and uses them for constant folding in the parser, allowing
constant folding to work will all kinds and combinations of constants.
Now string*number, $ARGS+$ARGS, string/string, etc will also be
constant folded and the implementation of constant folded operators and
runtime operators will be the same.
And thanks to the new ERRORK bytecode operation, errors are constant
folded too! (e.g. 1 / 0 [] * {} etc)
Previously constant folding of zero division (e.x. 1/0) produces a
compile error. This was incorrectly implemented by checking if the
division result is infinite, so produces wrong results compared to the
query where no constant folding is processed (e.x. 1e308/0.1). This
patch delays the operation when the divisor is zero. This makes the
results more consistent, but changes the exit code on zero division from
3 to 5. Also 0/0 now produces the zero division error, not NaN.
This patch also fixes the modulo operation. Previously constant folding
logic does not take care of the % operator, but now it folds if the both
dividend and divisor are safe numbers to cast to the integer type, and
the divisor is not zero. This patch also fixes some code that relies on
undefined cast behaviors in C. The modulo operation produces NaN if
either the dividend or divisor is NaN.
You only need to specify the return type in a function pointer
declaration in C.
If you use () in the declaration, the function pointer can be called
with any arguments, and the type of the arguments is decided for each
function call based on the types of the arguments used for the call.
(To declare a function pointer for a function with no arguments, you use
`(void)'.)
Since all the cfunction structs have a fptr that points to a functions
that return jv, not void, we can we can just declare cfunction.fptr as
jv (*)() and avoid having those annoying and technically not C-standard
compliant casts everywhere.
Error on non-number and nan codepoint, would asserd before
Replace negative codepoint and surrogate range with unicode replacement character, would assert before
Fixes#1160
This is unnecessary since we are using AC_USE_SYSTEM_EXTENSIONS in
configure.ac, which makes the compiler always run with -D_GNU_SOURCE=1
when building for GNU/Linux.
Also, the #define _GNU_SOURCE in src/util.c, and in
src/inject_errors.c (--enable-inject-errors), were not guarded by a
#ifndef _GNU_SOURCE so they were just triggering redefinition warnings.
Instead of just using {"captures":[]}.
sub functions are use captures for replacement expressions.
If we don't populate, captures for empty matches, the replacement
expression is run with an empty object as input instead of an object
containing the named captures with "" as value:
* before:
$ jq -n '"123foo456bar" | gsub("[^a-z]*(?<x>[a-z]*)"; "Z\(.x)")'
"ZfooZbarZnull"
* after:
$ jq -n '"123foo456bar" | gsub("[^a-z]*(?<x>[a-z]*)"; "Z\(.x)")'
"ZfooZbarZ"
---
I also removed a redundant
result = NULL;
if (result) {
...
}
The primary purpose of this commit is to rectify most problems with
`gsub` (and also `sub` with the `g` option), in particular fix#1425 ('\b'),
fix#2354 (lookahead), and fix#2532 (regex == `"^(?!cd ).*$|^cd "`).
This commit also partly resolves#2148 and resolves#1206 in that
`gsub` no longer loops infinitely; however, because the new `gsub`
depends critically on `match/2`, the behavior when regex == `""` is
sometimes non-standard.
The documentation has been updated to reflect the fact that `sub`
and `gsub` are intended to be regular in the second argument.
Also, `_nwise/1` has been tweaked to take advantage of TCO.
Extend jv_number to use decNumber for storing number literals. Any math
operations on the numbers will truncate them to double precision.
Comparisons when both numbers are literal numbers will compare them
without truncation.
Delay conversion of numbers to doubles until a math operation is performed,
to preserve precision. A literal jv_number will only need conversion to
double once, and will reuse the resultant double on subsequent
conversions.
Outputting literal jv_numbers preserves the original precision.
Add strong pthread requirement to manage contexts/allocations for
converting numbers between their decNumber, string, and double formats.
block_bind_incremental is block_bind_referenced in a loop backwards. For
an 1-inst block, it does the same thing and isn't too much more
expensive, so it's not really useful to keep both.
Also, block_bind_referenced was a better name for the function.
Avoid doing the internal binding of top-level symbols in the parser,
leaving that work to be done in a post-processing step. For builtins,
this lets us do a reference-aware bind step (block_bind_incremental)
*after* generating builtins/0.
Libraries are a bit trickier since they may be bound multiple times, so
instead of thinking through the implications I added (block_bind_self)
to resolve all internal symbols immediately.
Remove the special code which loads ~/.jq in builtin.c, and instead glue
an optional include which points to the same file onto the main program
in linker.c.
Fixes a minor bug where errors in ~/.jq would be labelled <builtin>.
The parser generates a call to error/0 on break. Having that exported
from C directly removes a language dependency on builtin.jq.
In any case, error/0 seems to be the more fundamental operation, seeing
as the old C impementation of error/1 did nothing useful with its input.
The distinction between input/0 and _input/0 has been unnecessary since
a9c613..7dc34b, which moved end-of-input handling into C and replaced
the jq definition of input with `def input: _input;`
If these values do not match exactly, it is because the UTF-8 is invalid
anyway and we counted codepoints differently than oniguruma did. Perhaps it
would be better to error out here, but at least one similar loop already
uses < vs != and since we're off the rails anyway this might be OK. It is
certainly better than overruning the buffer.
Resolves https://github.com/stedolan/jq/issues/1804
Mattias Wadman