Commit Graph
123 Commits
Author SHA1 Message Date
WinkleMad f1f39b42e3 Advance past the whole codepoint after a zero-width regex match (#3586) 2026-07-19 21:13:51 +09:00
itchynyandDirk Müller 46d1da3094 Tighten string length bounds and propagate invalid jv in implode
The bound added in CVE-2026-32316 (e47e56d22) still allowed
`sizeof(jvp_string) + (currlen + len) * 2 + 1` to wrap `size_t` on
32-bit platforms. Tighten the threshold so the final allocation
fits in 32-bit `size_t`.

Also break out of `jv_string_implode` and `f_string_implode` once
`jv_string_append_codepoint` returns an invalid `jv`; otherwise the
next iteration triggers the assertion in `jvp_string_ptr` (or
invokes undefined behavior under `-DNDEBUG`).

Fixes CVE-2026-54679.

Co-authored-by: Dirk Müller <dirk@dmllr.de>
2026-06-16 14:33:04 +09:00
itchyny a829f678f3 Fix one-byte over-read in BASE64_DECODE_TABLE (#3547)
The table was declared with 255 entries but indexed by an unsigned
char, so the decoder would read one byte past the array for input
bytes equal to 0xFF. Through jq's normal input paths the over-read
is unreachable because invalid UTF-8 bytes are normalized to U+FFFD
before reaching @base64d, but the table is still wrong at the C
level and reachable by libjq callers that construct strings via
the C API without UTF-8 validation.
2026-05-23 09:36:15 +09:00
Yu-Fu Fu 7122866869 Guard deep structural equality and comparison recursion (#3539)
jv_equal and jv_cmp overflows the C stack on deeply nested
input. Cap recursion at 10000 with -1 / INT_MIN sentinels;
operators that compose user expressions surface this as
"Equality check too deep" / "Comparison too deep".

Fixes CVE-2026-47770.
2026-05-22 20:07:16 +09:00
itchyny d1a12569d9 Limit the containment check depth
This fixes CVE-2026-40612.
2026-04-24 22:02:24 +09:00
tlsbollei fdf8ef0f08 Add runtime type checks to f_string_indexes
This fixes CVE-2026-39956.
2026-04-08 21:43:46 +02:00
Vladimír Marek 9f778b4cb5 Add Solaris support (#3515)
I love jq :) So while updating the version we ship in Solaris I have
tried to modify stuff to make everything work out of the box. Just

```
autoreconf -fiv
./configure
gmake
gmake check
```

I also went ahead and created Solaris test action should you be
interested in having it too. The changes are not extensive and I have
tried to comment them nicely in each commit. Hopefully that's enough.

Thanks for jq!
2026-04-08 16:02:41 +02:00
itchyny ed3c7f214b Fix tonumber and toboolean to reject strings with embedded null bytes (#3496) 2026-03-05 08:28:43 +09:00
itchyny bfcf82fde1 Refactor @uri and @urid and fix multi-byte UTF-8 corruption (#3495)
This commit improves performance by refactoring `@uri` and `@urid`
to pre-alloacte buffers and build the `jv` string at the end.
Also, this fixes `@urid` to prevent multi-byte UTF-8 corruption,
and to preserve embedded null bytes.
2026-03-05 08:28:09 +09:00
itchyny 262f0b81c1 Add missing validity check in f_strflocaltime after f_localtime (#3491) 2026-03-02 21:58:07 +09:00
itchyny 601a37f452 Refactor tm2jv to handle fractional seconds removing duplicated code (#3489) 2026-03-01 06:18:14 +09:00
Fabian Fleischer 061fd14d74 Fix off-by-one in modulo operator (#3486) 2026-02-27 04:35:36 +09:00
itchyny 22361cfb87 Improve error message truncation with closing delimiters (#3478)
Truncated strings now show closing delimiters: `..."` for strings,
`...]` for arrays, and `...}` for objects, making error messages
clearer.
2026-02-25 08:14:23 +09:00
itchyny b33a7634ba Fix reversed pointer subtraction in f_env bounds check (#3465) 2026-01-20 21:56:09 +09:00
itchyny 695830c453 Fix year 2038 problem on 32-bit platforms (fix #3369) (#3407) 2025-09-15 07:46:29 +09:00
Mattias Wadman 499c91bca9 Fixes CVE-2025-49014 which was introduced in 1.8.0 2025-06-19 00:11:01 +02:00
itchyny 7cc202aa6e Fix portability of strptime especially for Windows (#3342) 2025-06-14 08:00:55 +09:00
Brahm Lower c350566001 Adds "toboolean" builtin function for converting strings to booleans (#2098) 2025-05-18 16:31:10 +09:00
itchyny a8f27cc4db Fix warning: a function definition without a prototype is deprecated (#3259) 2025-05-07 21:25:32 +09:00
itchyny 668871bdfa Fix warning: sign-compare in @base64d format (#3287) 2025-03-06 17:07:54 +09:00
itchyny 94fd973ebb Fix calloc error on @base64d format (ref #3280) (#3286)
This commit fixes a regression of a8ce2ff, e.g. `"=" | @base64d`.
2025-03-06 04:30:46 +09:00
itchyny a8ce2ff09a Avoid zero-length calloc (#3280)
Using `calloc` function with a count of zero elements is implementation
defined and it may cause allocation errors on some platforms (ref #3277).
This commit fixes `jv_mem_calloc` callers to avoid undefined behavior.
2025-03-04 22:33:51 +09:00
itchyny 4003202ccf Fix segmentation fault on strftime/1 and strflocaltime/1 (#3271)
When we use a large format string to `strftime/1` and `strflocaltime/1`,
we get segmentation fault. This happens because the result buffer is
allocated on the stack using `alloca` function, and `strftime` function
writes out of the space when the size is not enough to format time.
I fixed the segmentation fault issue by allocating the result buffer on
the heap using `malloc` function, because `alloca` function does not
provide a way to detect insufficient space.
2025-03-04 22:28:09 +09:00
itchyny dc849e9bb7 Improve performance of repeating strings (#3272)
This commit improves the performance of repeating strings, by copying
the result string instead of the string being repeated. Also it adds
an error message when the result string is too long.
2025-03-04 22:13:55 +09:00
itchyny b7a1201b72 Preserve numerical precision on length/0 (#3275) 2025-03-04 22:09:33 +09:00
itchyny 17be97a8ad Define _BSD_SOURCE in builtin.c for OpenBSD support (#3278)
On OpenBSD, some prototypes are hidden behind `_BSD_SOURCE`.
Defining this variable fixes #3252, and also fixes #3276.
This can be seen as a regression of #2631.
2025-03-04 21:39:52 +09:00
itchyny 95ee663b28 improve performance of unique and unique_by (#3254)
Previously, `unique` and `unique_by` filters are implemented using
`group_by` and then `map(.[0])`. This commit re-implements in C, to
avoid unnecessary boxing of grouping, and improves the performance.
2025-03-04 19:59:41 +09:00
Marcin Serwin f3d4c1165d fix: copy the string returned by getenv (#3264) 2025-02-18 07:54:13 +09:00
Marcin Serwin 6d8e9f98ae fix: populate timezone data when formatting time (#3203)
The `jv2tm` function was zeroing fields of `struct tm` that were not
specified by the standard. However, depending on the libc this produced
incorrect timezone data when used together with formatting functions.
This change tries to fill the timezone data using either `mktime`,
`timegm`, or manually.

Apple's Libc implementation contains a bug which causes it to ignore
the offset data present in the `struct tm` in favor of the older heuristic
needed by legacy standards. This workaround temporarily sets the global
timezone so it gets picked up during formatting.
2025-02-17 21:41:01 +09:00
itchyny 6ce3bdf382 fix: preserve numerical precision on unary negation (#3242) 2025-02-06 07:50:21 +09:00
Mattias Wadman 2755664a8e Non-matched optional capture group should be null and offset -1 (#3238)
Fixes #3093
2025-02-04 19:18:58 +01:00
Eloy Coto 562ffec869 refactor: move bsearch function to C-code (#2945)
This commit fixes issue #527 and move the bsearch function to a native
C-code. This change improves the performance of this function.
2025-01-25 22:09:43 +09:00
Emanuele Torre 0b82b3841b builtin.c: typecheck builtin cfunctions in function_list
In C23 (default C standard used by GCC 15),  jv (*fptr)();  has become
equivalent to  jv (*fptr)(void);  so we can no longer assign builtin
implemenations directly to the fptr member of cfunctions without
generating a compile error.

Since there does not seem to be any straight-forward way to tell
autoconf to force the compiler to use C99 short of explicitly adding
-std=c99 to CFLAGS, it is probably a cleaner solution to just make the
code C23 compatible.

A possible solution could have been to just redeclare  cfunction.fptr
as void*, but then the functions' return type would not have been type
checked (e.g. if you tried to add a {printf, "printf", 2}, where printf
is a function that does not return jv, the compiler wouldn't have
complained.)
We were already not typechecking the arguments of the functions, so e.g.
  {binop_plus, "_plus", 3},  /* instead of {f_plus, "_plus, 3},       */
  {f_setpath, "setpath", 4}, /* instead of {f_setpath, "setpath", 3}, */
compile without errors despite not having the correct prototype.

So I thought of instead improving the situation by redefining
cfunction.fptr as a union of function pointers with the prototypes that
the jq bytecode interpreter can call, and use a macro to add the builtin
functions to function_list using to the arity argument to assign the
implementation function to the appropriate union member.

Now the code won't compile if the wrong arity, or an arity not supported
by the bytecode interpreter (>5 = 1input+4arguments), or a prototype not
jallable by the bytecode interpreter (e.g. binop_plus that doesn't
expect a  jq_state*  argument).

Also, the code now compiles with gcc -std=c23.

Fixes #3206
2024-12-01 11:49:44 +01:00
itchyny a7b22539a4 fix: make last(empty) yield no output values (#3179) 2024-11-13 20:35:50 +09:00
Emanuele Torre da2a0b970c mktime/0: remove unnecessary length>=6 input requirement (#3162)
This check was redundant and useless since jv2tm() used to require 8
elements, and errored anyway if not enough elements were provided.
After commit bc96146, this check also prevents mktime/0 from accepting
now supported inputs with less than 8 elements.

Ref: https://github.com/jqlang/jq/pull/3070#issuecomment-2299902072

Reported-by: itchyny <itchyny@cybozu.co.jp>
2024-08-22 08:17:54 +09:00
fletcher gornick 0e0cdd552e feat: uri decode function (#3161) 2024-08-22 08:07:24 +09:00
riastradh 2ee20ca604 Avoid ctype.h abuse: cast char to unsigned char first.
fix #3151
2024-07-11 23:17:06 +02:00
Nicolas Williams 6d02d53f51 Make tests pass when --disable-decnum 2024-05-15 12:48:00 -05:00
Emanuele Torre 22a03e9e06 @base64d: fix unhandled overflow
$ ./jq-before -n '238609295*"|||"|@base64d|"."'
    src/builtin.c:718:29: runtime error: signed integer overflow: 715827885 * 3 cannot be represented in type 'int'
    jq: error: cannot allocate memory
    Aborted (core dumped)

    $ ./jq-after -n '238609295*"|||"|@base64d|"."'
    jq: error (at <unknown>): string ("||||||||||...) is not valid base64 data

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67640
2024-03-27 10:16:15 +01:00
Mattias Wadman be437ec049 Add trim/0, ltrim/0 and rtrim/0 that trims leading and trailing whitespace (#3056) 2024-03-20 11:04:17 +01:00
Emanuele Torre b6da71bf68 declare static arrays const where possible
Some static arrays that could be readonly memory were declared mutable.
2024-03-20 00:22:54 +01:00
Emanuele Torre bc96146825 builtin.c: jv2tm: fix UB and accept array inputs with not all the values
Now, time functions accept array inputs even if they don't have all the
elements, 0 will be assumed if a value is not present.

Also, jv2tm now properly clamps large number values to a signed 32-bit
integer and rejects nan.

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65885
2024-03-19 02:01:53 +01:00
Emanuele Torre 1411ce63d5 strftime/1: fix validation of non-string argument with number input
There was a incorrect else, that caused jq to not ensure that the
argument to strftime/1 is a string when the input is a number; this ends
up calling jv_string_value on a non-string value, which does not work,
and causes an assert failure.

Also fix same bug in strflocaltime/1.

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67403
2024-03-15 14:30:23 +01:00
itchyny ce0e788ce2 improve tonumber/0 performance by parsing input as number literal
Previously, the tonumber/0 filter parses the input as JSON values, but
this is less-performant on large non-number strings. Parsing the input
string as number literal fixes the performance issue. Also, this fix
changes the filter to reject numbers with white spaces.
2024-03-01 16:33:17 +01:00
Ulrich Eckhardt 6980cc9713 src/builtin.c: Add missing semicolon 2024-02-08 19:27:48 +01:00
Emanuele Torre bd3c828f98 libm.h+builtin.c: add and use LIBM_DA and LIBM_DA_NO macros
For functions that from one double return an array with two numbers.
2024-01-26 16:16:25 -06:00
Emanuele Torre 605836b263 builtin.c: fix build with -Woverlength-strings
C99 only allows string literals long at most 4095 characters.
jq_builtins was a lot longer than that.

I rewrote all the optional libm error stubs in C so the value of
jq_builtins is not build dependent.

I replaced the command that generates builtin.inc with a POSIX compliant
od|sed command that encodes builtin.jq as a comma delimited list of
octal numbers (that can be embedded in C using a {} literal).

I also added -Woverlength-strings to AM_CFLAGS to verify that the
problem is fixed.

Fixes #1481
2024-01-26 16:16:25 -06:00
Emanuele Torre 1f1e619f4e builtins: make ltrimstr and rtrimstr error for non-string inputs
Previously, ltrimstr/rtrimstr would just let the input pass through for
non-string inputs or arguments.

That was happening because, they were leaking the errors returned by
startswith/endswith treating them as if they were jv_false().
The leak was resolved by #2977 for 1.7.1

This patch rewrites ltrimstr and rtrimstr in jq, and makes them not
ignore startswith and endswith errors anymore.
2024-01-16 16:57:37 -06:00
George Hopkins f954e82b14 Provide strptime implementation on all systems 2024-01-16 16:01:27 -06:00
Emanuele Torre 511d50b15b ltrimstr/1+rtrimstr/1: don't leak on invalid input or arguments
ltrimstr/rtrimstr was ignoring and leaking the error returned by
f_startswith()/f_endswith().

This also means that they just let the input pass through for non-string
inputs or arguments.

Only fix the leak for now; in the next release, #2969 will make them
rethrow the error returned by startswith/endswith.

Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64946
2023-12-13 15:13:46 +01:00