Add Streaming parser (--stream)

Streaming means that outputs are produced as soon as possible.  With the
`foreach` syntax one can write programs which reduce portions of the
streaming parse of a large input (reduce into proper JSON values, for
example), and discard the rest, processing incrementally.

This:

    $ jq -c --stream .

should produce the same output as this:

    $ jq -c '. as $dot | path(..) as $p | $dot | getpath($p) | [$p,.]'

The output of `jq --stream .` should be a sequence of`[[<path>],<leaf>]`
and `[[<path>]]` values.  The latter indicate that the array/object at
that path ended.

Scalars and empty arrays and objects are leaf values for this purpose.

For example, a truncated input produces a path as soon as possible, then
later the error:

    $ printf '[0,\n'|./jq -c --stream .
    [[0],0]
    parse error: Unfinished JSON term at EOF at line 3, column 0
    $
This commit is contained in:
Nicolas Williams committed 2014-12-26 23:05:56 -06:00
1 parent 906d2537b9
commit 5bfb9781f7
10 files changed
+480 -80

No files matched your search

+12
View File
@@ -39,6 +39,8 @@ struct jq_state {
int initial_execution;
jv attrs;
jq_input_cb input_cb;
void *input_cb_data;
};
struct closure {
@@ -1037,3 +1039,13 @@ jv jq_get_attr(jq_state *jq, jv attr) {
void jq_dump_disassembly(jq_state *jq, int indent) {
dump_disassembly(indent, jq->bc);
}
void jq_set_input_cb(jq_state *jq, jq_input_cb cb, void *data) {
jq->input_cb = cb;
jq->input_cb_data = data;
}
void jq_get_input_cb(jq_state *jq, jq_input_cb *cb, void **data) {
*cb = jq->input_cb;
*data = jq->input_cb_data;
}