Skip to content

Query data

The default action, a jq filter run against the active source (see Sources).

The top-level paths name the keys to fetch. The result is printed as pretty JSON by default (see Output formats)

Fetch the key "greeting"
iq '.greeting'
Fetch the key "book:1" (a key containing a colon needs bracket-quoting)
iq '.["book:1"]'
Fetch the key "book:1" and extract one field
iq '.["book:1"].title'
Fetch keys "book:1", "book:2" and project a field from each
iq '[ .["book:1"].title, .["book:2"].title ]'
Values are strings, convert before arithmetic calculations
iq '.["book:2"].price | tonumber + 5'

Escaping

Always wrap the filter in single quotes. jq syntax is full of characters that the shell otherwise expands or splits: brackets ([ ]), whitespace, |, *, $. Bracket-quoting a colon key like .["book:1"] reads as a glob to zsh (no matches found) or bash unless quoted.

Cross-source queries

Both Compose and Combine reduce per source, then combine. Pick what the query needs.

Compose source() Combine iq combine
Shape a single jq filter a positional spec per source, then one --with program
Correlated reads
(B keyed by A's rows)
✓ nest source() ✗ specs are independent
Quoting sub-filter is a quoted string inside the filter each spec is its own argument
Memory each reduced result held until combine, plus a correlated source() re-runs per row each reduced result held until combined

Use source() when a read depends on another source's values, or to keep everything in one composable filter.

Use iq combine for a straightforward join, union, or aggregate across a few sources.

Compose source()

source("name"; "<jq>") runs <jq> against source name (reduced, streamed and pushed down like any query) and yields its results as a stream. A one-argument source("name") yields the whole source.

Both arguments of source() are strings, so they must be quoted. It also yields a stream. Collect it before indexing with INDEX(source(…); .id) or [source(…)], not source(…) | INDEX(.id).

A filter that calls source() runs over a null input. Every read is an explicit source() call, and there is no implicit primary source. As a result, it needs no active source. Names resolve through the registry like --src, active-group namespacing included.

Join users and orders in a single filter (no active source needed)
iq 'INDEX(source("users"; ".[]"); .id) as $u
    | source("orders"; ".[] | select(.total > 99)")
    | {name: $u[.userId].name, total}'

Correlated lookups re-run

A source() opened inside a stream runs its sub-filter once per element. The connection is reused, but the sub-filter re-executes.

Hoist a constant lookup into a binding INDEX(source("users"; ".[]"); .id) as $u | … and index $u per element instead.

Optimize memory usage

Binding source() to a jq variable materializes that call's whole result set in memory (jq indexing needs a concrete array), even though the read itself streams.

Push the reduction into the sub-filter, source("orders"; ".[] | select(.total > 99)"), not source("orders"; ".[]") filtered outside, so only the rows you need are held.

A bare source("big") over a large source buys no streaming benefit. When each side is large and independent, prefer iq combine.

Combine combine

iq combine <source>[=<jq>]... --with <jq> [flags]

Query several sources and combine their results with one jq program.

Each positional is a source spec (<source>[=<jq>]) reduced at the source (bounded reads, streaming scans, and predicate pushdown all still apply). The spec's results bind to a jq variable named after the source, with /, . and - becoming _ (so prod/books=.[] binds $prod_books). --with is the final program, so it can join, union ($a + $b), aggregate, or fan across any number of sources.

Each source reduces at the source, and a pushable filter pushes down. As a result, this never copies whole datasets to join them. A spec with no filter binds the whole keyspace, which is a whole-keyspace read. It needs --unbounded, exactly as the same expression does on a plain query.

short long default description
--no-compile ✗ disable server-side predicate pushdown. Run each spec's filter client-side
--with <string> required final jq over the bound source results (each spec's results bound to $name), run over a null input
Join users with orders on a shared id, across two sources
iq combine 'users=.[] | {id, name}' \
           'orders=.[] | select(.total > 99)' \
   --with '($users | INDEX(.id)) as $u | $orders[] | . + {name: $u[.userId].name}'

Reduce, then combine

Each spec is evaluated independently. Its (already reduced) result is held in memory before --with runs. For this reason, keep a stage's output small with select/projection/aggregation.

A spec that must materialize its whole source (keys, ., map(...)) still needs --unbounded, exactly like a single-source query. A .[]-rooted spec streams without it.

Unbounded --unbounded

Permit a filter that loads the whole dataset into memory. It also materializes a .[]-rooted filter instead of streaming it. As a result, a filter that collapses the keyspace into one value (keys, ., map(...)) runs only with it (see Read strategies).

The flag also lets a read load a Redis key with more than 1,000,000 elements. Without it, the read fails and names the key, its type, and its size (see Redis). It does the same for an HBase row with more than 1,000,000 cells in one column family (see HBase). A key in an RDB dump follows the same limit, and the flag lifts it there too (see File dumps). The same flag does this for iq diff, iq schema, and the copy and move modes.

iq combine and iq data carry their own copies of the flag where they apply.

Client-side --no-compile

Disable pushdown.

short long default description
--no-compile ✗ disable predicate pushdown. Run the full .[] | select() filter client-side

Native form --native

Read each item in the native form of the source driver instead of normalized JSON. DynamoDB, MongoDB and HBase have a native form today. DynamoDB JSON gives every attribute value a type tag, for example {"price": {"N": "1.50"}}. MongoDB canonical Extended JSON tags each BSON type, for example {"year": {"$numberInt": "2020"}}. The HBase raw form holds the row key, each qualifier and each cell value as base64.

short long default description
--native ✗ read items in the native form of the source (DynamoDB JSON, MongoDB Extended JSON or HBase raw bytes). No predicate is pushed down, and the filter must start with .[]
Read DynamoDB, MongoDB and HBase items with their types
iq --native --src books '.[] | select(.year.N == "2020") | .price'
iq --native --src mbooks '.[] | select(.year["$numberInt"] == "2020") | .title'
iq --native --src hbooks '.[] | .cf'

The filter sees the tags, so it reads .year.N and not .year. iq scans the whole table and runs the filter on the client. --native works with --json, --jsona, --jsonl, and --yaml. See Native form for the copy and the dump.