Survey · aoi‑probe 0.1.0 and vendor docs · Generated 2026-07-30

State of CLI machine output

None of 16 widely used CLIs has all four basics of machine‑readable output.

The four basics are one JSON object per line, a field naming each record's type, a version of the output format carried in the output, and a final record with an explicit success boolean. In the dataset generated 2026-07-30, 4 tools have three of them: cargo, terraform, git trace2 and pytest-reportlog. 10 of the 16 mark record types, under 9 different spellings, and only one ends with a success boolean (cargo's build-finished).

The usual objection to a standard like this is that the problem's already solved: terraform emits JSON, ripgrep emits JSON, just use those. So we measured. We ran tools that have a machine‑output mode and inspected the bytes; where a flagship mode needs credentials or a cluster, the answer comes from primary documentation instead, and every row says which.

The problem isn't solved. Many tools built part of a contract, and no two built the same part. When 10 tools invent a type field on their own and spell it 9 ways, that convergence is the case for a standard, and the divergence is what going without one costs.

Four basics, 16 tools

None hasall four. Each row is one tool's main machine‑output mode.

The mode column says what we could reach: stream is the tool's streaming or bulk output, query is a data‑returning command in its machine‑output mode, and meta means only a version command ran without credentials. Docs rows come from vendor documentation; probe rows were also run locally.

Machine-output basics for each of the 16 surveyed tools
Tool	Mode	Checked by	NDJSON	Type field	Format version	Terminal event	Success flag	Basics
cargo	stream	probe + docs	yes	reason	no	build-finished	success	3/4
terraform	stream	probe + docs	yes	type	ui	no	no	3/4
ripgrep	stream	probe + docs	yes	type	no	summary	no	2/4
git trace2	stream	docs	yes	event	evt	atexit	no	3/4
git	stream	probe + docs	no	leading sigil character (#, 1, 2, u, ?, !)	no	no	no	1/4
pytest-reportlog	stream	docs	yes	$report_type	pytest_version	SessionFinish	no	3/4
docker	meta	probe + docs	varies	Type + Action	no	no	no	1/4
kubectl	meta	probe + docs	no	kind	apiVersion	no	no	2/4
syft	query	docs	no	metadataType	schema.version	no	no	2/4
gh	query	docs	no	no	no	no	no	0/4
aws-cli	query	docs	no	no	no	no	no	0/4
journalctl	stream	docs	varies	no	no	no	no	0/4
stripe-cli	stream	docs	tty-dependent	type	no	no	no	1/4
ansible	query	docs	no	no	no	no	no	0/4
eslint	query	docs	no	no	no	no	no	0/4
trivy	meta	probe + docs	no	no	SchemaVersion	no	no	1/4
Count		7/16 probed	5/16	10/16	6/16	4/16	1/16	0/16 at 4/4

5/16 always write newline‑delimited JSON, and 3more do so only in some modes or only when stdout isn't a terminal (docker, journalctl and stripe-cli).
10/16mark each record's type, spelled reason, type, event, leading sigil character (#, 1, 2, u, ?, !), $report_type, Type + Action, kind, metadataType and $message_type. Cargo carries two in one stream: reason on the envelope and $message_type inside embedded rustc diagnostics.
6/16 put a version in the output itself. Terraform's sits under a key called ui inside an event typed version, where no key‑name search will find it. Pytest‑reportlog's is the pytest version rather than a format version, though a consumer can still branch on it.
4/16 emit a terminal completion event, and 1/16 carries an explicit success boolean: cargo's build-finished. Ripgrep's summary carries statistics but no verdict, so a clean no‑match run looks the same as an aborted one.
None has all four. 4 have three: cargo lacks a format version, and terraform, git trace2 and pytest-reportlog lack a success flag.

What each tool is missing

Most of these tools are a field or two away from a contract an agent could rely on.

A score would hide what matters here. Each tool below has a specific gap you could file as an issue.

cargo--message-format jsonrun locally

No version declaration anywhere in the stream. A consumer cannot branch on format version. Source

terraform-json on plan/apply/initrun locally

No terminal event on failure, and no success boolean at any time. Source

ripgrep--jsonrun locally

No version field of any kind, and no success indicator on the terminal event. Source

git trace2GIT_TRACE2_EVENTfrom docs

Integer code rather than a boolean; no stability promise and no offline documentation (api-trace2 ships no man page). Source

gitstatus --porcelain=v2run locally

Not JSON at all, and no in-band version. Arguably the most-invoked tool in any agent shell session. Source

pytest-reportlog--report-log=FILEfrom docs

File-only — it can never write to stdout, so it cannot be piped. Source

docker--format json / system events --format jsonrun locally

Three different output shapes under one --format json flag: `events`/`ps` are NDJSON, `inspect` is a JSON array, `version` is a single object. The tool does not agree with itself. Source

kubectl-o jsonrun locally

Buffered single document, no terminal event, no success boolean, no offline schema discovery. Source

syft-o syft-jsonfrom docs

A single buffered document — structurally cannot stream, so no terminal event and no success boolean. Source

gh--json f1,f2 (fields mandatory)from docs

Buffered array, no envelope, no version, no completion signal. Source

aws-cli--output jsonfrom docs

No envelope, no version, no completion signal. Source

journalctl-o json / json-seq / json-prettyfrom docs

`--show-cursor` emits a non-JSON `-- cursor: …` trailer onto stdout. Source

stripe-clilisten --format jsonfrom docs

Stdout is not pure: EndpointResponse lines, POST errors, and logs-tail warnings print regardless of --format. Source

ansibleANSIBLE_STDOUT_CALLBACK=ansible.posix.json (no flag)from docs

Buffered and emitted only at run end, so an aborted run produces nothing at all. `-v` writes to stdout by default and corrupts the document. Source

eslint-f jsonfrom docs

A bare array with no envelope and no totals — the consumer must sum errorCount. An inherited TIMING env var injects a text table into stdout. Source

trivy-f jsonrun locally

A version number with no policy attached is not a compatibility contract. Source

Same ideas, different spellings

Patterns that recur across tools, as recorded in the dataset's findings.

10 of the 16 surveyed tools independently invented a type discriminator, in 9 mutually incompatible spellings (counting the $message_type cargo nests inside rustc diagnostics).

spellings: reason · type · event · leading sigil char · $report_type · Type+Action · kind · metadataType · $message_type

sharpest example: cargo carries TWO differently-spelled discriminators in one stream: `reason` at the envelope level and `$message_type` inside embedded rustc diagnostics.

Three tools independently invented a sigil to mark envelope keys apart from payload keys. All three chose a different sigil.

spellings: terraform: @ (@level, @message, @module, @timestamp) · pytest/rustc: $ ($report_type, $message_type) · journalctl: __ and _ (__CURSOR, _BOOT_ID)

A consumer reading `.version` gets a different KIND of answer from each tool.

examples: semgrep: the tool's version · npm ls: the described project's version · cargo metadata: the output format's version

1 of the 16 surveyed tools emits a terminal event with an explicit success boolean: cargo's build-finished.success.

runners up: Everyone else who signals completion downgrades to an integer requiring an out-of-band code table: git trace2 `code`, pytest-reportlog `exitstatus`, pytest-json-report `exitcode`. Ansible gives counters. ESLint makes you sum errorCount.

No tool in the survey can be asked, offline and unauthenticated, to describe its own output.

closest: gh enumerates field NAMES offline but publishes no types; aws ships 428 API models that describe the API rather than the CLI envelope; syft publishes 111 schemas but its own schema.url needs network and points at main.

How this was measured

So you can check it.

We used two methods on purpose. Probe: aoi‑probe runs the command and inspects the bytes. It makes no AOI assumptions, which is what lets it measure tools that don't conform, and every probed command is credential‑free and network‑free so the run reproduces anywhere. Docs: primary vendor documentation and source, for tools whose flagship mode needs a cluster, a daemon or a token.

7 of the 16 tools were probed as well as read; the other 9 come from documentation alone. Where both methods covered a tool they agreed, and each method reached the headline below on its own.

No surveyed tool has all four of {NDJSON, type discriminator, in-band format version, terminal event with an explicit success boolean}. Four have three of them: cargo, terraform, git trace2 and pytest-reportlog.

Two traps are worth naming, because they're the easiest errors to make with this data. Syft's schema promise doesn't extend to grype: grype's published schemas cover grype db search, never grype -o json. And cargo's “stable and versioned” promise covers cargo metadata, not the build stream, which carries no version at all.

The raw dataset is landscape.json, generated 2026-07-30. The counts, matrix and findings on this page are rendered from that file, so they can't drift from the evidence.

The specification these tools are measured against is AOI‑CLI. Why a standard makes the case that tools converging without agreeing is a problem worth fixing.