onetrace 0.2.0

The onetrace-textdiff/0.3 format

onetrace diff A B --text --json prints one JSON document describing the text that changed between two verified runs. The same document is written under text in the JSON report in --out for every --text run. The plain text --text prints is drawn from the same structure, so the two forms always agree.

The document is canonical JSON, the SDK's own form: keys sorted, no insignificant whitespace, UTF-8. Canonical JSON carries no numbers, so every count, index, position, line number and offset is a decimal string, and exit is too.

Top level

Field Meaning
format "onetrace-textdiff/0.3"
text_form "escaped": every text field is displayed text, escaped as below. It is never raw record text.
envelope diff's own result: identical, diverged, not comparable, refused or could not check
exit diff's exit code, as a string. --text never changes it.
shown false when no text is shown: a run didn't verify, or the runs can't be compared. reason then says which, and there are no nodes.
first_line the sentence the plain form prints first, e.g. first difference: retrieve — output text changed; settings unchanged; corpus link changed; input help_centre changed: the clauses that apply, in that order
unknown_stages --stage names that matched no node
nodes the nodes shown, in the ladder's order
corpus_traces present when a first-difference node's corpus link changed: the one-hop trace into the linked ingest runs, below. Absent otherwise. A first difference with no corpus link gets a line in the plain form only (trace: no corpus link recorded at <node>; …), and nothing here.

A node

Field Meaning
node, verdict the node id and its ladder verdict (FIRST DIFFERENCE, downstream, reconverged, same, or COULD NOT CHECK for a node --stage names)
settings the settings lines, e.g. settings: none changed or settings: 1 changed then constants.temperature: 0.2 -> 0.7. The corpus link is not counted as a setting: when it changed, corpus link changed: <digest> -> <digest> follows as a line of its own (with each run id once a trace reached it). A reconverged node carries its settings lines only if a setting or the link changed.
inputs one line per input whose digest or trust class changed, or that is on one side only, ending content not stored
causes on every FIRST DIFFERENCE node only: every candidate cause compared, each {kind, name, baseline, candidate, status} with status same or changed. Kinds are input (value {digest, trust_class}), tool (instrument.id, instrument.kind, instrument.version), setting (instrument.config_digest, instrument.manifest_digest, constants.<name>) and corpus link (assertions.corpus_manifest, by digest). The console's cause table is drawn from this list and nothing else.
presence when the node exists in one run only
outputs the outputs whose digests differ

An output

Field Meaning
name, media_type as the receipt records them
verified true when the bytes were read and their digest equals the receipt's. Otherwise false, with reason (not shown: <name> does not match its receipt, or … is missing), and no hunks.
kind text (a line diff: text, or JSON in canonical form), chunk index, hits, binary, not diffed (over the size or line cap), or not shown
presence only in the baseline / only in the candidate, when the output exists on one side only
notes lines that go with the output. With hunks: what the lines hide (final newline removed, line endings CRLF -> LF). Without: why there are none (binary: …, too large to diff (…), whitespace-only change: …), or, for chunk index and hits, the rendered lines themselves.
entries for chunk index and hits: one per chunk or hit that a notes line names, as the record holds it, not escaped (unlike notes, which are display lines): id, or, for a chunk with no id of its own, doc and its 1-based position; change (changed, added, removed, entered, left, moved); and note, the index of its line in notes. A cut output (truncated) keeps the entries of the lines it keeps, renumbered. New in 0.3.
hunks the line diff, below
truncated present when --max-lines cut the output: {"more_lines": "<n>"}, the same number the plain form prints in … n more lines. The JSON keeps exactly the lines the plain form still shows.

A hunk and its lines

A hunk is {baseline: {start, lines}, candidate: {start, lines}, lines: [...]}, with an optional note (no line could anchor this block …). Each line is:

Field Meaning
op " " unchanged, "-" removed, "+" added. (The plain form's ~ line, the changed words marked, isn't here: its content is the spans.)
baseline, candidate the line's 1-based number on each side, or null on the side it isn't on
text the line as displayed, escaped, and windowed if long
spans [start, end) character offsets into text: on a - line the removed words, on a + line the added ones

Line numbers:

Spans:

Long lines: a line over 2,000 characters is shown as a window of 2,000 characters, starting 40 characters before its first difference. It's written … N characters … <window> … M more characters, and spans count those markers.

Escaping ("text_form": "escaped")

Text is escaped one-to-one, the same way wherever the SDK shows record text:

  1. A backslash is doubled: \ becomes \\. This applies to text outputs. A JSON output's canonical form already writes its own backslashes as escapes, so there they are shown as written.

  2. Every control character and line break becomes \uXXXX, in lowercase hex. That means the Unicode categories Cc, Cf, Zl and Zp, fixed as of Unicode 15.0 so output never varies with the Python that runs it. Beyond U+FFFF the character is written as a surrogate pair (󠀁). The code points are:

    0000–001F 007F–009F 00AD 0600–0605 061C 06DD 070F 0890–0891 08E2 180E 200B–200F 2028–202E 2060–2064 2066–206F FEFF FFF9–FFFB 110BD 110CD 13430–1343F 1BCA0–1BCA3 1D173–1D17A E0001 E0020–E007F

    So TAB is \u0009, CR \u000d, ESC \u001b, and a right-to-left override ‮.

  3. An undecodable byte is \xNN (text outputs only). No \uXXXX can be mistaken for it.

  4. A lone surrogate that a JSON output's own \ud800 escape decodes to is \uXXXX.

A renderer displays text as given, HTML-escaped if it is HTML, and never escapes it again. Escaping it again would double every backslash, and spans index into text exactly as delivered.

A corpus trace

When a first-difference node's corpus link changed, the two linked ingest runs are found by chain head (one level down in --runs-root, default the folder holding the baseline), verified, and compared. corpus_traces is a list with one entry per distinct pair of chain heads, in the ladder's order. Each entry:

Field Meaning
node the first-difference node the trace starts from
also_at the other first-difference nodes with the same pair, which get no entry of their own
status traced, could not check, or not followed (--no-follow)
baseline, candidate each {chain_head, run_id, path, verified}. path is relative to --runs-root, written with /. A side that wasn't found has run_id, path and verified null. One that doesn't verify has run_id null and verified false. A side with no link has chain_head null too. With not followed, only chain_head.
reason with could not check: linked ingest run not found: <digest>; pass --runs-root, linked ingest run did not verify: <path>, the baseline records no corpus link at <node> (or the candidate), the linked ingest runs are not comparable: <why>, or the linked ingest runs could not be compared: <why>
heading traced into the linked ingest runs: <run id> -> <run id>
first_line e.g. started at: load — 1 of 214 documents changed (help-centre/warranty.md); settings unchanged
documents {changed, total, names}, counted from the chunk indexes by digest, or null when the runs have none
second_hop the ingest runs' own changed corpus link, {node, baseline, candidate}, named and never followed; otherwise null
envelope the ingest comparison's result, identical or diverged. A pair that isn't comparable, or whose comparison couldn't be made, is not traced: its entry is could not check, with the reason.
nodes the ingest comparison's nodes, shaped as above

Nothing from an ingest run that doesn't verify appears, its run id and document names included.

What stays the same

--text, --json and the corpus trace never change a verdict or an exit code. The inputs' content is never shown, because inputs are fingerprinted, not stored. Stored chunk or hit text is shown only when the digest of its bytes equals the recorded digest.