Docs / Record features

The model-call record

A model-call record is one canonical JSON document per model call, written as an ordinary output of the stage that made the call. It is evidence beside the stage receipt: it never changes the receipt format and never adds a member to the receipt or the chain. It never changes the reading of a run's receipts; its own verifier rows read as every other row does, and a FAIL on one fails the run like any other FAIL. A run that writes no record is byte-for-byte the run it would be without the feature.

It is written by onetrace.model_call, installed with onetrace[model-call]. The engine adapters import vLLM or transformers only when used, so the engine stays your own installed version; onetrace pins no engine. The transformers adapter is tested only where torch and transformers are installed (tests/test_model_call_capture.py, a tiny model built from a config on the CPU); its tests skip without them. CI runs them in a job of their own on a CPU-only torch, pinned by hash. The record's schema is docs/model_call_record.schema.json, labelled model-call-record/1.0.

What a record holds, and how honestly

Every scalar in a record is a string. Every field that describes the call is a value object with a status, a source, and — where there is one — a value. A field the writer cannot fill is written as not observed or not applicable, with its reason in source: never left out, and never given a default.

The five states:

Four fields are written by the helper itself, as plain strings with no state — they describe how the record was written, not the call: format, call.kind, call.api, and credential. credential is present or absent — never the key, its digest, or any part of it.

The record says only what it observed

The record is written so that a status is never a claim the writer did not make:

The fingerprints

fingerprints is a list of distribution digests. A transformers prefill records an exact form (the raw bytes of the last position's logits) and a tolerant form (the top-16 token ids of that position, rounded). A vLLM call records a topk_exact and a tolerant form over the first generated token's top-16 log-probabilities. A hosted call records a tolerant form over the provider's first-token log-probabilities, when the response carries at least 16.

A vLLM request that delivered no token has no first-token log-probabilities, and the vLLM fingerprint has no value defined for it. A 1.1 record writes both forms not observed, with the source RequestOutput.outputs[0] holds no generated token: no first-token log-probabilities to digest. A call with no capture still stops at such a request with an IndexError, as onetrace 0.6.1 does.

Several calls in one stage

The first record a stage writes is model_call.json, then model_call.2.json, model_call.3.json, and so on, in write order. The package refuses a name it has already used in a stage before it writes, so a repeat never reaches onetrace.

The capture levels, and the steps file

A call's answer — its chosen token ids — is the model's own output, not the operator's. The record never holds a token id or any text, at any capture level.

When a per-step capture (L1: the chosen token ids; L2: the top-k candidate ids) is taken, it is written as a separate output, model_call.<n>.steps.json, classed model-generated, and the record names it only by digest, in the optional steps field. The token ids live in that model-generated file; the operator-authored record holds a digest of it and nothing more.

A prefix-digest chain over the token ids is not a way to hide them: each step's digest, with the model's vocabulary, reveals the tokens to anyone with the vocabulary. The steps file is treated as the model's output, classed model-generated, for that reason.

The cost of taking L1 or L2 is not measured. L3 (a full per-step capture on the serving engine) is not measured.

model-call-record/1.1: the self-hosted steps file, from onetrace 0.7

This section gives six parts of model-call-record/1.1 for a self-hosted call, from design/model_call_l3.md and vLLM 0.30.0's source. This build writes 1.1 for a self-hosted vLLM call captured at L1, L2 or L3: capture.vllm's Engine.generate(..., level=..., ctx=...) writes each request's steps file and its record through the stage context. A call with no capture is written as 1.0, as before. onetrace-verify before 0.7 does not check a 1.1 record's content (docs/format.md). The record format is not part of the Internet-Draft: the draft describes the receipt and its chain, and the record formats sit outside it, as design/format_04_members.md says.

The steps file at L1, L2 and L3

model_call.<n>.steps.json, classed model-generated, as in 1.0: one JSON object, written canonically.

Member Type L1 L2 L3
level string "L1" "L2" "L3"
method string "token-ids" "top-k-ids" "sha256-tree-512/v0"
token_ids list of decimal strings yes yes yes
top_k_ids list of lists of decimal strings — yes yes
digests list of 64-character lowercase hex strings — — yes

The record's steps.* members

The record's format is model-call-record/1.1 wherever these members are written. A 1.0 record holding one of them is outside its format.

Member State Source Value
steps observed the steps file's name, model_call.<n>.steps.json the file's sha256:<hex>
steps.level observed the capture this adapter wrote L1, L2, L3
steps.method observed the capture this adapter wrote token-ids, top-k-ids, sha256-tree-512/v0
steps.count observed len(RequestOutput.outputs[0].token_ids) a decimal string
steps.k_requested declared, not observed, at L2 and L3 SamplingParams.logprobs as the caller built it, not read back from the engine the request's logprobs, so k is never more than sampling.logprobs
steps.k_returned_min, steps.k_returned_max observed, at L2 and L3; not applicable at steps.count "0" RequestOutput.outputs[0].logprobs: the ids ranked 1 to k at each step; at "0", no step: the request delivered no token the fewest and the most ids any step holds

top_k_ids[i] is the ids vLLM ranks 1 to k at step i. The entry vLLM adds for the sampled token when its rank is outside the top k is left out.

Where L3's digests are taken

steps.surplus

The model runner

engine.settings.model_runner, an engine setting under the open engine.settings. prefix, values V1 and V2:

The engine core

engine.settings.engine_core, an engine setting under the open engine.settings. prefix, values in the calling process and in another process:

The executor

engine.settings.executor, an engine setting under the open engine.settings. prefix: the engine's parallel_config.distributed_executor_backend as vLLM resolved it, uni, mp, ray, external_launcher, or a class given as <module>.<qualname>.

How the request ended

output.finish, the member 1.0 already has. For a self-hosted vLLM record at 1.1 its value is one of seven:

Value Read from
eos finish_reason stop, stop_reason none, or a token id that is one of the model's eos token ids and not one of the caller's stop_token_ids
stop-string finish_reason stop, stop_reason a string
stop-token finish_reason stop, stop_reason any other token id
length finish_reason length
abort finish_reason abort
repetition finish_reason repetition
error finish_reason error

The refusal at L3

L3 covers greedy decoding with nothing set that acts on the row. The adapter checks the engine and every request of the call before it submits any, and when one is not covered it refuses the whole call (L3Refused), naming each request and member in request order, each with its fix: line, and nothing runs:

L3 refused for request 2: SamplingParams.presence_penalty is 0.5; with it set, the digested row is not the row the token was chosen from
fix: ask for L2 for this call, or leave presence_penalty at its default
SamplingParams member Covered when Acts on the row
temperature 0 with any other value, the choice is not the argmax of the digested row
repetition_penalty, presence_penalty, frequency_penalty 1, 0 and 0 after the digest: the token is chosen from a row the digest did not cover
logit_bias, allowed_token_ids, bad_words unset before the digest: the digest covers a changed row, not the model's
min_tokens 0 before the digest, as logit_bias
thinking_token_budget unset after the digest, as the penalties
structured_outputs unset before the digest: vLLM applies the mask to the logits before it samples

An L3 record carries each of these as sampling.<name>, declared, not observed, source SamplingParams.<name> as the caller built it, not read back from the engine, at its value: 1.0, 0.0, 0, or none for an unset list or map. vLLM refuses structured outputs on an engine started with skip_tokenizer_init; the adapter checks it all the same. Each request's record carries its own request's members, whether the call is given one SamplingParams or one per request.

The engine is checked too, before any request:

Engine setting Refused when The line
the V1 pin the engine was not started by capture.vllm.start(level="L3") L3 refused: the engine was not started by capture.vllm.start(level="L3"), so the V1 model runner is not pinned
HiSparse attention attention_config.hisparse_config is set: it forces the V2 model runner over the pin L3 refused: the engine's attention_config.hisparse_config is set; …
a watermark watermark_config is set: it forces the V2 model runner over the pin L3 refused: the engine's watermark_config is set; …
the runner read back it reads V2 after the pin L3 refused: the engine's configuration reads the V2 model runner after the adapter's pin
the engine core its client is read as running it in another process: L3's digests stay there L3 refused: the engine core runs in another process (llm.llm_engine.engine_core is <class>), so L3's digests stay in that process, with fix: start the engine with capture.vllm.start(level="L3", ...) in the process environment, with nothing that sets VLLM_ENABLE_V1_MULTIPROCESSING to 1
the worker any parallel_config member below with another value, or none that can be read; any other parallel size above 1; the executor as built not read, or not the configured one: L3 is covered only where the engine's executor runs its worker in the calling process one line per member, L3 refused: the engine's parallel_config.<member> is <value>; …, with fix: ask for L2, or start the engine with <member> <value that runs here>; for the executor as built, the lines below
a logits processor any but L3's own that is not argmax-invariant, wherever it sits: one before L3's changes the row the digest covers, one after changes the row the token is chosen from L3 refused: engine logits processor <i>: <module>.<class>; before L3's digest …, with fix: ask for L2, or start the engine without it

What a one-GPU, text-only engine does not reach

On a one-GPU, text-only vLLM 0.30.0 engine, four things this page describes are not reachable. Each code path stays, with its test against the fake engine:

On a GPU of compute capability 12.x

On a GPU of compute capability 12.x (Blackwell), FlashInfer's compile step needs a CUDA toolchain of 12.9 or later, which a pip nvidia-cuda-nvcc provides. That is a requirement of the GPU and of FlashInfer, not of onetrace: the adapter compiles nothing. The GPU check met it on such a GPU, and set the toolchain up without changing vLLM.

sha256-tree-512/v0

Over a byte string m:

The input is the logits row's float32 values as they sit in memory, little-endian, in vocabulary order, with no copy, conversion or reordering. The empty input gives SHA-256 of the empty string; a logits row is never empty, and the SDK refuses an empty row before it digests one. The output is the final digest as 64 lowercase hexadecimal characters, with no prefix: everywhere else in a record, sha256:<hex> is the plain SHA-256 of the named bytes, and this digest is not.

def tree512(m: bytes) -> str:
    while len(m) > 512:
        m = b"".join(hashlib.sha256(m[i:i + 512]).digest() for i in range(0, len(m), 512))
    return hashlib.sha256(m).hexdigest()

The method's test vectors. The byte strings have byte i = i mod 251. The row is 151,936 float32 values, value i = float32(r.random() × 64 − 32) for r = random.Random(20261030), packed little-endian. tests/test_tree512.py checks each, through the reference in tests/tree512.py.

Input sha256-tree-512/v0
0 bytes e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
1 byte 6e340b9cffb37a989ca544e6bb780a2c78901d3fb33738768511a30617afa01d
512 bytes d86e386278a71782a283f96aae4f4e7437471abef71136bd2811f98245488d89
513 bytes af03b0360647e12bfa680604a52fe9216fd9304c071daa000254ab0650a613b3
1,024 bytes d926a38a6e800e3795e8acbf9c0842d279faf72f61af254f17ae4aecaeb70187
1,025 bytes bb8b24b99a2a4b500a9380ef88e9bf210fd25a84c48d78e59d926a911532ab83
the row (607,744 bytes) 458ded0881470ec3e41a77bfd901aacf699ccc3a6fc0694e370dedd44e3bb9a8

At 512 bytes and below the method equals plain SHA-256. The 1-, 513- and 1,025-byte inputs test the definition at a chunk edge; a float32 row never has those lengths.

Comparing two records

onetrace.model_call.compare(a, b) reads two records and returns one of three results, and nothing else:

A comparison names the mode it applied (1, 2 or 3) and what it rested on. It measures nothing. vLLM with batch-invariant mode not observed on, at a submitted batch size over 1, is not compared: the reason says that, and claims no property of such a configuration. The evidence for that condition is one measurement, named in full, applied to every such record with no other configuration having been measured.

When the two sides' per-step token-id sequences are given to compare and the result is diverged, the first differing step is added as a line of that diverged result — never as a fourth result. The three results stand.

Comparing two records is onetrace.model_call.compare: the verbs onetrace diff and onetrace localize do not read records in 0.6.0 or 0.6.1.

The verifier's rows

A verifier that reads a run gains rows only for a run that holds a record; a run with no record gains no row, and its output is unchanged. For each record the run holds, the rows read exactly as the receipt rows do — PASS, FAIL or NOT-RUN, never a guess, the matched text of a credential never echoed:

A model-call-record/1.1 record gains its own rows; a 1.0 record gains none:

ot.model_from(provisioning_run, stages=[...], serve_stage=...), passed as @ot.run(model=...), links a run to the run that fetched, optionally quantised and served its model — the same shape as the corpus link, one hop further. The link is the provisioning run's chain head, recorded on exactly the named steps; the provisioning run is verified when the query run starts, and refused if it does not verify, as a corpus link is. Following a model link one hop in a comparison, and cross-checking a record's model.files_digest against the provisioning run's own digest, are later work and are not done yet.

What this does not claim

The schema

docs/model_call_record.schema.json, labelled model-call-record/1.0, is the schema of 1.0, and docs/model_call_record_1_1.schema.json, labelled model-call-record/1.1, is the schema of 1.1. Each uses only the validation keywords a strict reader applies, so the reader can read it; it declares the optional steps capture member; x-groups groups the fields for a reader, with a group for the capture member; and x-most-bytes bounds a record's size, with its reason (the record holds digests and short value objects, and the per-step capture lives in a separate output, so the record does not grow with capture depth).

The 1.1 schema adds the capture's own members (steps.*), identity.digest and the call kind endpoint, and groups the capture's members together. It holds members and their value-object form; the states and values the 1.1 section gives are the verifier's rows. record.validate and the schema agree on both versions: a record the SDK writes passes both, and a member outside its version fails both.

The wheel carries a byte-identical copy of each as package data, onetrace/model_call/model_call_record.schema.json and onetrace/model_call/model_call_record_1_1.schema.json, readable with importlib.resources.files("onetrace.model_call").joinpath(<name>).