HERALD CT Log — Schema and Wire Format

Status: normative for HERALD v0.2 conformant CT logs.
Source: EXT §2 (audit-log specification), RFC 6962 (Certificate Transparency).
Distribution: as lib/herald-ct-log/SCHEMA.md in the HERALD repository; consumed by T-102 (basic CT log impl), T-103 (STH publication), T-104 (SIP generation), T-105 (consistency proofs), T-106 (LOTS template — already done).

1. Overview

A HERALD CT log is an append-only, publicly-readable Merkle tree whose leaves are signed records of submitted HERALD envelopes. Operators run logs; submitters submit envelopes; verifiers consult logs to confirm inclusion.

The structure is RFC 6962-derived. Differences from RFC 6962:

Leaves are HERALD log entries (envelope minimum-metadata) rather than X.509 certificates.
Domain-separation tags herald.audit_sth.v1, herald.audit_sct.v1, herald.audit_sip.v1 (per EXT §13).
Gossip is mandatory per [EXT §2.5].
LOTS (Log Operator Transparency Statement) is mandatory per [EXT §2.6] (template at T-106).
Minimum-metadata default per [EXT §2.7]; submitter opt-in for broader fields.

2. Log entry

2.1 Minimum profile (default)

A log entry under the minimum profile is a CBOR-deterministic structure (per SPEC §3.5) with exactly four fields:

LogEntry := {
  "id":          <bytes>,    # the envelope id, SPEC §3
  "parent_hash": <bytes>,    # SHA-256(envelope.parent) or 32 zero bytes for genesis
  "trace_hash":  <bytes>,    # SHA-256(envelope.trace)
  "hour_bucket": <uint64>    # Unix timestamp floored to the nearest 3600s boundary
}

Validation rules:

id MUST be exactly the envelope's content-addressed id field (typically a 256-bit hash).
parent_hash MUST be 32 bytes of zero for envelopes whose parent field is null (genesis envelopes); otherwise SHA-256 of the parent envelope's id bytes.
trace_hash MUST be SHA-256 of the trace envelope's id bytes.
hour_bucket is the envelope's ts floored to the hour: hour_bucket = (ts / 3600) * 3600.

No other fields. The minimum profile is exact; logs operating in minimum profile MUST reject submissions with log_metadata_consent declaring additional fields.

2.2 Broader-public profile

A log operating in broader-public profile per its LOTS MAY include additional fields, but MUST publish the field-acceptance policy at a well-known URL and MUST honour each submitter's log_metadata_consent exactly:

LogEntry (broader-public) := {
  "id":           <bytes>,
  "parent_hash":  <bytes>,
  "trace_hash":   <bytes>,
  "hour_bucket":  <uint64>,
  # Following fields MUST appear if and only if submitter opted in:
  "from":            <text, optional>,    # full sigil
  "to":              <text, optional>,    # full sigil(s)
  "ts":              <uint64, optional>,  # full Unix seconds
  "attestation_types": <[text], optional>, # types only, not values
  "size_bytes":      <uint64, optional>,
  "manifest_ref":    <bytes, optional>    # capability-manifest reference
}

A log MUST publish the per-entry profile in the SCT/SIP so verifiers know what fields to expect.

2.3 Private-compliance profile

A log operating in private-compliance profile MAY accept submissions with broader fields and with submitter pre-acknowledgement (per [EXT §2.7]). The acknowledgement mechanism is declared in LOTS §4.2.

Per [EXT §2.7], private compliance logs that ingest envelopes without submitter awareness are non-conformant.

2.4 Forbidden content

Regardless of profile, a log MUST NOT publish:

payload (envelope body content)
whisper.encrypted (HPKE ciphertext)
Any signature material beyond what's needed for proof verification
Any field not in the submitter's log_metadata_consent (where applicable)

A log that violates this rule is non-conformant and MUST be flagged via the misbehaviour gossip path ([EXT §2.5]).

3. Merkle tree construction

Per [EXT §2.1] and RFC 6962 §2:

Leaf hash: SHA-256(0x00 || cbor_canonical(LogEntry)).
Internal node: SHA-256(0x01 || left_child_hash || right_child_hash).
Empty tree: SHA-256(empty string) (per RFC 6962 §2.1).

The leaf-prefix 0x00 and internal-prefix 0x01 are domain-separation between leaves and inner nodes; they prevent second-preimage attacks on the tree shape.

The tree is append-only: leaves are added in submission order, never reordered or removed. The tree size monotonically increases.

4. Signed Tree Head (STH)

Per [EXT §2.3]. An STH is the operator's signed commitment to the log's state at a point in time.

4.1 STH structure

STH := {
  "tree_size":  <uint64>,    # number of leaves
  "root_hash":  <bytes>,     # 32 bytes, the Merkle root
  "timestamp":  <uint64>,    # Unix seconds at which the STH was minted
  "log_id":     <bytes>      # SHA-256 of the log operator's public key
}

4.2 STH signature

The STH is signed with domain-separation tag herald.audit_sth.v1:

sth_signature = Ed25519_sign(operator_privkey, "herald.audit_sth.v1" || cbor_canonical(STH))

The signed structure is:

SignedSTH := {
  "sth":       STH,
  "alg":       "ed25519",
  "sig":       <bytes, 64 bytes Ed25519 signature>,
  "scope":     "operator_attestation"
}

4.3 STH publication

A log MUST publish a fresh STH at least once per declared MMD (default 24h, typical 30min-2h).
Each STH supersedes prior STHs for the same log_id.
STHs are published at the well-known URL declared in LOTS §9.
STHs are gossiped per [EXT §2.5].

4.4 STH validation

A verifier receiving an STH MUST:

1. Look up the log_id's pubkey via the log's LOTS.
2. Verify the Ed25519 signature using herald.audit_sth.v1 domain separation.
3. Reject if tree_size = 0 and root_hash != SHA-256("") (consistency check).
4. Reject if timestamp is absurd (e.g. 0 or far-future).

5. Submission Certificate (SCT)

Per [EXT §2.4]. An SCT is the operator's promise that the submitted envelope will be merged within MMD.

5.1 SCT structure

SCT := {
  "envelope_id":  <bytes>,    # the submitted envelope's id
  "log_id":       <bytes>,    # SHA-256 of operator's pubkey
  "submitted_at": <uint64>,   # Unix seconds at submission
  "mmd_seconds":  <uint64>,   # the MMD applied to this submission
  "profile":      "minimum" | "broader-public" | "private-compliance"
}

5.2 SCT signature

The SCT is signed with herald.audit_sct.v1:

sct_signature = Ed25519_sign(operator_privkey, "herald.audit_sct.v1" || cbor_canonical(SCT))

The signed structure is:

SignedSCT := {
  "sct":      SCT,
  "alg":      "ed25519",
  "sig":      <bytes>,
  "scope":    "operator_attestation"
}

5.3 SCT semantics

An SCT does NOT prove inclusion. It promises that inclusion will follow within MMD.
A submitter holding an SCT can demonstrate "I submitted at time T to log L" but not "the envelope is in the tree."
If the log fails to merge by MMD, the submitter MUST receive an SIP-failure notice. The log is non-conformant.
Verifiers MAY accept an SCT in lieu of an SIP for envelopes within MMD of submission, with policy. State-transition envelopes (birth, molt, deprecation) MUST have an SIP, not just an SCT, before being treated as conformant per EXT §1.3.

6. Signed Inclusion Proof (SIP)

Per [EXT §2.4]. An SIP proves a specific envelope is included in the tree at a published STH.

6.1 SIP structure

SIP := {
  "envelope_id":  <bytes>,
  "log_id":       <bytes>,
  "leaf_index":   <uint64>,   # 0-indexed position in the tree
  "audit_path":   <[bytes]>,  # array of 32-byte hashes for path-to-root
  "sth":          STH         # the STH at which inclusion is proven
}

6.2 SIP signature

The SIP itself is not separately signed; the inclusion is proven cryptographically by the audit path, and the STH is signed (§4). The combined audit_path + sth is sufficient.

For convenience, logs MAY emit a wrapping signature:

SignedSIP := {
  "sip":      SIP,
  "alg":      "ed25519",
  "sig":      <Ed25519 signature over cbor_canonical(SIP) with tag herald.audit_sip.v1>,
  "scope":    "operator_attestation"
}

But verifiers MUST verify the inclusion mathematically (audit-path-to-root) regardless of the wrapping signature. The signature is convenience, not normative.

6.3 SIP verification

A verifier receiving an SIP MUST:

1. Compute leaf_hash = SHA-256(0x00 || cbor_canonical(LogEntry)).
2. Walk audit_path from the leaf to the root, applying internal-node hashing.
3. Confirm the resulting root equals sip.sth.root_hash.
4. Verify sip.sth per §4.4.
5. (Optional) Verify the wrapping signature.

If steps 1–4 succeed, the envelope is provably included at sip.sth.tree_size leaves.

7. Consistency proofs

Per [EXT §2.5]. A consistency proof shows that two STHs on the same log are consistent (one is a strict prefix-superset of the other).

7.1 Consistency proof structure

ConsistencyProof := {
  "log_id":      <bytes>,
  "first_size":  <uint64>,   # tree size at STH1
  "second_size": <uint64>,   # tree size at STH2 (must be ≥ first_size)
  "proof":       <[bytes]>   # array of 32-byte hashes per RFC 6962 §2.1.2
}

7.2 Consistency proof verification

Per RFC 6962 §2.1.2 algorithm. A verifier given two STHs and a consistency proof MUST:

1. If first_size == second_size and STH1.root_hash == STH2.root_hash, the trees are equal — accept.
2. If first_size > second_size, the proof is malformed — reject.
3. Else, run RFC 6962's PROOF verification algorithm.

If verification fails, the log has produced inconsistent STHs — the log is forked. Verifiers MUST:

Refuse to honour any inclusion proof from this log going forward.
Report the fork to the misbehaviour gossip path ([EXT §2.5]).
Per [EXT §2.5]: "Verifiers MUST reject envelopes whose only inclusion evidence comes from a log that has produced inconsistent STHs in the last 90 days."

8. Gossip

Per [EXT §2.5]. A log MUST participate in at least one gossip channel.

8.1 STH gossip

Channels (any one suffices for conformance; multiple recommended):

DNS TXT publication. STH digest at _herald-sth.<log-domain> (TXT record format below).
P2P gossip. Peer-to-peer exchange among monitors and conduits.
Cross-publication in unrelated logs. A HERALD CT log MAY include other logs' STH digests as opaque payloads.

DNS TXT record format:

"v=herald-sth-1; log_id=<base64url>; sth_root=<base64url>; sth_size=<uint>; sth_ts=<uint>; sig=<base64url>"

8.2 Trace-genesis gossip (per R-215)

If the operator implements R-215 trace-genesis pinning:

Query endpoint: /trace_genesis/<trace_id> returning all candidate genesis envelopes for that trace.
DNS TXT publication: _herald-trace.<trace_id>.<log-domain> (record format per R-215).

8.3 Misbehaviour reporting

The operator MUST consume gossiped misbehaviour reports and respond per LOTS §5.3 commitments.

9. Submission API

A HERALD CT log exposes a write API at the LOTS-declared endpoint:

POST /submit
Content-Type: application/cbor
Body: signed envelope per SPEC §3
Response: 200 OK + SignedSCT (CBOR), or
          4xx (rejection: malformed envelope, profile mismatch, etc.) + reason code, or
          429 (rate-limited; retry after Retry-After)

Rejection reason codes use the refusal reason-code registry (D-310) where applicable: envelope_invalid_signature, envelope_malformed, envelope_replay, policy_violation, rate_limit_exceeded.

10. Read API

A HERALD CT log exposes read APIs at the LOTS-declared endpoints:

GET /sth                           → SignedSTH (latest)
GET /sth/<tree_size>               → SignedSTH at the given size, or 404
GET /sip/<envelope_id>             → SignedSIP, or 202 (still pending) + Retry-After
GET /sip/<envelope_id>?at=<size>   → SignedSIP rooted at the STH for tree_size=<size>
GET /consistency/<size1>/<size2>   → ConsistencyProof
GET /entries?from=<idx>&to=<idx>   → JSON array of LogEntry within range, capped at 1000

These endpoints are HTTP/1.1 or HTTP/2; CBOR is preferred for binary endpoints. JSON responses are permitted for /entries for ease of monitor tooling.

11. Inclusion semantics

11.1 When is an envelope "in" the log?

An envelope is provably included in a HERALD CT log at tree size N when:

An SIP exists rooted at an STH with tree_size ≥ leaf_index + 1.
The STH's signature verifies.
The STH is consistent with all other STHs the verifier has observed for this log_id.

11.2 When is an envelope NOT in the log?

A HERALD CT log does NOT support negative inclusion proofs ("envelope X is not in the log"). This is consistent with RFC 6962 — append-only logs do not commit to absence.

To assert an envelope's absence, a verifier asks the log for /sip/<envelope_id> and expects 404 (not found) at the latest STH. The 404 is non-cryptographic; it could be a lie. For high-stakes absence claims, the verifier consults multiple gossiped STHs and confirms the latest is the one they queried against.

12. MMD compliance

Per [EXT §2.3]. The operator declares MMD in LOTS; the log MUST honour it.

12.1 Real-time MMD metrics

A log MUST publish at the LOTS-declared MMD-metrics URL:

{
  "log_id":               <bytes>,
  "window":               <text, e.g. "last_24h">,
  "submissions":          <uint>,
  "merged_within_mmd":    <uint>,
  "merged_after_mmd":     <uint>,    # non-conformant if > 0 in window
  "average_merge_time_s": <float>,
  "p99_merge_time_s":     <float>
}

A log with merged_after_mmd > 0 is non-conformant and the operator MUST publish a SEV-1 incident notice per LOTS §8.

12.2 SCT-SIP delivery contract

When a submitter receives an SCT, they SHOULD be able to retrieve a corresponding SIP within MMD. Specifically:

At any time T < SCT.submitted_at + SCT.mmd_seconds, a GET /sip/<envelope_id> request returns 202 (still pending) — acceptable.
At time T ≥ SCT.submitted_at + SCT.mmd_seconds, a GET /sip/<envelope_id> request MUST return either 200 (with SignedSIP) or signal a SEV-1 incident. 202 after MMD is non-conformant.

13. Misbehaviour catalogue

A log is non-conformant if it does any of:

Misbehaviour	Detection
Forked STHs (split-view attack)	Cross-gossip of STHs reveals divergent root_hash at same tree_size
Stale STHs (not republished)	STH series gap > MMD
Missing SIP after MMD	Submitter receives SCT, then 202 after MMD
Profile violation	Log entry contains fields beyond submitter's consent
Forbidden content publication	Log contains `payload` or `whisper.encrypted`
MMD breach	`merged_after_mmd > 0` in published metrics
Stale LOTS	LOTS signature date > 12 months
Missing gossip	No declared gossip channel observable
Unsigned STH	STH lacks `sth_signature` or signature fails

Each is reportable via the misbehaviour gossip path. Logs accumulate non-conformance signals; reputation systems and verifiers progressively distrust persistently-non-conformant logs.

14. Implementation roadmap

This schema is consumed by:

T-102 (basic log impl): implements the Merkle tree, the leaf-hashing rule, the submit/read APIs.
T-103 (STH publication): implements §4 STH minting and signing.
T-104 (SIP generation): implements §6 inclusion-proof generation.
T-105 (consistency proofs): implements §7 consistency proofs.
T-106 (LOTS template): done — provides the operator-transparency document this schema references.
T-115 (reference Herald): uses this schema for its own audit log.
T-116 (Bestiary), T-117 (inheritance): consume this schema's SIP/STH semantics for due-process and challenge-window enforcement.

15. Test vectors

Reference implementations MUST pass a conformance suite covering:

Minimum-profile encoding round-trip (T-001 vectors extended).
STH signature verification with herald.audit_sth.v1 domain separation.
SCT-SIP delivery within declared MMD (integration test).
Inclusion-proof verification (audit-path-to-root).
Consistency-proof verification (RFC 6962 §2.1.2 cases).
Forked-log detection (artificial divergent STHs).
Profile-mismatch rejection.

A sibling task (T-101-vectors or part of T-017 extension) should produce these.

16. Cross-references

[EXT §2] audit-log specification (the normative source)
[EXT §1.3] state-transition envelopes require SIP before authoritative
[EXT §2.6] LOTS — operator transparency document (template at T-106)
[EXT §2.7] minimum-metadata default and opt-in escalation
[EXT §13] domain-separation tags herald.audit_sth.v1, herald.audit_sct.v1, herald.audit_sip.v1
[SPEC §3.5] CBOR-deterministic canonicalisation profile
[SPEC §6] verification algorithm
[THREATS §4.4] split-view attack (the forked-STH defence)
[THREATS §4.14] genesis confusion (R-215 trace-genesis gossip)
[RFC 6962] Certificate Transparency v1
[RFC 6962-bis] Certificate Transparency v2 (informative)
T-106 (LOTS template), R-215 (genesis-pinning gossip)
D-307 SR-009 (audit-log centralisation systemic risk)
D-310 (refusal reason-code registry — used in §9 submission API)

— T-101 schema, by driver-claude, 2026-05-05.

Maintained at lib/herald-ct-log/SCHEMA.md in the HERALD repository.

HERALD CT Log — Schema and Wire Format