v1 Compatibility Contract

v1 Compatibility Contract

This page defines the public compatibility promises T4 intends to keep for the v1.x release line.

Anything described here is part of the v1 contract. Internal implementation details that are not described here may change between minor releases.

Go API

The exported github.com/t4db/t4 package API is stable across v1.x, including:

• Open, Node, Config, KeyValue, Event, watch options, transaction types, restore and branch types.
• Public errors: ErrKeyExists, ErrNotLeader, ErrClosed, ErrCompacted, and ErrConditionalWritesIgnored.
• Public enum/string constants for read consistency, follower wait mode, transaction conditions, transaction operations, and event types.

V1.x releases may add exported fields, methods, constants, or errors. They will not remove or change the meaning of existing exported API without a major version bump.

Expert hooks are stable enough to compile across v1.x, but their behavior follows the documented contract rather than every internal implementation detail:

• GCSStore, GCSConfig and NewGCSStoreFromConfig are the Google Cloud Storage backend, alongside S3Store. Its credentials come from Application Default Credentials unless CredentialsFile is set, which differs from the S3 path: that one consults no ambient chain unless Profile selects one. The difference is deliberate — on GCP the ambient chain is how a workload is given an identity — and is part of the contract for each backend rather than a shared rule.
• PebbleOptions customizes Pebble and may affect performance, durability, and compatibility with future Pebble versions.
• WAL/WALWriter is intended for constrained runtimes and tests. Custom implementations must preserve T4’s write-ahead durability semantics.
• Logger and MetricsRegisterer are supported embedding hooks.

Configuration

Existing Config fields and CLI flags are stable across v1.x. New fields and flags may be added, and their defaults preserve existing behavior unless the deviation is called out below.

Default values are part of the v1 contract unless explicitly documented as tuning guidance:

Setting	v1 default
ReadConsistency	linearizable
SegmentMaxSize	50 MiB
SegmentMaxAge	10 s
WALSyncUpload	true
CheckpointInterval	15 min
CheckpointEntries	0
AutoCompactMode	inferred; effectively off unless retention is set
AutoCompactRetention	0 (disabled)
AutoCompactRevisionRetention	0 (disabled)
AutoCompactInterval	AutoCompactSampleInterval when enabled
AutoCompactSampleInterval	clamp(retention/7, 1 min, 24 h) when enabled
NodeID	hostname, or node-0 if hostname lookup fails
AdvertisePeerAddr	PeerListenAddr
LeaderWatchInterval	20 s (see note below)
FollowerMaxRetries	5
FollowerWaitMode	quorum
PeerBufferSize	10,000
PeerBufferBytes	64 MiB (see note below)
WatchSendTimeout	30 s

PeerBufferBytes is a bound that did not exist before it was added, so its default does not reproduce the earlier behavior. The leader’s follower catch-up buffer was previously bounded by PeerBufferSize alone, and entries are retained with their values: 10,000 entries is tens of megabytes of small objects and gigabytes of large ones. The default now caps that buffer at 64 MiB, whichever bound is reached first. A cluster holding large values therefore keeps a shorter catch-up window than it used to, and a follower that falls outside it resyncs from the object store instead of replaying from the leader’s memory. Set it negative to restore the count-only bound.

LeaderWatchInterval changed meaning together with its default: it used to be how often the leader re-read the lock (default 5 min) and is now the leader’s lock renewal interval while it hears all followers (default 20 s, minimum 2 s). An explicitly configured value keeps working but now sets the renewal interval; values below 2 s are rejected.

Environment variables documented in Configuration remain supported across v1.x. Command-line flags take precedence over environment variables.

Local Data Directory

The local data directory contains:

<data-dir>/
  db/   Pebble database
  wal/  local WAL segment files

The directory is owned by T4. Operators should not edit files under db/ or wal/.

V1.x guarantees that a newer v1.x T4 binary can open a data directory created by an older v1.x T4 binary, subject to normal clean shutdown or WAL recovery. Downgrading to an older binary is best-effort and only supported when no newer persisted format has been written.

Pebble’s internal file format is not a T4 protocol. T4 may upgrade its Pebble dependency in a v1.x release if the new version can read existing v1 data directories.

Object Store Layout

When ObjectStore is configured, T4 uses the following keys under the configured prefix:

manifest/latest
checkpoint/<term>/<revision>/manifest.json
checkpoint/<term>/<revision>/<meta>
sst/<hash16>/<name>
wal/<term>/<first-revision>
leader-lock
branches/<id>

term is zero-padded to 10 decimal digits. revision and first-revision are zero-padded to 20 decimal digits. This keeps lexicographic order aligned with chronological order.

SST keys use the first 16 hex characters of the file’s SHA-256 digest:

sst/<first16hexOfSHA256>/<pebble-sst-filename>

Object Store Interface

Custom object stores must implement object.Store:

type Store interface {
Put(ctx context.Context, key string, r io.Reader) error
Get(ctx context.Context, key string) (io.ReadCloser, error)
Delete(ctx context.Context, key string) error
DeleteMany(ctx context.Context, keys []string) error
List(ctx context.Context, prefix string) ([]string, error)
}

List must return keys in lexicographic order. Delete and DeleteMany must not fail when the object is already absent. Put must be atomic from readers’ perspective.

For production multi-node use, stores should also implement object.ConditionalStore so leader election can use atomic conditional writes:

type ConditionalStore interface {
Store
GetETag(ctx context.Context, key string) (*GetWithETag, error)
PutIfAbsent(ctx context.Context, key string, r io.Reader) error
PutIfMatch(ctx context.Context, key string, r io.Reader, matchETag string) error
}

Point-in-time restore requires object.VersionedStore:

type VersionedStore interface {
Store
GetVersioned(ctx context.Context, key, versionID string) (io.ReadCloser, error)
}

Encrypted Object Format

When object-store encryption is enabled, object keys keep the same layout but each object body is replaced with an encrypted stream:

[4 bytes magic "T4E1"][1 byte key id][12 bytes object nonce]
repeated encrypted frames:
  [4 bytes plaintext length][ciphertext || 16 byte GCM tag]
final encrypted frame:
  [4 bytes zero length][16 byte GCM tag]

The encryption algorithm is AES-256-GCM. The logical object key, header, frame index, frame length, and final-frame bit are AEAD associated data. A reader must reject ciphertext copied to a different key, changed frame lengths, missing final frames, and authentication failures.

The key id byte is reserved for future key rotation. Current implementations write 0 and use the single configured key provider for all reads and writes.

Manifest Schema

manifest/latest is JSON:

{
  "format_version": 1,
  "checkpoint_key": "checkpoint/0000000001/00000000000000000100/manifest.json",
  "revision": 100,
  "term": 1,
  "last_wal_key": "wal/0000000001/00000000000000000050"
}

Fields:

Field	Type	Required	Notes
format_version	unsigned integer	no	Missing or 0 means format version 1 for backward compatibility.
checkpoint_key	string	yes	Object key of the latest checkpoint index.
revision	integer	yes	Highest revision included in the checkpoint.
term	unsigned integer	yes	Leader term that wrote the checkpoint.
last_wal_key	string	no	Last fully uploaded WAL segment whose last entry is covered by revision.

Checkpoint Index Schema

Each checkpoint index is stored at checkpoint/<term>/<revision>/manifest.json:

{
  "format_version": 1,
  "term": 1,
  "revision": 100,
  "sst_files": [
    "sst/0123456789abcdef/000001.sst"
  ],
  "ancestor_sst_files": [
    "sst/fedcba9876543210/000002.sst"
  ],
  "pebble_meta": [
    "CURRENT",
    "MANIFEST-000001",
    "OPTIONS-000002"
  ]
}

Fields:

Field	Type	Required	Notes
format_version	unsigned integer	no	Missing or 0 means format version 1.
term	unsigned integer	yes	Leader term that wrote the checkpoint.
revision	integer	yes	Highest revision included in the checkpoint.
sst_files	string array	yes	SST object keys in the checkpoint’s own store.
ancestor_sst_files	string array	no	SST object keys in the branch ancestor store. Only used by branch checkpoints.
pebble_meta	string array	yes	Pebble metadata filenames stored next to the checkpoint index.

Checkpoint Format Versions

Checkpoint format version 1 is the v1 baseline.

Format	Written when
1	The database holds no meta keyspace state.
2	The meta keyspace (see WAL Entry Frame) is non-empty. Older nodes must not restore it.

Writers use the lowest format that can represent the checkpoint.

Rules for v1.x:

• Adding optional JSON fields with omitempty is backward-compatible.
• Missing format_version is treated as version 1.
• An incompatible checkpoint format change must increment format_version.
• A node must reject checkpoint formats newer than it understands rather than silently restoring corrupt state.
• A newer v1.x node can read checkpoints written by older v1.x nodes.
• Downgrade is only safe if the newer binary has not written a checkpoint format the older binary does not understand.

Branch Registry Schema

Branch registry entries are stored at branches/<id> in the source object store:

{
  "ancestor_checkpoint_key": "checkpoint/0000000001/00000000000000000100/manifest.json"
}

Fields:

Field	Type	Required	Notes
ancestor_checkpoint_key	string	yes	Source checkpoint index key pinned by the branch.

The branch id is the object key suffix after branches/. It is not duplicated inside the JSON body.

Leader Lock Schema

The leader lock is stored at leader-lock:

{
  "node_id": "node-a",
  "term": 42,
  "leader_addr": "node-a.internal:3380",
  "last_seen_nano": 1775684807000000000,
  "committed_rev": 12345
}

Fields:

Field	Type	Required	Notes
node_id	string	yes	Stable id of the lock owner.
term	unsigned integer	yes	Monotonic leader term.
leader_addr	string	yes	Peer address followers use for WAL streaming.
last_seen_nano	integer	yes	Unix nanoseconds written by the leader during election or liveness touch.
committed_rev	integer	yes	Highest committed revision known to the leader when writing the lock. Used as an election fence.

The lock is updated with conditional object-store writes when the store supports ConditionalStore.

WAL Segment Names

Remote WAL segments are stored as:

wal/<term>/<first-revision>

Both components are zero-padded decimal strings:

wal/0000000001/00000000000000000042

WAL Segment Header

Each WAL segment starts with a 20-byte header. Integers are big-endian.

[4 bytes] magic "T4" <format-version byte> "\n"
[8 bytes] term uint64
[8 bytes] first_sequence int64

The third magic byte is the WAL format version. Writers open segments at format 2 (T4\x02\n) and raise the header to format 3 in place before appending the first entry that contains a meta op. Readers accept every format up to the newest they know and refuse newer segments (see Upgrade).

Format	Change
1	Original format.
2	Entries carry a WAL sequence id separate from revision, and an etcd-style version.
3	Same entry layout as 2. The segment may contain meta ops (codes 6, 7), standalone or in a transaction.

WAL Entry Frame

After the header, a segment is a sequence of framed entries. Integers are big-endian.

[4 bytes] payload_len uint32
[4 bytes] crc32c uint32 over payload
[N bytes] payload

Payload layout:

[1 byte ] op
[8 bytes] revision int64
[8 bytes] term uint64
[8 bytes] lease int64
[8 bytes] create_revision int64
[8 bytes] prev_revision int64
[8 bytes] id int64                 (format 2+)
[8 bytes] version int64            (format 2+)
[4 bytes] key_len uint32
[4 bytes] val_len uint32
[key_len bytes] key bytes
[val_len bytes] value bytes

id is the WAL sequence: it increases by one for every entry, including metadata-only entries such as compaction. revision is the user-visible revision and does not advance for metadata-only entries. In format 1 the sequence is the revision.

Operation codes:

Code	Operation
1	create
2	update
3	delete
4	compact
5	transaction
6	meta put
7	meta delete

For transaction entries, the outer entry has an empty key and stores encoded sub-operations in value.

Meta ops write the meta keyspace: unversioned node metadata stored next to the key-value data. They have no history, produce no watch events, and do not advance the revision; they carry the revision of the preceding data write and consume only a WAL sequence id, like compaction. A transaction consumes a revision only if it contains at least one create, update, or delete.

A reader that meets an op code it does not know must fail closed: it refuses to replay the segment or apply the entry, rather than guessing its meaning. Future releases may add op codes only together with a mechanism that keeps them away from binaries that cannot read them.

WAL Transaction Payload

Transaction payloads are encoded as:

[4 bytes] count uint32             (high bit set: sub-operations include version)
repeat count times:
  [1 byte ] op
  [8 bytes] lease int64
  [8 bytes] create_revision int64
  [8 bytes] prev_revision int64
  [8 bytes] version int64          (only when the high bit of count is set)
  [4 bytes] key_len uint32
  [4 bytes] val_len uint32
  [key_len bytes] key bytes
  [val_len bytes] value bytes

The sub-operation count is count & 0x7fffffff. Sub-operation op must be create, update, delete, meta put, or meta delete; any other code is rejected.

etcd Compatibility

The etcd v3 compatibility surface is documented in Migrating from etcd. Unsupported RPCs and unsupported transaction forms are part of the v1 contract unless that page is updated in a future minor release.

T4’s v1 goal is compatibility with the etcd v3 application surface: KV, Watch, Lease, Auth, and practical multi-key conditional transactions.

T4 does not aim to emulate etcd’s Raft cluster administration or maintenance internals. The following etcd features are intentional v1 non-goals:

Feature	v1 stance	Rationale
Alarm	Non-goal	etcd alarms are tied to etcd backend quota and corruption states. T4 exposes its own health and metrics model instead.
Snapshot	Non-goal	T4 uses S3 checkpoints, point-in-time restore, and branching rather than etcd snapshot files.
Hash	Non-goal	It hashes etcd’s whole storage backend. HashKV is supported, but its hash compares T4 nodes only, not T4 with etcd.
MemberAdd / MemberRemove / MemberUpdate / MemberPromote	Non-goal	T4 has no static Raft membership. Nodes join by sharing the object store and peer configuration.
MoveLeader	Non-goal	T4 has no Raft leadership-transfer operation. A future T4-native stepdown command may make sense, but not etcd-compatible MoveLeader.
Nested transactions	Non-goal	Rare in normal application clients and not needed for T4’s core compare/put/delete transaction model.
Range deletes inside transaction branches	Compatibility gap	Omitted from the primary v1 application surface. It can be revisited if real workloads require it.