API Reference

Opening a node

func Open(cfg Config) (*Node, error)

Opens or creates a node. Blocks until startup (checkpoint restore, WAL replay, leader election) is complete. Returns an error if the data directory is unusable or if the S3 store cannot be reached to read the manifest.

func (n *Node) Close() error

Seals the active WAL segment, stops background goroutines, and closes the database. Safe to call multiple times.

Write operations

All writes are rejected with ErrNotLeader if the node is a follower and the leader cannot be reached for forwarding.

// Put creates or replaces key unconditionally. Returns the new revision.
Put(ctx context.Context, key string, value []byte, lease int64) (int64, error)

// Create writes key only if it does not exist.
// Returns ErrKeyExists if the key is already present.
Create(ctx context.Context, key string, value []byte, lease int64) (int64, error)

// Update is a compare-and-swap on the key's modification revision.
// revision=0 makes the update unconditional.
// Returns (newRevision, previousKV, succeeded, error).
Update(ctx context.Context, key string, value []byte, revision, lease int64) (int64, *KeyValue, bool, error)

// Delete removes key. Returns revision=0 if key did not exist.
Delete(ctx context.Context, key string) (int64, error)

// DeleteIfRevision deletes key only if its current modification revision matches.
// Returns (newRevision, previousKV, succeeded, error).
DeleteIfRevision(ctx context.Context, key string, revision int64) (int64, *KeyValue, bool, error)

// Compact discards the history of all keys up to and including revision.
// The current value of every key is preserved.
Compact(ctx context.Context, revision int64) error

// Txn evaluates all Conditions atomically under the write lock.
// If all conditions pass, Success ops are applied; otherwise Failure ops are applied.
// All writes in the chosen branch are committed at a single revision — every key
// gets the same ModRevision.
// Returns ErrNotLeader if the node is a follower and the leader is unreachable.
Txn(ctx context.Context, req TxnRequest) (TxnResponse, error)

Transaction types

// TxnRequest is the input to Node.Txn.
type TxnRequest struct {
    // Conditions are evaluated atomically. All must pass for Success to run.
    Conditions []TxnCondition

    // Success ops are applied when all Conditions hold. May be nil (no-op check).
    Success []TxnOp

    // Failure ops are applied when any Condition fails. May be nil.
    Failure []TxnOp
}

// TxnResponse is returned by Node.Txn.
type TxnResponse struct {
    Succeeded   bool                // true if all Conditions passed
    Revision    int64               // revision of the write, or current revision for a no-op
    DeletedKeys map[string]struct{} // set of keys actually removed by the write
}

// TxnOp is one write operation in a transaction branch.
type TxnOp struct {
    Type  TxnOpType // TxnPut or TxnDelete
    Key   string
    Value []byte // used for TxnPut
    Lease int64  // used for TxnPut; must reference a live lease, or 0
}

// TxnOpType identifies the kind of operation within a transaction branch.
type TxnOpType uint8

const (
    TxnPut    TxnOpType = iota // upsert: create or replace key
    TxnDelete                  // unconditional delete
)

// TxnCondition is one predicate in the If clause.
type TxnCondition struct {
    Key    string
    Target TxnCondTarget // which metadata field to inspect
    Result TxnCondResult // comparison operator

    // Exactly one of the following is used, depending on Target:
    ModRevision    int64
    CreateRevision int64
    Version        int64  // 0 = key does not exist; see version note below
    Value          []byte
    Lease          int64
}

// TxnCondTarget identifies which field to compare.
type TxnCondTarget uint8

const (
    TxnCondMod     TxnCondTarget = iota // compare ModRevision (last write revision)
    TxnCondVersion                      // compare Version (write count; 0 = absent)
    TxnCondCreate                       // compare CreateRevision
    TxnCondValue                        // compare Value bytes
    TxnCondLease                        // compare Lease ID
)

// TxnCondResult is the comparison operator.
type TxnCondResult uint8

const (
    TxnCondEqual    TxnCondResult = iota
    TxnCondNotEqual
    TxnCondGreater
    TxnCondLess
)

Version note: T4 does not yet track a per-key write counter. TxnCondVersion supports only Version == 0 (key absent) and Version != 0 (key present). All other VERSION comparisons return an error. Use TxnCondMod for revision-based guards instead.

Read operations

Reads are always served from the local Pebble instance (leader or follower).

// Get returns the current value for key, or nil if the key does not exist.
Get(key string) (*KeyValue, error)

// List returns all live keys whose names begin with prefix, in lexicographic order.
List(prefix string) ([]*KeyValue, error)

// Count returns the number of live keys whose names begin with prefix.
Count(prefix string) (int64, error)

Watch

// Watch streams events for keys matching prefix using etcd semantics:
// startRev=0 means start from the current revision (no history replay),
// startRev=N means replay events from revision N (inclusive).
// The returned channel is closed when ctx is cancelled or the node shuts down.
Watch(ctx context.Context, prefix string, startRev int64) (<-chan Event, error)

Event

type Event struct {
    Type   EventType // EventPut or EventDelete
    KV     *KeyValue // key/value after the operation
    PrevKV *KeyValue // previous value, nil on first creation or if not available
}

type EventType int

const (
    EventPut    EventType = iota // key was created or updated
    EventDelete                  // key was deleted
)

Synchronisation

// WaitForRevision blocks until the node has applied at least rev, then returns nil.
// Returns ctx.Err() if the context is cancelled first.
WaitForRevision(ctx context.Context, rev int64) error

Useful when a follower needs to serve a read that is consistent with a write performed on the leader.

Introspection

CurrentRevision() int64  // highest applied revision
CompactRevision() int64  // compaction watermark (0 if never compacted)
IsLeader() bool          // true for leader and single-node roles
Config() Config          // returns the configuration used to open the node

KeyValue

type KeyValue struct {
    Key            string
    Value          []byte
    Revision       int64 // revision at which this value was written
    CreateRevision int64 // revision at which this key was first created
    PrevRevision   int64 // previous modification revision (0 if none)
    Lease          int64 // lease ID (0 = no lease)
}

Errors

var ErrKeyExists error  // Create: key already present
var ErrNotLeader error  // write on a follower when the leader is unreachable

Both are suitable for use with errors.Is.

Point-in-time restore (S3 versioning)

Note: this mechanism requires S3 versioning. For most use cases, prefer Branches which work without versioning.

A RestorePoint tells a node to bootstrap from a specific moment captured in S3 via object version IDs, rather than from the latest checkpoint. It is applied once, on first boot, and ignored on subsequent restarts.

type PinnedObject struct {
    Key       string // object key in S3
    VersionID string // S3 version ID of that object
}

type RestorePoint struct {
    // Store to read pinned objects from. Typically the source node's S3 prefix.
    // May differ from Config.ObjectStore (the new node's write prefix).
    Store object.VersionedStore

    // Checkpoint archive to restore from. If Key is empty, no checkpoint is
    // restored and WALSegments are replayed into a fresh database.
    CheckpointArchive PinnedObject

    // WAL segments to replay after the checkpoint, in ascending sequence order.
    WALSegments []PinnedObject
}

Set Config.RestorePoint to activate:

node, err := t4.Open(t4.Config{
    DataDir:      "/var/lib/t4-branch",
    ObjectStore:  branchStore,
    RestorePoint: &t4.RestorePoint{
        Store:             sourceStore,
        CheckpointArchive: t4.PinnedObject{Key: "...", VersionID: "..."},
        WALSegments: []t4.PinnedObject{
            {Key: "wal/000042.seg", VersionID: "..."},
        },
    },
})

object.VersionedStore

type VersionedStore interface {
    Store
    GetVersioned(ctx context.Context, key, versionID string) (io.ReadCloser, error)
}

object.S3Store satisfies this interface. S3 versioning must be enabled on the source bucket.

Branches

Branches let you fork a database at any checkpoint with zero S3 data copies. Shared SST files stay in the source prefix and are referenced by the branch’s checkpoint index.

Fork

func Fork(ctx context.Context, sourceStore object.Store, branchID string) (checkpointKey string, err error)

Reads the latest checkpoint manifest from sourceStore, registers the branch (writes branches/<branchID> to the source store), and returns the checkpoint key to pass to BranchPoint. Pass --checkpoint / a specific key if you want to fork from an earlier revision.

Unfork

func Unfork(ctx context.Context, sourceStore object.Store, branchID string) error

Removes the branch registry entry. The next GC cycle on the source can reclaim any SSTs no longer referenced by any live checkpoint or branch.

BranchPoint

type BranchPoint struct {
    // SourceStore is the object store of the database being forked.
    SourceStore object.Store

    // CheckpointKey is the manifest.json key returned by Fork.
    CheckpointKey string
}

Set Config.BranchPoint and Config.AncestorStore when starting a branch node for the first time:

sourceStore := object.NewS3Store(object.S3Config{Bucket: "my-bucket", Prefix: "prod/"})
branchStore := object.NewS3Store(object.S3Config{Bucket: "my-bucket", Prefix: "branch-a/"})

cpKey, err := t4.Fork(ctx, sourceStore, "branch-a")

node, err := t4.Open(t4.Config{
    DataDir:       "/var/lib/t4-branch-a",
    ObjectStore:   branchStore,
    AncestorStore: sourceStore,
    BranchPoint: &t4.BranchPoint{
        SourceStore:   sourceStore,
        CheckpointKey: cpKey,
    },
})

On first boot the node downloads the SSTs and Pebble metadata from sourceStore. On subsequent boots BranchPoint is ignored (the local data directory already exists).

When the branch is no longer needed:

if err := t4.Unfork(ctx, sourceStore, "branch-a"); err != nil {
    log.Fatal(err)
}