Skip to content

API.md

Latest commit

 

History

History
1639 lines (1332 loc) · 65.6 KB

API.md

File metadata and controls

1639 lines (1332 loc) · 65.6 KB
Raw

Docengine API reference

Docengine has two application-facing APIs:

  1. doc is the recommended high-level API. It uses only standard-library and doc types and separates read-only and read-write handles.
  2. document.Session is the lower-level application API. It exposes revision checks, storage policy, snapshot leases, event streams, derived views, compaction, and persistence state directly.

Use doc unless the application needs a capability that specifically requires Session-level control. This document describes only these two APIs; it is not an implementation-package reference. Generated declarations are available for doc and document.

1. doc: high-level application API

import "github.com/moresleep512/docengine/doc"

All offsets are zero-based byte offsets and ranges are half-open [start, end). Text inserted by a mutation must be valid UTF-8, and mutation boundaries must not split a UTF-8 encoding. Byte-oriented reads may start and end at any in-bounds byte.

Methods without a Context use context.Background. Their ...Context counterparts reject a nil Context and return its cancellation error when work can still be stopped safely.

1.1 Open and create

func Read(path string, options ...Option) (*Reader, error)
func ReadContext(ctx context.Context, path string, options ...Option) (*Reader, error)

func Open(path string, options ...Option) (*Document, error)
func OpenContext(ctx context.Context, path string, options ...Option) (*Document, error)

func New(path string, options ...Option) (*Document, error)
func NewContext(ctx context.Context, path string, options ...Option) (*Document, error)
API Parameters Result and use
Read / ReadContext path must identify an existing regular UTF-8 file. options configure this handle. Returns *Reader. The returned type has no edit, undo, redo, save, or maintenance methods.
Open / OpenContext Same path and option rules as Read. Returns *Document, which embeds all Reader methods and adds mutation.
New / NewContext path must not exist. WithCreateMode controls its permission bits. Exclusively creates an empty file and returns a writable *Document. It never truncates an existing file.

Opening validates the complete file before returning. If path is a symbolic link, the resolved target is fixed for the lifetime of the handle.

If New creates the file but a later open step fails, the error can be inspected as *CreateError:

var createErr *doc.CreateError
if errors.As(err, &createErr) && createErr.Created {
	// createErr.Path exists and was deliberately retained.
}

CreateError fields:

Field Meaning
Path Requested path that was created.
Operation Post-creation operation that failed, such as sync or open.
Created Always true for this error type.
Cause Original error. Unwrap exposes it to errors.Is/errors.As.

(*CreateError).Error() formats the failure. (*CreateError).Unwrap() returns Cause, including nil for a nil receiver.

Typical lifecycle:

document, err := doc.OpenContext(ctx, "notes.txt")
if err != nil {
	return err
}
defer document.Close()

result, err := document.InsertContext(ctx, 0, "Title\n")
if err != nil {
	return err
}
state, err := document.SaveRevisionContext(ctx, result.Revision)

1.2 Options and configuration

Option is an opaque configuration value. Applications obtain it through the following functions rather than implementing it:

func WithStorage(options StorageOptions) Option
func WithReadLimit(maximum int64) Option
func WithSearch(options SearchOptions) Option
func WithSearchIndex(options SearchIndexOptions) Option
func WithCoordinates(options CoordinateOptions) Option
func WithPages(options PageOptions) Option
func WithCreateMode(mode os.FileMode) Option
API Parameter meaning
WithStorage Sets recovery/runtime directories, ownership, resource limits, sync interval, and automatic checkpoint threshold.
WithReadLimit Sets the largest allocation made by one high-level Read. The default is DefaultMaxReadBytes (8 MiB); the hard maximum is MaximumReadBytes (1 GiB). It does not limit ReadAt or WriteTo.
WithSearch Sets the default query budgets used by Reader and Snapshot search helpers.
WithSearchIndex Enables optional persistent search acceleration when Directory is non-empty.
WithCoordinates Sets coordinate checkpoint and query-cache budgets.
WithPages Sets logical page, fragment, task, cache, in-flight, and window budgets.
WithCreateMode Sets only the permission bits used by New; it does not chmod an existing file.

StorageOptions:

Field Meaning
RecoveryDir Directory used for recoverable unsaved edits. Empty selects the Session default.
RuntimeDir Directory used for this handle's temporary runtime state. Empty selects a private Session default.
RecoveryOwnership Whether an explicitly supplied recovery directory is shared or removable when empty.
RuntimeOwnership Same policy for the runtime directory.
Limits Hard per-handle limits described below.
JournalSyncInterval Background recovery durability cadence. Zero selects the bounded default.
AutoCheckpointBytes Automatically save/checkpoint after recovery state reaches this size. Zero disables automatic checkpoints.

Ownership values:

Value Meaning
OwnershipDefault Let Docengine resolve policy from whether a directory was supplied.
OwnershipShared Never remove the directory.
OwnershipOwned Docengine may remove the directory only when it is empty and recognized as owned runtime state.

Limits:

Field Limits
MaxBatchEdits Operations accepted by one Apply.
MaxInsertBytes UTF-8 bytes accepted by one edit's Text.
UndoBytes Disk budget for retained undo/redo text.
MaxJournalBytes Hard recovery-state size limit.
EventHistory Events retained for resumable streams.
ChangeHistory Revision transitions retained for Changes and transforms.
MaxAnchorBatch Anchors or ranges accepted by one transform.
MaxSnapshotLeases Live Snapshot, Coordinates, and Pager leases.
MaxSubscriptions Concurrent event streams.

Zero limit fields select bounded defaults. Invalid option combinations return ErrInvalidOption.

Reader.Config() returns:

type Config struct {
	MaxReadBytes int64
	CreateMode   os.FileMode
	Search       SearchOptions
	SearchIndex  SearchIndexOptions
	Coordinates  CoordinateOptions
	Pages        PageOptions
	Storage      StorageConfig
}

StorageConfig contains the resolved absolute directories, resolved ownership, effective limits, sync interval, and checkpoint threshold actually used by the open handle.

1.3 State, modes, and errors

Reader.State() returns an immutable State:

Field Meaning
Mode ModeReadOnly or ModeReadWrite.
Status Current lifecycle state.
Path Absolute path requested by the caller.
ResolvedPath Fixed real target used for persistence.
Name Base file name.
Bytes Current logical byte length, excluding an on-disk BOM.
Revision Current logical revision.
SavedRevision Highest revision committed to the base file.
Dirty Current revision is newer than SavedRevision.
Recovered Unsaved edits were restored while opening.
HasBOM Base file uses a UTF-8 BOM; saves preserve it.
EOL EOLNone, EOLLF, EOLCRLF, or EOLMixed.
DurabilityUncertain Replacement committed, but directory durability was not confirmed.
RecoveryDurabilityUncertain Latest recoverable edits could not be confirmed durable.
Fault Cause of a permanent read-only persistence fault.

Status values:

Value Meaning
StatusReady Healthy and equal to the saved revision.
StatusDirty Healthy with unsaved edits.
StatusFaulted Reads remain available; mutation and save are rejected.
StatusClosing Close was requested and cleanup is in progress.
StatusClosed Close barrier completed.

Use errors.Is with package errors:

Error Meaning
ErrInvalidOption Invalid option, provider, enum, or nil writer.
ErrInvalidQuery Invalid search query.
ErrInvalidRange Negative, inverted, out-of-bounds, or incompatible range.
ErrReadLimit One allocating read exceeds its configured maximum.
ErrNotRegular New created a path that is not a regular file.
ErrInvalidContext Nil Context.
ErrClosed Handle or owned value no longer accepts the operation.
ErrFaulted Handle is permanently read-only after a persistence fault.
ErrNothingToUndo / ErrNothingToRedo No retained transaction in that direction.
ErrInvalidUTF8 File or inserted text is not valid UTF-8.
ErrInvalidUTF8Boundary A mutation boundary splits a rune encoding.
ErrExternalChange The target file changed outside this handle.
ErrConflict Revision expectation cannot be met.
ErrHistoryExpired Required revision transition is no longer retained.
ErrRevisionUnavailable Requested revision is future or not an observable boundary.
ErrLimitExceeded Configured work/storage limit was exceeded.
ErrEventCursor Event resume cursor is invalid.
ErrStale Revision, generation, lineage, or source identity does not match.
ErrNotFound Requested fragment, mapping, or item does not exist.
ErrBudgetExceeded One page/window request exceeds its budget.
ErrBusy Live work or leases prevent the operation.
ErrUnavailable Optional configured capability is unavailable.

Errors can join a normalized category with an underlying cause. Do not compare error strings.

1.4 Reader: current-document read API

Reader represents an existing document with read-only API permissions. Document embeds it, so every method below is also available on a Document.

State and configuration

func (r *Reader) State() State
func (r *Reader) Config() Config
func (r *Reader) Stats() Stats
  • State reports content, revision, persistence, and lifecycle state.
  • Config reports effective configuration.
  • Stats reports in-memory diagnostics and remains available after close. It performs no filesystem I/O.

Read methods

func (r *Reader) Read(start, end int64) ([]byte, error)
func (r *Reader) ReadContext(ctx context.Context, start, end int64) ([]byte, error)
func (r *Reader) ReadAt(target []byte, offset int64) (int, error)
func (r *Reader) WriteTo(writer io.Writer) (int64, error)
func (r *Reader) WriteToContext(ctx context.Context, writer io.Writer) (int64, error)
Parameter Meaning
start, end Half-open byte range copied by Read. The range must be within one current immutable revision.
target Caller-owned destination for ReadAt.
offset First byte copied by ReadAt.
writer Destination for a complete streamed revision. Must not be nil.
  • Read returns one allocated, revision-consistent copy and enforces MaxReadBytes.
  • ReadAt follows io.ReaderAt conventions. Separate calls can observe different revisions; use Snapshot for a multi-call consistent view.
  • WriteTo captures one immutable current revision, streams it in bounded chunks, and releases the temporary Snapshot automatically. Its byte count can be non-zero when the writer later fails.

Snapshot and derived views

func (r *Reader) Snapshot() (*Snapshot, error)
func (r *Reader) SnapshotContext(ctx context.Context) (*Snapshot, error)
func (r *Reader) Coordinates() (*Coordinates, error)
func (r *Reader) CoordinatesContext(ctx context.Context) (*Coordinates, error)
func (r *Reader) Pages() (*Pager, error)
func (r *Reader) PagesContext(ctx context.Context) (*Pager, error)

Each call captures the current revision. The returned value owns a bounded lease and must be closed.

Search methods

func (r *Reader) Search(text string) (SearchResult, error)
func (r *Reader) SearchContext(ctx context.Context, text string) (SearchResult, error)
func (r *Reader) SearchFold(text string) (SearchResult, error)
func (r *Reader) SearchFoldContext(ctx context.Context, text string) (SearchResult, error)
func (r *Reader) Find(query SearchQuery, options SearchOptions) (SearchResult, error)
func (r *Reader) FindContext(ctx context.Context, query SearchQuery, options SearchOptions) (SearchResult, error)
func (r *Reader) CollectSearchIndex() error
func (r *Reader) CollectSearchIndexContext(ctx context.Context) error
  • Search performs an exact literal query.
  • SearchFold uses Unicode simple-fold matching.
  • Find accepts an explicit literal, regular-expression, or term query and per-call budgets.
  • CollectSearchIndex removes unused retired persistent generations. It returns ErrUnavailable when no index directory is configured.

All search methods bind their result to one immutable revision.

Change history

func (r *Reader) Changes(fromRevision, toRevision uint64) (Change, error)
func (r *Reader) TransformAnchors(fromRevision, toRevision uint64, anchors []Anchor) ([]Anchor, error)
func (r *Reader) TransformAnchorsContext(ctx context.Context, fromRevision, toRevision uint64, anchors []Anchor) ([]Anchor, error)
func (r *Reader) TransformRanges(fromRevision, toRevision uint64, values []Range) ([]Range, error)
func (r *Reader) TransformRangesContext(ctx context.Context, fromRevision, toRevision uint64, values []Range) ([]Range, error)
  • fromRevision and toRevision must be observable retained boundaries.
  • A reverse request returns the inverse transformation.
  • Batch transforms preserve input order and return no partial result on error.
  • Context variants perform cancellable preflight work; the retained history lookup itself is in memory.

Events and virtualization refresh

func (r *Reader) Events(options EventOptions) (*Events, error)
func (r *Reader) RefreshPages(previous *Pager, provider FragmentProvider) (*Pager, error)
func (r *Reader) RefreshPagesContext(ctx context.Context, previous *Pager, provider FragmentProvider) (*Pager, error)
  • Events creates a bounded stream described in section 1.10.
  • RefreshPages creates a new Pager for the Reader's current revision using previous lineage and resource policy. provider may be nil when only logical pages are required. The previous Pager remains usable.

Close

func (r *Reader) Close() error
func (r *Reader) CloseContext(ctx context.Context) error

Close is idempotent. It stops new handle operations and begins one shared cleanup barrier. If CloseContext returns because its Context expired, cleanup continues and a later Close observes the same final result. Existing Snapshots remain usable until separately closed.

1.5 Document: mutation and persistence API

Document embeds Reader and adds:

func (d *Document) Write(text string) (Result, error)
func (d *Document) WriteContext(ctx context.Context, text string) (Result, error)
func (d *Document) Replace(start, end int64, text string) (Result, error)
func (d *Document) ReplaceContext(ctx context.Context, start, end int64, text string) (Result, error)
func (d *Document) Insert(offset int64, text string) (Result, error)
func (d *Document) InsertContext(ctx context.Context, offset int64, text string) (Result, error)
func (d *Document) Delete(start, end int64) (Result, error)
func (d *Document) DeleteContext(ctx context.Context, start, end int64) (Result, error)
func (d *Document) Apply(edits ...Edit) (Result, error)
func (d *Document) ApplyContext(ctx context.Context, edits []Edit) (Result, error)
func (d *Document) Undo() (Result, error)
func (d *Document) UndoContext(ctx context.Context) (Result, error)
func (d *Document) Redo() (Result, error)
func (d *Document) RedoContext(ctx context.Context) (Result, error)
func (d *Document) Save() (State, error)
func (d *Document) SaveContext(ctx context.Context) (State, error)
func (d *Document) SaveRevision(revision uint64) (State, error)
func (d *Document) SaveRevisionContext(ctx context.Context, revision uint64) (State, error)
func (d *Document) Maintain(options MaintenanceOptions) (MaintenanceResult, error)
func (d *Document) MaintainContext(ctx context.Context, options MaintenanceOptions) (MaintenanceResult, error)

Mutation parameters:

Parameter Meaning
text Valid UTF-8 inserted or used as complete replacement content.
start, end Current half-open byte range. Both must be UTF-8 boundaries.
offset Current UTF-8 byte boundary for insertion.
edits Ordered edits applied sequentially. Each edit sees the state produced by its predecessors.

Write replaces the entire current document. Replace, Insert, and Delete are single-edit conveniences. Apply publishes all edits atomically or none of them and records the complete batch as one undo unit. Each Edit element advances the revision, including an element whose replacement leaves the bytes unchanged. An empty edit slice is an identity operation and creates no undo entry. Result.Revision is the final revision after the batch.

result, err := document.Apply(
	doc.Edit{Start: 0, End: 0, Text: "prefix "},
	doc.Edit{Start: 7, End: 12, Text: "value"},
)

Edit fields:

Field Meaning
Start, End Sequential half-open range to replace.
Text Replacement UTF-8 text.

Result fields:

Field Meaning
Revision Final current revision.
Bytes Final logical byte length.
Dirty Whether final content is newer than the saved revision.
Change Complete length-only transformation produced by the operation.

Undo reverses the newest retained transaction. Redo reapplies the newest undone transaction. Both publish new revisions. A new edit after undo discards the redo branch.

Save commits the latest selected revision. SaveRevision(revision) guarantees that the committed revision is at least revision; it can commit a newer revision that became current before selection. Always inspect the returned State on error because replacement or another commit step may already have succeeded.

Maintain reclaims internal edit/undo storage without changing document content or revision:

type MaintenanceOptions struct {
	Checkpoint bool
}

When Checkpoint is true, maintenance also commits the current revision before rebasing recoverable state.

MaintenanceResult fields:

Field Meaning
OperationID Correlates maintenance events.
State State observed at return.
PiecesBefore, PiecesAfter Structural item counts before and after.
UndoBytesBefore, UndoBytesAfter Retained undo storage before and after.
JournalCheckpointed A persistence checkpoint completed.
Committed Maintenance publication occurred even if cleanup later failed.

1.6 Snapshot: fixed-revision API

A Snapshot remains tied to one revision even when the parent handle changes or closes.

func (s *Snapshot) Revision() uint64
func (s *Snapshot) Bytes() int64
func (s *Snapshot) Read(start, end int64) ([]byte, error)
func (s *Snapshot) ReadContext(ctx context.Context, start, end int64) ([]byte, error)
func (s *Snapshot) ReadAt(target []byte, offset int64) (int, error)
func (s *Snapshot) WriteTo(writer io.Writer) (int64, error)
func (s *Snapshot) WriteToContext(ctx context.Context, writer io.Writer) (int64, error)
func (s *Snapshot) Search(text string) (SearchResult, error)
func (s *Snapshot) SearchContext(ctx context.Context, text string) (SearchResult, error)
func (s *Snapshot) SearchFold(text string) (SearchResult, error)
func (s *Snapshot) SearchFoldContext(ctx context.Context, text string) (SearchResult, error)
func (s *Snapshot) Find(query SearchQuery, options SearchOptions) (SearchResult, error)
func (s *Snapshot) FindContext(ctx context.Context, query SearchQuery, options SearchOptions) (SearchResult, error)
func (s *Snapshot) Coordinates() (*Coordinates, error)
func (s *Snapshot) CoordinatesContext(ctx context.Context) (*Coordinates, error)
func (s *Snapshot) Pages() (*Pager, error)
func (s *Snapshot) PagesContext(ctx context.Context) (*Pager, error)
func (s *Snapshot) Close() error
  • Revision and Bytes describe the immutable view.
  • Read enforces the parent handle's allocation limit.
  • ReadAt implements io.ReaderAt.
  • WriteTo streams the complete Snapshot and is not restricted by the allocating read limit.
  • Snapshot search is always streaming against this exact revision; it does not switch to a newer persistent generation.
  • Snapshot Coordinates and Pager own additional references and remain usable after the Snapshot itself closes.
  • Close is idempotent and releases the Snapshot's own reference.

1.7 Search API

Query constructors:

func Literal(text string, mode SearchCase) SearchQuery
func Regexp(pattern string) SearchQuery
func Term(text string) SearchQuery

SearchKind values:

Value Meaning
SearchLiteral Literal byte/rune text matching.
SearchRegexp Go RE2 regular expression.
SearchTerm Query terms produced by the configured Analyzer.

SearchCase values are SearchExact and SearchUnicodeFold.

SearchQuery contains Kind, Text, and Case. Prefer the constructor functions so invalid combinations are less likely.

SearchOptions:

Field Meaning
ReadBufferBytes Streaming scan buffer.
MaxScannedBytes Maximum source bytes examined.
MaxPatternBytes Maximum query/pattern size.
MaxResults Maximum returned matches.
MaxCandidates Maximum candidate blocks verified when an index is used.
DisableFallback Reject a query that cannot use the configured index instead of scanning.
SourceKey Opaque source identity copied into each Match; empty uses the document path.

SearchIndexOptions:

Field Meaning
Directory Persistent index directory. Empty disables persistence.
BlockBytes, OverlapBytes Logical candidate block and overlap sizes.
MaxBlocks Maximum indexed blocks.
MaxAnalyzerTokensPerBlock Per-block Analyzer token limit.
MaxAnalyzerTermBytes Maximum UTF-8 bytes in one Analyzer term.
Analyzer Optional deterministic term policy.
Require Make persistent acceleration mandatory instead of allowing fallback.

Analyzer:

type Analyzer interface {
	Identity() string
	RequiredOverlapBytes() int64
	Analyze(context.Context, []byte) ([]Token, error)
	QueryTerms(context.Context, string) ([]string, error)
}
  • Identity must change whenever deterministic Analyzer behavior changes.
  • RequiredOverlapBytes declares context needed across block boundaries.
  • Analyze returns Token{Term, Start, End} values relative to the supplied block.
  • QueryTerms converts a Term query into candidate terms.

SearchResult:

Field Meaning
Revision, Generation Exact content and index generation used.
Matches Ordered Match values with revision, [Start, End), and SourceKey.
Complete True only when no configured budget truncated the result.
Incomplete Bit set of truncation reasons.
UsedIndex Persistent candidates were used.
ScannedBytes Source bytes examined during verification/scan.
CandidateBlocks Candidate blocks considered.

IncompleteReason bits are IncompleteScanLimit, IncompleteResultLimit, IncompleteCandidateLimit, and IncompleteFallbackDisabled. Use result.Incomplete.Has(reason) because more than one bit may be set.

result, err := reader.FindContext(
	ctx,
	doc.Regexp(`\bitem-[0-9]+\b`),
	doc.SearchOptions{MaxResults: 100, MaxScannedBytes: 64 << 20},
)
if err != nil {
	return err
}
if !result.Complete {
	// Decide whether to increase a budget or present a partial result.
}

1.8 Changes, anchors, and coordinate API

Change describes a sequential document-length transformation:

Field Meaning
BeforeRevision, AfterRevision Observable revision boundaries.
BeforeBytes, AfterBytes Lengths at those boundaries.
Edits Ordered ChangeEdit values.

ChangeEdit contains Start, OldLength, and NewLength; it deliberately does not contain inserted text.

Anchor combines Offset with:

  • AffinityBefore: remain before text inserted at exactly this boundary.
  • AffinityAfter: move after text inserted at exactly this boundary.

Range contains independently affine Start and End anchors.

Coordinates methods:

func (c *Coordinates) Position(offset int64) (Position, error)
func (c *Coordinates) PositionContext(ctx context.Context, offset int64) (Position, error)
func (c *Coordinates) Byte(line, column int64) (int64, error)
func (c *Coordinates) ByteContext(ctx context.Context, line, column int64) (int64, error)
func (c *Coordinates) RuneByte(offset int64) (int64, error)
func (c *Coordinates) RuneByteContext(ctx context.Context, offset int64) (int64, error)
func (c *Coordinates) LineStart(line int64) (int64, error)
func (c *Coordinates) LineStartContext(ctx context.Context, line int64) (int64, error)
func (c *Coordinates) Refresh() (*Coordinates, error)
func (c *Coordinates) RefreshContext(ctx context.Context) (*Coordinates, error)
func (c *Coordinates) Stats() CoordinateStats
func (c *Coordinates) Close() error
API Parameters and result
Position Converts a byte offset to Position{Byte, Rune, Line, Column}. Line and Column are zero-based; Column counts runes.
Byte Converts a zero-based line and rune-column to a byte offset.
RuneByte Converts a zero-based document rune offset to a byte offset.
LineStart Returns the byte offset for a zero-based line.
Refresh Builds a new Coordinates view for the owning Reader's current revision. It requires retained change history; the old view remains usable.
Stats Returns revision, byte/rune/line counts, checkpoint/reuse/scan counts, cache use, and closed state.
Close Releases the owned revision lease.

CoordinateOptions fields are CheckpointBytes, CacheBytes, and DisableCache. DisableCache cannot be combined with a non-zero cache size.

CoordinateStats fields:

Field Meaning
Revision, Bytes, Runes, Lines Identity and totals of the indexed Snapshot.
Checkpoints, CheckpointBytes Number and configured spacing of decoding checkpoints.
ReusedCheckpoints, ScannedBytes Incremental-build reuse and bytes scanned to produce this index.
CacheBytes, CacheEntries, MaxCacheBytes Current query-cache use and limit.
CacheHits, CacheMisses Query-cache counters.
Closed Whether this Coordinates handle has released its lease. 
coordinates, err := reader.CoordinatesContext(ctx)
if err != nil {
	return err
}
defer coordinates.Close()

position, err := coordinates.PositionContext(ctx, byteOffset)

1.9 Event and statistics API

Open a stream with:

type EventOptions struct {
	Buffer        int
	AfterSequence uint64
	FutureOnly    bool
}

func (r *Reader) Events(options EventOptions) (*Events, error)
func (e *Events) Next() (Event, error)
func (e *Events) NextContext(ctx context.Context) (Event, error)
func (e *Events) Close() error
  • Buffer == 0 selects the default bounded delivery buffer.
  • AfterSequence replays retained events strictly after a previously observed sequence.
  • FutureOnly skips retained history and cannot be combined with AfterSequence.
  • Next waits for one event and returns io.EOF when the stream or parent document closes.
  • Close detaches only this stream.

Event fields:

Field Meaning
Sequence Global monotonic event sequence.
Dropped Events omitted for this slow stream before this delivery.
Kind Transition kind.
Origin OriginApply, OriginUndo, or OriginRedo for changes.
State State at the transition.
Change Length-only transformation for a content event.
Persistence Save operation ID, target revision, byte progress, and commit state.
Maintenance Maintenance operation ID, byte/item progress, checkpoint, and commit state.
Virtualization Fragment publication/refresh progress.
Cause Failure or durability cause when applicable.

EventKind values:

  • lifecycle: EventOpened, EventRecovered, EventChanged, EventClosed;
  • save: EventSaveStarted, EventSaveProgress, EventSaved, EventSaveFailed;
  • recovery durability: EventJournalSyncFailed, EventJournalSyncRestored;
  • maintenance: EventMaintenanceStarted, EventMaintenanceProgress, EventMaintained, EventMaintenanceFailed;
  • virtualization: EventVirtualizationStarted, EventVirtualizationProgress, EventVirtualizationCompleted, EventVirtualizationFailed.

ChangeOrigin values are OriginNone, OriginApply, OriginUndo, and OriginRedo.

PersistenceProgress and MaintenanceProgress provide the exact fields used by save and maintenance events:

Type Fields
PersistenceProgress OperationID, TargetRevision, CompletedBytes, TotalBytes, Committed.
MaintenanceProgress OperationID, CompletedBytes, TotalBytes, PiecesBefore, PiecesAfter, JournalCheckpointed, Committed.

VirtualProgressKind values are VirtualPublish and VirtualRefresh. VirtualProgressStage values are VirtualStarted, VirtualAdvanced, VirtualCompleted, and VirtualFailed. VirtualProgress reports OperationID, Kind, Stage, Revision, BaseGeneration, PublishedGeneration, Bytes, IndexedThrough, Fragments, Complete, Published, and Cause.

events, err := reader.Events(doc.EventOptions{FutureOnly: true})
if err != nil {
	return err
}
defer events.Close()

event, err := events.NextContext(ctx)
if err == nil && event.Dropped != 0 {
	// Rebuild derived state instead of applying only event.Change.
}

Reader.Stats() returns:

type Stats struct {
	Recovery  RecoveryStats
	Lifecycle LifecycleStats
	Events    EventStats
	History   HistoryStats
}
Type Fields and meaning
RecoveryStats JournalBytes and MaxJournalBytes report current and maximum recovery storage. AutoCheckpointBytes and NextAutoCheckpointBytes report checkpoint thresholds. AutomaticCheckpoints counts completed automatic checkpoints; AutomaticCheckpointQueued reports queued work.
LifecycleStats ActiveSnapshots, PeakSnapshots, and MaximumSnapshots report lease use. WaitingSaves and SaveActive report save serialization. AutomaticCheckpointPending, Closing, and Closed report background and lifecycle state.
EventStats Sequence, HistoryEntries, MaximumHistory, Streams, MaximumStreams, DiscardedDeliveries, HistoryGapEvents, SequenceExhausted, and Closed describe the event stream subsystem.
HistoryStats OldestRevision, CurrentRevision, Changes, and Limit describe the retained transform window.

1.10 Page and Fragment API

Pager exposes logical UTF-8 pages and optional host-defined Fragments. A Measure is a non-negative fixed-point quantity whose scale belongs to the host.

Configuration:

type PageBudget struct {
	Bytes     int64
	Pages     int
	Fragments int
	Measure   Measure
}

type PageOptions struct {
	TargetBytes          int64
	MaximumPageBytes     int64
	MaximumFragments     int
	MaximumTasks         int
	MaximumKeyBytes      int64
	CacheBytes           int64
	MaximumInflightBytes int64
	Window               PageBudget
	DisableCache         bool
}

Zero fields select bounded defaults. Window is the default maximum work for one query; a per-call PageBudget can request a smaller bound.

Identity and result types:

Type Fields and use
PageState Revision, Fragment Generation, and document Bytes. Pass it back with asynchronous publication.
PageKey Revision, Generation, Index, Start, and End form the exact identity returned by Pager. Treat the whole value as opaque; modified or foreign keys are rejected.
Page Key; StartLine/EndLine; ContinuesFromPrevious/ContinuesToNext; FragmentID, DataKey, and FragmentIndex; ContinuationIndex/ContinuationCount; MeasureStart/MeasureEnd; Indexed; and copied Content.
PageWindow State, ordered Pages, Bytes, Fragments, Measure, IndexedThrough, Complete, TruncatedBefore, and TruncatedAfter.

Pager methods:

func (p *Pager) State() PageState
func (p *Pager) Window(offset, before, after int64, budget PageBudget) (PageWindow, error)
func (p *Pager) WindowContext(ctx context.Context, offset, before, after int64, budget PageBudget) (PageWindow, error)
func (p *Pager) FragmentWindow(id string, continuation, before, after int, budget PageBudget) (PageWindow, error)
func (p *Pager) FragmentWindowContext(ctx context.Context, id string, continuation, before, after int, budget PageBudget) (PageWindow, error)
func (p *Pager) MeasureWindow(offset Measure, affinity Affinity, before, after Measure, budget PageBudget) (PageWindow, error)
func (p *Pager) MeasureWindowContext(ctx context.Context, offset Measure, affinity Affinity, before, after Measure, budget PageBudget) (PageWindow, error)
func (p *Pager) Read(key PageKey) (Page, error)
func (p *Pager) ReadContext(ctx context.Context, key PageKey) (Page, error)
func (p *Pager) Publish(batch FragmentBatch) (PageStats, error)
func (p *Pager) PublishContext(ctx context.Context, batch FragmentBatch) (PageStats, error)
func (p *Pager) Refresh(provider FragmentProvider) (PageStats, error)
func (p *Pager) RefreshContext(ctx context.Context, provider FragmentProvider) (PageStats, error)
func (p *Pager) Stats() PageStats
func (p *Pager) Close() error
func (p *Pager) CloseContext(ctx context.Context) error
API Parameter meaning
Window offset is a document byte anchor; before/after are byte overscan.
FragmentWindow id identifies a Fragment; continuation chooses one continuation page; before/after count neighboring Fragments.
MeasureWindow offset is an accumulated Measure; affinity resolves exact boundaries; before/after are Measure overscan.
Read Loads the exact page identified by an unmodified PageKey.
Publish Atomically publishes a FragmentBatch only when its PageState matches the Pager.
Refresh Calls a provider and compare-and-swap publishes its complete result.
CloseContext Waits for accepted page tasks; cancellation can stop waiting while cleanup continues.

Fragment provider types:

type Fragment struct {
	ID      string
	Start   int64
	End     int64
	Measure Measure
	DataKey string
}

type FragmentProvider interface {
	Fragments(context.Context, FragmentRequest) (FragmentResult, error)
}

type FragmentProviderFunc func(context.Context, FragmentRequest) (FragmentResult, error)

FragmentProviderFunc.Fragments makes a function satisfy the interface; a nil function returns ErrInvalidOption.

FragmentRequest supplies the exact State, limits in MaxFragments, MaxKeyBytes, and MaxFragmentMeasure, and a Report(FragmentProgress) callback. Progress watermarks must be monotonic. FragmentResult returns an indexed byte watermark, a completeness flag, and the complete Fragment replacement below that watermark.

Manual publication uses:

type FragmentBatch struct {
	State          PageState
	IndexedThrough int64
	Complete       bool
	Fragments      []Fragment
}

PageStats fields:

Group Fields
Identity and results State, LogicalPages, Pages, Fragments, IndexedThrough, Complete, TotalMeasure.
Page policy TargetBytes, MaximumPageBytes, Window.
Cache CacheBytes, CacheEntries, MaximumCacheBytes.
Work ActiveTasks, MaximumTasks, ActiveInflightBytes, MaximumInflightBytes.
Keys MaximumKeyBytes, KeyBytes.
Lifecycle Closing, Closed.

1.11 Immutable RangeSet

RangeSet[T] stores host values against anchored ranges. It is immutable: every mutation returns a new root, and old roots remain usable.

Construct and refresh:

func NewRangeSet[T any](reader *Reader, options RangeOptions) (*RangeSet[T], error)
func NewRangeSetAt[T any](revision uint64, bytes int64, options RangeOptions) (*RangeSet[T], error)
func RefreshRangeSet[T any](reader *Reader, set *RangeSet[T]) (*RangeSet[T], error)
func RefreshRangeSetContext[T any](ctx context.Context, reader *Reader, set *RangeSet[T]) (*RangeSet[T], error)
  • NewRangeSet uses the Reader's current revision and length.
  • NewRangeSetAt uses explicit identity.
  • RefreshRangeSet obtains the retained Change from the set's revision to the Reader's current revision and returns the transformed root.

RangeOptions fields are MaxEntries, MaxIDBytes, and MaxResults.

type RangeEntry[T any] struct {
	ID    string
	Range Range
	Layer int64
	Value T
}

ID is stable and unique within a root. Putting the same ID replaces its prior entry. Layer is a host-defined ordering value. Value is opaque and copied with ordinary Go assignment.

Methods:

func (s *RangeSet[T]) Revision() uint64
func (s *RangeSet[T]) Bytes() int64
func (s *RangeSet[T]) Stats() RangeStats
func (s *RangeSet[T]) Get(id string) (RangeEntry[T], bool)
func (s *RangeSet[T]) Put(entry RangeEntry[T]) (*RangeSet[T], error)
func (s *RangeSet[T]) PutAll(entries []RangeEntry[T]) (*RangeSet[T], error)
func (s *RangeSet[T]) PutAllContext(ctx context.Context, entries []RangeEntry[T]) (*RangeSet[T], error)
func (s *RangeSet[T]) Remove(id string) (*RangeSet[T], bool, error)
func (s *RangeSet[T]) RemoveAll(ids []string) (*RangeSet[T], bool, error)
func (s *RangeSet[T]) RemoveAllContext(ctx context.Context, ids []string) (*RangeSet[T], bool, error)
func (s *RangeSet[T]) All(maxResults int) (RangeResult[T], error)
func (s *RangeSet[T]) AllContext(ctx context.Context, maxResults int) (RangeResult[T], error)
func (s *RangeSet[T]) Intersect(start, end int64, maxResults int) (RangeResult[T], error)
func (s *RangeSet[T]) IntersectContext(ctx context.Context, start, end int64, maxResults int) (RangeResult[T], error)
func (s *RangeSet[T]) At(offset int64, maxResults int) (RangeResult[T], error)
func (s *RangeSet[T]) AtContext(ctx context.Context, offset int64, maxResults int) (RangeResult[T], error)
func (s *RangeSet[T]) Transform(change Change) (*RangeSet[T], error)
func (s *RangeSet[T]) TransformContext(ctx context.Context, change Change) (*RangeSet[T], error)
API Behavior
Revision, Bytes, Stats Report root identity and entry count.
Get Looks up one stable ID.
Put, PutAll Return a new root; a batch is applied atomically in input order.
Remove, RemoveAll Return a new root and whether any ID existed. Missing IDs are harmless.
All Returns deterministic ordered entries.
Intersect Returns entries overlapping [start,end) plus zero-width entries whose point lies inside it.
At Returns ranges containing offset plus zero-width entries exactly at it.
Transform Requires an exact Change beginning at this root's revision/length and returns the later root.

maxResults == 0 uses the configured maximum. RangeResult.Complete is false when the result limit truncated the query.

1.12 Immutable Composition

Composition creates one immutable logical byte sequence from ranges of Readers or Snapshots.

type CompositionSource struct {
	ID       string
	Reader   *Reader
	Snapshot *Snapshot
}

type CompositionSegment struct {
	SourceID string
	Start    int64
	End      int64
}

func Compose(sources []CompositionSource, segments []CompositionSegment, options CompositionOptions) (*Composition, error)
func ComposeContext(ctx context.Context, sources []CompositionSource, segments []CompositionSegment, options CompositionOptions) (*Composition, error)

Each source must set exactly one of Reader or Snapshot. A Reader is captured at its current revision. Segment ranges must reference a supplied ID, stay within that source, and use UTF-8 boundaries. Segment order is composition order.

CompositionOptions fields:

Field Meaning
MaxSources Source-count bound.
MaxSegments Segment-count bound.
MaxIDBytes UTF-8 byte limit for source IDs.
MaxMappings Default source-map result limit.
MaxReadBytes Allocation limit for Composition.Read.

Methods:

func (c *Composition) State() CompositionState
func (c *Composition) Read(start, end int64) ([]byte, error)
func (c *Composition) ReadContext(ctx context.Context, start, end int64) ([]byte, error)
func (c *Composition) ReadAt(target []byte, offset int64) (int, error)
func (c *Composition) ReadAtContext(ctx context.Context, target []byte, offset int64) (int, error)
func (c *Composition) WriteTo(writer io.Writer) (int64, error)
func (c *Composition) WriteToContext(ctx context.Context, writer io.Writer) (int64, error)
func (c *Composition) Locate(offset int64, affinity Affinity) (Location, error)
func (c *Composition) LocateContext(ctx context.Context, offset int64, affinity Affinity) (Location, error)
func (c *Composition) MapRange(start, end int64, maxMappings int) (MappingResult, error)
func (c *Composition) MapRangeContext(ctx context.Context, start, end int64, maxMappings int) (MappingResult, error)
func (c *Composition) MapSourceRange(sourceID string, sourceRevision uint64, start, end int64, maxMappings int) (MappingResult, error)
func (c *Composition) MapSourceRangeContext(ctx context.Context, sourceID string, sourceRevision uint64, start, end int64, maxMappings int) (MappingResult, error)
func (c *Composition) Close() error
func (c *Composition) CloseContext(ctx context.Context) error
API Parameters and result
State Returns generation, byte length, source count, segment count, and closed state.
Read, ReadAt, WriteTo Read the immutable logical sequence; Read enforces MaxReadBytes.
Locate Maps one logical boundary to one source boundary using affinity.
MapRange Splits one non-empty logical range into ordered source mappings.
MapSourceRange Returns every placement of one source revision/range, including duplicate placements.
Close Releases all captured source references.

CompositionState contains Generation, Bytes, Sources, Segments, and Closed.

Location contains Segment, LogicalOffset, SourceID, SourceRevision, SourceOffset, and the applied Affinity.

Mapping contains Segment, CompositionStart, CompositionEnd, SourceID, SourceRevision, SourceStart, and SourceEnd. MappingResult contains Mappings and Complete; Complete is false when maxMappings truncates the result.

composition, err := doc.Compose(
	[]doc.CompositionSource{
		{ID: "header", Reader: header},
		{ID: "body", Snapshot: bodySnapshot},
	},
	[]doc.CompositionSegment{
		{SourceID: "header", Start: 0, End: header.State().Bytes},
		{SourceID: "body", Start: 0, End: bodySnapshot.Bytes()},
	},
	doc.CompositionOptions{},
)
if err != nil {
	return err
}
defer composition.Close()

1.13 Runtime reclamation

func ReclaimRuntime(root string, before time.Time) (ReclaimStats, error)

root is a parent directory containing Docengine-owned runtime directories. Only recognized, unlocked entries older than before are eligible. ReclaimStats reports Scanned, Reclaimed, and Skipped. Unknown files, symbolic links, live handles, and malformed ownership state are preserved.

2. document.Session: low-level application API

import "github.com/moresleep512/docengine/document"

Session is for applications that need explicit revision comparison, resolved directory policy, raw snapshot leases, direct event channels, direct derived indexes, or detailed persistence state. It opens an existing file; file creation remains an application responsibility or can be performed through doc.New.

Session methods expose some parameter and return types from the coordinate, search, and virtual packages. Those values are inputs or owned results of the Session API; this section documents how Session uses them without describing their independent package APIs.

2.1 Open and resolved configuration

func Open(path string, options OpenOptions) (*Session, error)
func OpenContext(ctx context.Context, path string, options OpenOptions) (*Session, error)

path must identify an existing regular UTF-8 file. OpenContext can be canceled during validation and recovery. On success, the caller owns the Session and must call Close.

OpenOptions:

Field Meaning
RecoveryDir Directory for recoverable unsaved state. Empty uses a shared directory below os.TempDir().
SessionDir Directory for this Session's temporary state. Empty uses a new owned directory below os.TempDir().
RecoveryDirOwnership Ownership policy for an explicit recovery directory.
SessionDirOwnership Ownership policy for an explicit Session directory.
Limits Hard Session limits. Zero fields select defaults.
JournalSyncInterval Recovery durability cadence. Zero uses DefaultJournalSyncInterval.
AutoCheckpointJournalBytes Enables automatic save/checkpoint at this size. Zero disables it. It cannot exceed MaxJournalBytes.

DirectoryOwnership:

Value Meaning
DirectoryOwnershipDefault Resolve to shared for an explicit path; use the default policy when path is empty.
DirectoryShared Never remove the directory.
DirectoryOwned Remove only an empty recognized directory during cleanup.

Specifying ownership without an explicit path is invalid.

SessionLimits and defaults:

Field Default Meaning
MaxBatchOperations DefaultMaxBatchOperations (256) Operations in one atomic batch.
MaxInsertBytes DefaultMaxInsertBytes (1 MiB) Bytes in one operation's Insert string.
UndoBytes DefaultUndoBytes (256 MiB) Retained undo/redo text.
MaxJournalBytes DefaultMaxJournalBytes (4 GiB) Hard recoverable-state growth limit.
EventHistory DefaultEventHistory (256) Retained events.
ChangeHistory DefaultChangeHistory (256) Retained revision transitions.
MaxAnchorBatch DefaultMaxAnchorBatch (65,536) Anchors or ranges in one transform.
MaxSnapshotLeases DefaultMaxSnapshotLeases (1,024) Host-owned Snapshot/index/Pager leases.
MaxSubscriptions DefaultMaxSubscriptions (128) Concurrent event subscriptions.

Public upper bounds include MaximumInsertBytes, MaximumEventHistory, MaximumChangeHistory, MaximumAnchorBatch, MaximumSnapshotLeases, MaximumSubscriptions, and MaximumSubscriptionBuffer. MinimumJournalBytes is the smallest valid journal/checkpoint threshold.

After open:

func (s *Session) Config() SessionConfig

SessionConfig mirrors OpenOptions with absolute directories, resolved ownership, effective non-zero limits, effective sync interval, and checkpoint threshold. It remains available after close.

2.2 Metadata, fault state, and errors

func (s *Session) Metadata() Metadata
func (s *Session) Fault() error

Metadata:

Field Meaning
Path Absolute requested path.
ResolvedPath Fixed real target.
Name Base file name.
ByteLength Current logical bytes, excluding a BOM.
Revision Current logical revision.
CommittedRevision Highest base-file revision.
Dirty Current revision is newer than committed.
Recovered Unsaved operations were restored on open.
HasBOM Base uses a UTF-8 BOM.
EOL EOLLF, EOLCRLF, EOLMixed, or empty for no newline.
DurabilityUncertain Base replacement committed but directory sync was not confirmed.
RecoveryDurabilityUncertain Latest unsaved operations were not confirmed durable.
PersistenceFaulted Session is permanently read-only until reopened.

EOLStyle is the type of Metadata.EOL. Its non-empty values are EOLLF, EOLCRLF, and EOLMixed; the zero value means that no newline was observed.

Fault() returns the original cause for PersistenceFaulted; it returns nil for a healthy Session.

Session error values:

Error Meaning
ErrInvalidOptions Invalid open configuration.
ErrLimitExceeded Configured operation/storage/lease limit.
ErrRevisionConflict expectedRevision differs from current revision.
ErrInvalidUTF8 Base or inserted content is invalid UTF-8.
ErrInvalidUTF8Boundary Edit boundary splits a rune.
ErrInvalidContext Nil Context.
ErrClosed Session is closed or closing for new work.
ErrNothingToUndo / ErrNothingToRedo No retained transaction.
ErrExternalChange Base target changed externally.
ErrRevisionOverflow Another operation revision cannot be represented.
ErrFaulted Mutation/persistence rejected by terminal fault state.
ErrChangeHistoryExpired Requested old transition was discarded.
ErrRevisionUnavailable Revision is future or inside an atomic batch rather than an observable boundary.
ErrInvalidSubscription Invalid buffer/cursor combination.
ErrEventSequence Resume cursor is newer than the stream.
ErrSessionInUse Owned Session directory is locked by a live Session.

RecoveryOpenError exposes JournalPath, optional QuarantinedPath, Reason, and Err. ChangeHistoryError exposes the requested and retained revision windows. Both implement Unwrap; use errors.As for fields and errors.Is for the category.

2.3 Read and Snapshot lease API

func (s *Session) ReadAt(p []byte, off int64) (int, error)
func (s *Session) Snapshot() (revision uint64, lease SnapshotLease, err error)
  • ReadAt reads current content into p beginning at byte off and follows io.ReaderAt conventions. Separate calls can observe different revisions.
  • Snapshot atomically returns the current revision and immutable lease.

SnapshotLease:

type SnapshotLease interface {
	io.ReaderAt
	Len() int64
	WriteTo(io.Writer) (int64, error)
	Close() error
}

The lease remains readable across later edits, save, compaction, and Session close. It consumes MaxSnapshotLeases until closed.

revision, snapshot, err := session.Snapshot()
if err != nil {
	return err
}
defer snapshot.Close()

_, err = snapshot.WriteTo(destination)

2.4 Atomic edit, undo, and redo API

type ReplaceOperation struct {
	Start        int64
	DeleteLength int64
	Insert       string
}

func (s *Session) ApplyBatch(
	ctx context.Context,
	expectedRevision uint64,
	operations []ReplaceOperation,
) (ApplyResult, error)

Parameters:

Parameter Meaning
ctx Cancellation while validating/preparing the batch. Nil is invalid.
expectedRevision Compare-and-apply revision. A mismatch returns ErrRevisionConflict without mutation.
operations Ordered replacements. Each operation uses the content produced by preceding operations.
Start Sequential byte offset.
DeleteLength Bytes removed beginning at Start.
Insert Valid UTF-8 replacement text.

The batch publishes all operations or none and becomes one undo unit. Each operation advances the revision, so the final revision increases by len(operations). An empty operation slice is a successful identity result and does not change revision or history.

ApplyResult fields:

Field Meaning
Revision Final Session revision.
ByteLength Final logical length.
Dirty Final revision is newer than committed.
Changes Exact sequential coordinate.ChangeMap. 
metadata := session.Metadata()
result, err := session.ApplyBatch(ctx, metadata.Revision, []document.ReplaceOperation{
	{Start: 0, DeleteLength: 0, Insert: "prefix "},
	{Start: 7, DeleteLength: 4, Insert: "value"},
})

Undo/redo:

func (s *Session) Undo() (ApplyResult, error)
func (s *Session) UndoContext(ctx context.Context) (ApplyResult, error)
func (s *Session) Redo() (ApplyResult, error)
func (s *Session) RedoContext(ctx context.Context) (ApplyResult, error)

Undo applies the inverse of the newest retained batch; redo reapplies the newest undone batch. Both publish new revisions. Cancellation before publication leaves content and history stacks unchanged.

2.5 Save and commit API

func (s *Session) Save() (Metadata, error)
func (s *Session) SaveContext(ctx context.Context) (Metadata, error)
func (s *Session) CommitAtLeast(expectedRevision uint64) (Metadata, error)
func (s *Session) CommitAtLeastContext(ctx context.Context, expectedRevision uint64) (Metadata, error)
API Meaning
Save Select and commit the newest revision after entering the save serializer.
CommitAtLeast Commit a Snapshot whose revision is at least expectedRevision; return ErrRevisionConflict if that revision is unavailable.

Only one save is active at a time; later saves wait. New edits can continue while an immutable selected revision is being written. If newer edits exist after commit, returned Metadata remains dirty.

Context cancellation before commit leaves the base unchanged. Once replacement commits, committed Metadata takes precedence over later cancellation. Always inspect returned CommittedRevision, DurabilityUncertain, RecoveryDurabilityUncertain, and PersistenceFaulted when an error is returned.

Calling Save again with no new edit retries unresolved durability work.

2.6 Retained change API

func (s *Session) ChangeHistoryStats() ChangeHistoryStats
func (s *Session) ChangesBetween(fromRevision, toRevision uint64) (coordinate.ChangeMap, error)
func (s *Session) TransformAnchors(fromRevision, toRevision uint64, anchors []coordinate.Anchor) ([]coordinate.Anchor, error)
func (s *Session) TransformRanges(fromRevision, toRevision uint64, values []coordinate.AnchoredRange) ([]coordinate.AnchoredRange, error)
  • ChangeHistoryStats reports OldestRevision, CurrentRevision, retained Entries, and configured Limit.
  • ChangesBetween composes exact observable boundaries and supports reverse direction.
  • TransformAnchors and TransformRanges preserve order/affinity and enforce MaxAnchorBatch.
  • These APIs remain available after Session close because they use retained in-memory history.

ChangeHistoryError reports:

type ChangeHistoryError struct {
	FromRevision    uint64
	ToRevision      uint64
	OldestRevision  uint64
	CurrentRevision uint64
	Err             error
}

2.7 Coordinate, search, and Pager handles

Each method below captures one immutable current revision. Every returned Index or Pager owns a Snapshot lease and must be closed. The option and result types in these signatures come from their respective packages, but they are documented here only as parameters and results of Session methods.

Coordinate methods:

func (s *Session) CoordinateIndex(ctx context.Context, options coordinate.Options) (*coordinate.Index, error)
func (s *Session) RebuildCoordinateIndex(ctx context.Context, previous *coordinate.Index, changes coordinate.ChangeMap) (*coordinate.Index, error)
func (s *Session) RefreshCoordinateIndex(ctx context.Context, previous *coordinate.Index) (*coordinate.Index, error)
API Parameters, result, and use
CoordinateIndex ctx controls construction. options.CheckpointBytes bounds bytes decoded by one query; CacheBytes bounds cached query windows; DisableCache requires CacheBytes == 0. Session supplies the lineage, so an application-supplied options.Lineage is replaced. The returned Index represents the current immutable revision.
RebuildCoordinateIndex previous must have been created by this Session. changes must begin at the previous Index revision and end at the current Session revision. The call can reuse unaffected checkpoints.
RefreshCoordinateIndex previous has the same ownership requirement. Session obtains the exact ChangeMap from retained history, so this is the usual refresh operation. It returns ErrChangeHistoryExpired or ErrRevisionUnavailable when that map is no longer available.

The returned Index exposes byte-to-position, position-to-byte, rune-to-byte, line-start, statistics, and close operations. The previous Index remains independently usable after either rebuild method.

coordinates, err := session.CoordinateIndex(ctx, coordinate.Options{
	CheckpointBytes: 64 << 10,
	CacheBytes:      4 << 20,
})
if err != nil {
	return err
}
defer coordinates.Close()

position, err := coordinates.ByteToPosition(ctx, byteOffset)

Search methods:

func (s *Session) SearchIndex(ctx context.Context, options search.IndexOptions) (*search.Index, error)
func (s *Session) RefreshSearchIndex(ctx context.Context, previous *search.Index) (*search.Index, error)

search.IndexOptions parameters used by SearchIndex:

Field Meaning
Directory Persistent index directory.
BlockBytes, OverlapBytes UTF-8-safe indexing block size and boundary overlap. Zero selects package defaults.
MaxBlocks Maximum blocks accepted for one generation.
MaxAnalyzerTokensPerBlock, MaxAnalyzerTermBytes Analyzer work and term-size limits.
Analyzer Optional deterministic term-query policy.
Lineage Replaced by Session; applications should leave it nil.

SearchIndex opens a matching generation or builds one for the current immutable revision. RefreshSearchIndex requires a non-nil Index made by this Session, derives the exact retained ChangeMap, and publishes a current generation. A foreign Index, an expired revision window, or an invalid option returns an error without changing previous. Both old and new Index values remain independently queryable until closed.

index, err := session.SearchIndex(ctx, search.IndexOptions{
	Directory: indexDirectory,
})
if err != nil {
	return err
}
defer index.Close()

report, err := index.Search(
	ctx,
	search.Literal("needle", search.CaseExact),
	search.Options{MaxResults: 100},
)

Pager methods:

func (s *Session) VirtualPager(ctx context.Context, options virtual.Options) (*virtual.Pager, error)
func (s *Session) RefreshVirtualPager(ctx context.Context, previous *virtual.Pager, provider virtual.FragmentProvider) (*virtual.Pager, error)

virtual.Options parameters used by VirtualPager:

Field Meaning
TargetPageBytes, MaximumPageBytes Preferred and hard logical-page byte sizes.
MaximumFragments, MaximumTasks, MaximumKeyBytes Fragment count, concurrent task, and retained key limits.
CacheBytes, DisableCache Page-cache budget or explicit cache disablement.
MaximumInflightBytes Bytes reserved by concurrent page reads and window requests.
Window Default byte/page/fragment/measure budget for one window request.
Observer Optional progress observer. Session preserves it after publishing matching Session events.
Lineage Replaced by Session; applications should leave it nil.

VirtualPager builds logical pages for the current revision. RefreshVirtualPager requires a non-nil Pager made by this Session, preserves its exact policy and lineage, moves to the current revision, and optionally calls provider to publish host-defined Fragments. A nil provider keeps logical page fallback. The previous Pager remains usable.

pager, err := session.VirtualPager(ctx, virtual.Options{
	TargetPageBytes:  64 << 10,
	MaximumPageBytes: 256 << 10,
})
if err != nil {
	return err
}
defer pager.Close()

window, err := pager.WindowByByte(ctx, virtual.ByteWindowRequest{
	Revision:   pager.Stats().Revision,
	Generation: pager.Stats().Generation,
	Offset:     byteOffset,
})

These methods return ErrLimitExceeded when the shared snapshot-lease budget cannot accept another host-owned result.

2.8 Session event API

type SubscribeOptions struct {
	Buffer        int
	AfterSequence uint64
	FutureOnly    bool
}

func (s *Session) Subscribe(options SubscribeOptions) (*Subscription, error)
func (s *Subscription) Events() <-chan SessionEvent
func (s *Subscription) Close() error
func (s *Session) EventStats() EventStats
  • Buffer == 0 uses DefaultSubscriptionBuffer; it cannot exceed MaximumSubscriptionBuffer.
  • AfterSequence resumes strictly after an observed sequence.
  • FutureOnly skips retained events and cannot be combined with a cursor.
  • Events returns the ordered channel. It closes when the subscription or Session closes.
  • Subscription.Close is idempotent and does not close the Session.

SessionEvent:

Field Meaning
Sequence Session-wide monotonic sequence.
Dropped Events omitted only for this subscriber.
Kind EventKind.
Origin Apply/undo/redo origin for content changes.
Metadata State at publication.
Changes Exact change map for EventChanged.
Persistence Save operation correlation and commit state.
Compaction Compaction operation correlation and commit state.
Virtualization Pager publication/refresh progress.
Cause Failure or durability cause.

EventKind values:

  • EventOpened, EventRecovered, EventChanged, EventClosed;
  • EventSaveStarted, EventSaveProgress, EventSaved, EventSaveFailed;
  • EventJournalSyncFailed, EventJournalSyncRestored;
  • EventCompactionStarted, EventCompactionProgress, EventCompacted, EventCompactionFailed;
  • EventVirtualizationStarted, EventVirtualizationProgress, EventVirtualizationCompleted, EventVirtualizationFailed.

ChangeOrigin values are ChangeOriginNone, ChangeOriginApply, ChangeOriginUndo, and ChangeOriginRedo.

PersistenceProgress fields are OperationID, TargetRevision, CompletedBytes, TotalBytes, and Committed.

CompactionProgress fields are OperationID, byte progress, before/after piece counts, JournalCheckpointed, and Committed.

When Dropped != 0, rebuild derived state from Metadata and a matching Snapshot instead of applying only Changes.

EventStats fields are Sequence, HistoryEntries, MaximumHistory, Subscriptions, MaximumSubscriptions, DiscardedDeliveries, HistoryGapEvents, SequenceExhausted, and Closed. It remains available after close.

2.9 Compaction and reclamation API

type CompactOptions struct {
	CheckpointJournal bool
}

func (s *Session) Compact(ctx context.Context, options CompactOptions) (CompactionResult, error)

Compaction reclaims structural and undo storage without changing logical content or revision. CheckpointJournal first commits the current revision so recoverable state can also be rebased.

CompactionResult:

Field Meaning
OperationID Correlates compaction events.
Metadata State observed at return.
Pieces Before/after structural counts and reclamation result.
UndoBytesBefore, UndoBytesAfter Undo storage sizes.
JournalCheckpointed Persistence checkpoint completed.
Committed Compaction publication occurred even if later cleanup failed.

Runtime cleanup:

func ReclaimStaleSessionDirectories(root string, before time.Time) (ReclaimStats, error)

The function scans child directories below root and removes only recognized, unlocked Session state older than before. ReclaimStats reports Scanned, Reclaimed, and Skipped. Unknown entries, symbolic links, malformed state, and live locks are preserved. DefaultStaleSessionAge is the recommended default age threshold.

2.10 Diagnostic and close API

func (s *Session) RecoveryStats() RecoveryStats
func (s *Session) LifecycleStats() LifecycleStats
func (s *Session) EventStats() EventStats
func (s *Session) ChangeHistoryStats() ChangeHistoryStats
func (s *Session) Close() error
func (s *Session) CloseContext(ctx context.Context) error

RecoveryStats fields:

Field Meaning
JournalBytes, MaxJournalBytes Current and maximum recovery storage.
AutoCheckpointBytes, NextAutoCheckpointBytes Configured and next automatic checkpoint thresholds.
AutomaticCheckpoints Completed automatic checkpoints.
AutomaticCheckpointQueued Whether another automatic checkpoint is queued.

LifecycleStats fields:

Field Meaning
ActiveSnapshotLeases, PeakSnapshotLeases, MaxSnapshotLeases Current, peak, and maximum Snapshot-derived leases.
WaitingSaves, SaveActive Save-serializer queue and active state.
AutomaticCheckpointPending Whether automatic checkpoint work is pending.
Closing, Closed Close-barrier state.

All four statistics methods remain available after close.

Close is idempotent and shared across callers. It rejects new work, lets accepted operations reach their defined cancellation/commit boundary, releases owned resources, and closes event channels. If CloseContext returns because its Context expired, cleanup continues; a later close call waits for and returns the same final cleanup result.

Complete direct Session example:

session, err := document.OpenContext(ctx, "notes.txt", document.OpenOptions{
	Limits: document.SessionLimits{
		MaxSnapshotLeases: 64,
		MaxSubscriptions:  8,
	},
})
if err != nil {
	return err
}
defer session.Close()

before := session.Metadata()
applied, err := session.ApplyBatch(ctx, before.Revision, []document.ReplaceOperation{
	{Start: 0, DeleteLength: 0, Insert: "Title\n"},
})
if err != nil {
	return err
}

committed, err := session.CommitAtLeastContext(ctx, applied.Revision)
if err != nil {
	// committed.CommittedRevision and durability fields still matter.
	return err
}
_ = committed