Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@ See [CONTRIBUTING.md](https://github.com/dfa1/rocksdb-ffm/blob/main/CONTRIBUTING
- [ ] Tests use JUnit 5 + AssertJ with `@TempDir`, and every test follows `// Given` / `// When` / `// Then`
- [ ] `./mvnw test` passes
- [ ] `./mvnw package -pl core -am -DskipTests` passes (Javadoc fails the build on any error or warning)
- [ ] New native bindings: `private static final` `MH_` fields, each with its ``/// `<c prototype>` `` comment from `c.h`
- [ ] New native bindings: `private static final` `MH_` fields, each with its ``/// `<c prototype>` `` comment from `c.h`, initialized inline with `NativeLibrary.downcall(`
- [ ] `CHANGELOG.md` has a line under `## [Unreleased]` linking this PR (and the issue, if any), ending with `— thanks [@your-handle](https://github.com/your-handle)` if you're an external contributor
- [ ] Docs updated in the matching [Diátaxis](https://diataxis.fr/) page (tutorial / how-to / reference / explanation), snippets compile against `core`
12 changes: 10 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Every class wrapping a native pointer **must** implement `AutoCloseable`.
throw `IllegalStateException`.
- **Immortal defaults:** a default object that a native call only reads or copies (e.g. the `Options` behind
`ColumnFamilyDescriptor.of(name)`, `TransactionOptions` for `beginTransaction(WriteOptions)`) is a single
package-private `static final X DEFAULTS = newX().makeImmortal();` declared after the class's `MH_` static block, not
package-private `static final X DEFAULTS = newX().makeImmortal();` declared after the class's `MH_` fields, not
created and closed per call. `makeImmortal()` makes its `close()` a no-op while `ptr()` keeps working, so
an accidental `close()` can't break every later caller. Never use it for an object a native call retains,
mutates, or takes ownership of — verify against `db/c.cc` that the call copies (e.g. `ColumnFamilyOptions(opt->rep)`).
Expand Down Expand Up @@ -250,7 +250,15 @@ type (`ReadWriteDB`, `TtlDB`, …) or its capability interface, not on `RocksDB`
- always keep the MethodHandles private static final
- never pass a `MethodHandle` as a method parameter, even internally: `invokeExact` on a `static final` field lets the JIT treat the target as a compile-time constant; routed through a parameter, that constant-folding is lost. Each call site must invoke its own `MH_` field directly, even if that means near-duplicate call sites instead of one shared helper
- **Documented exception:** any class with the same cold-path, no-arg-getter/single-primitive-setter shape — every `*Options` class, plus [StatisticsHistogramData], [SstFileManager], [Env], [Cache], [Replayer], [ColumnFamilyHandle], [Snapshot], [WriteBufferManager], and the read-only event-listener payload views ([CompactionJobInfo], [FlushJobInfo], [MemTableInfo], [ExternalFileIngestionInfo]) that have at least one plain-shape field — routes through the static helpers on `NativeFields` instead of inlining its own `try { mh.invokeExact(...) } catch (Throwable t) { ... }` at every call site: `NativeFields.getInt(mh, ptr())`, `NativeFields.setInt(mh, ptr(), value)`, and the same pattern for `getLong`/`setLong`, `getDouble`/`setDouble`, `getBoolean`/`setBoolean`, `getMemorySize`/`setMemorySize`. The payload views hold a raw `MemorySegment ptr` field rather than extending `NativeObject`, which is fine: `NativeFields`'s helpers take `ptr` as a plain parameter, so nothing about them requires a `NativeObject` receiver. No per-instance delegate object either — `NativeFields` is stateless shared FFM plumbing, the same shape as `NativeCalls.errHolder`/`checkError`/`wrapInvokeFailure` are already static on `NativeCalls`. These are cold-path calls — made once (or once per read/write call in `ReadOptions`/`WriteOptions`, still nowhere near `Get`/`Put`/iterator internals) — so the JIT constant-folding the base rule protects is immaterial here, and consolidating the near-identical `catch (Throwable t)` blocks meaningfully shrinks the permanently-unreachable defensive code cataloged in ADR 0004. Scope stays tight: only a plain no-arg getter or a setter taking `ptr()` plus exactly one primitive qualifies. A pure, always-succeeding value transform in the argument expression is still in scope — e.g. `NativeCalls.toByte(value)`, a null-to-sentinel ternary (`x == null ? -1L : x.toBytes()`), or a bitmask encode/decode (`TraceFilter.toMask(filters)`) — since none of those can throw, so there is no risk of a validation failure being misattributed to `NativeCalls.wrapInvokeFailure`'s "native setter failed" message. What stays a raw, per-call-site `MethodHandle` invocation is a pointer-argument call, a multi-arg call, or a value expression that can itself throw (e.g. `TransactionDBOptions`'s `toMillisOrNoTimeout`, which rejects a negative `Duration`) or that maps onto more than a direct value substitution (e.g. a boolean mapped onto a non-0/1 native enum). Do not route a hot-path class (`WriteBatch`, `Transaction(DB)`, `RocksIterator`, `SstFileWriter`, `PinnableSlice`/`PinnableHandle`, `WalIterator`, ...) through `NativeFields` without the same justification
- every `MH_` field must have a `/// \`<c prototype>\`` comment on the line immediately above it, copied verbatim from `rocksdb/include/rocksdb/c.h` (strip the `extern ROCKSDB_LIBRARY_API` prefix); no duplicate comment in the `static` block
- every `MH_` field must have a `/// \`<c prototype>\`` comment on the line immediately above it, copied verbatim from `rocksdb/include/rocksdb/c.h` (strip the `extern ROCKSDB_LIBRARY_API` prefix)
- every `MH_` field is initialized inline, never assigned in a `static {}` block, in exactly this shape (`java scripts/CheckPrototypes.java` enforces it):
```java
/// `rocksdb_filterpolicy_t* rocksdb_filterpolicy_create_bloom(double bits_per_key);`
private static final MethodHandle MH_CREATE_BLOOM = NativeLibrary.downcall(
"rocksdb_filterpolicy_create_bloom", FunctionDescriptor.of(ValueLayout.ADDRESS, ValueLayout.JAVA_DOUBLE)
);
```
Symbol name and descriptor on one indented line, `);` on its own line. A descriptor with per-argument comments may wrap its arguments, but keeps the same `downcall(` / `);` frame. Why: the prototype comment, the symbol and the descriptor sit together, and there is no second place to look for a handle's definition
- a shared capability interface (`RocksDBReadOperations`, `RocksDBWriteOperations`, `RocksDBMonitoringOperations`, `RocksDBTracingOperations`, ...) never holds its own `MH_` fields directly — an interface field is implicitly `public static final`, so `private static final MethodHandle` (the rule above) cannot live on the interface itself. Instead its default methods delegate to a package-private sibling class named `<InterfaceName>Bindings` (e.g. `RocksDBTracingOperationsBindings`) that holds the `MH_` fields plus the invoke/error-handling logic. This is the shape [#131](https://github.com/dfa1/rocksdb-ffm/issues/131) used to split `RocksDB.java`'s read/write/compaction/monitoring helpers out — one `Bindings` class per capability interface, not one shared catch-all
- don't map multiple times the same symbol from C library of rocksdb
- try to create always a java wrapper for that (i.e. PinnableSlice)
Expand Down
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,8 @@ Cloned without `--recurse-submodules`? Run `git submodule update --init --recurs
- New features come in three tiers: `MemorySegment`, `ByteBuffer`, `byte[]`.
- Open factories take the path first, then `Options`, then other options, descriptors, flags or a TTL (`openReadWrite(path, options, descriptors)`).
- `MethodHandle`s are `private static final`, each with a `/// \`<c prototype>\`` comment copied from
`c.h`. Errors go through `NativeCalls.errHolder`/`checkError`/`wrapInvokeFailure`.
`c.h`, initialized inline as `= NativeLibrary.downcall(` / symbol and descriptor / `);` (checked by
`java scripts/CheckPrototypes.java`). Errors go through `NativeCalls.errHolder`/`checkError`/`wrapInvokeFailure`.
- No C `malloc`: `core` never binds libc (benchmarks may, as an FFM-overhead stand-in). Native memory comes from an `Arena`, the heap (via a `critical(true)` accessor
downcall), or RocksDB itself (freed with `rocksdb_free`) — see `AGENTS.md` for the callback-buffer pattern.

Expand Down
191 changes: 91 additions & 100 deletions core/src/main/java/io/github/dfa1/rocksdbffm/BackupEngine.java
Original file line number Diff line number Diff line change
Expand Up @@ -39,120 +39,111 @@
public final class BackupEngine extends NativeObject {

/// `rocksdb_backup_engine_t* rocksdb_backup_engine_open(const rocksdb_options_t* options, const char* path, char** errptr);`
private static final MethodHandle MH_OPEN;
private static final MethodHandle MH_OPEN = NativeLibrary.downcall(
"rocksdb_backup_engine_open",
FunctionDescriptor.of(ValueLayout.ADDRESS,
ValueLayout.ADDRESS, // options
ValueLayout.ADDRESS, // path
ValueLayout.ADDRESS) // errptr
);
/// `rocksdb_backup_engine_t* rocksdb_backup_engine_open_opts(const rocksdb_backup_engine_options_t* options, rocksdb_env_t* env, char** errptr);`
private static final MethodHandle MH_OPEN_OPTS;
private static final MethodHandle MH_OPEN_OPTS = NativeLibrary.downcall(
"rocksdb_backup_engine_open_opts",
FunctionDescriptor.of(ValueLayout.ADDRESS,
ValueLayout.ADDRESS, // options
ValueLayout.ADDRESS, // env
ValueLayout.ADDRESS) // errptr
);
/// `void rocksdb_backup_engine_create_new_backup_flush(rocksdb_backup_engine_t* be, rocksdb_t* db, unsigned char flush_before_backup, char** errptr);`
private static final MethodHandle MH_CREATE_NEW_BACKUP_FLUSH;
private static final MethodHandle MH_CREATE_NEW_BACKUP_FLUSH = NativeLibrary.downcall(
"rocksdb_backup_engine_create_new_backup_flush",
FunctionDescriptor.ofVoid(
ValueLayout.ADDRESS, // be
ValueLayout.ADDRESS, // db
ValueLayout.JAVA_BYTE, // flush_before_backup
ValueLayout.ADDRESS) // errptr
);
/// `void rocksdb_backup_engine_purge_old_backups(rocksdb_backup_engine_t* be, uint32_t num_backups_to_keep, char** errptr);`
private static final MethodHandle MH_PURGE_OLD_BACKUPS;
private static final MethodHandle MH_PURGE_OLD_BACKUPS = NativeLibrary.downcall(
"rocksdb_backup_engine_purge_old_backups",
FunctionDescriptor.ofVoid(
ValueLayout.ADDRESS, // be
ValueLayout.JAVA_INT, // num_backups_to_keep (uint32_t)
ValueLayout.ADDRESS) // errptr
);
/// `void rocksdb_backup_engine_verify_backup(rocksdb_backup_engine_t* be, uint32_t backup_id, char** errptr);`
private static final MethodHandle MH_VERIFY_BACKUP;
private static final MethodHandle MH_VERIFY_BACKUP = NativeLibrary.downcall(
"rocksdb_backup_engine_verify_backup",
FunctionDescriptor.ofVoid(
ValueLayout.ADDRESS, // be
ValueLayout.JAVA_INT, // backup_id (uint32_t)
ValueLayout.ADDRESS) // errptr
);
/// `void rocksdb_backup_engine_restore_db_from_latest_backup(rocksdb_backup_engine_t* be, const char* db_dir, const char* wal_dir, const rocksdb_restore_options_t* restore_options, char** errptr);`
private static final MethodHandle MH_RESTORE_FROM_LATEST;
private static final MethodHandle MH_RESTORE_FROM_LATEST = NativeLibrary.downcall(
"rocksdb_backup_engine_restore_db_from_latest_backup",
FunctionDescriptor.ofVoid(
ValueLayout.ADDRESS, // be
ValueLayout.ADDRESS, // db_dir
ValueLayout.ADDRESS, // wal_dir
ValueLayout.ADDRESS, // restore_options
ValueLayout.ADDRESS) // errptr
);
/// `void rocksdb_backup_engine_restore_db_from_backup(rocksdb_backup_engine_t* be, const char* db_dir, const char* wal_dir, const rocksdb_restore_options_t* restore_options, const uint32_t backup_id, char** errptr);`
private static final MethodHandle MH_RESTORE_FROM_BACKUP;
private static final MethodHandle MH_RESTORE_FROM_BACKUP = NativeLibrary.downcall(
"rocksdb_backup_engine_restore_db_from_backup",
FunctionDescriptor.ofVoid(
ValueLayout.ADDRESS, // be
ValueLayout.ADDRESS, // db_dir
ValueLayout.ADDRESS, // wal_dir
ValueLayout.ADDRESS, // restore_options
ValueLayout.JAVA_INT, // backup_id (uint32_t)
ValueLayout.ADDRESS) // errptr
);
/// `const rocksdb_backup_engine_info_t* rocksdb_backup_engine_get_backup_info(rocksdb_backup_engine_t* be);`
private static final MethodHandle MH_GET_BACKUP_INFO;
private static final MethodHandle MH_GET_BACKUP_INFO = NativeLibrary.downcall(
"rocksdb_backup_engine_get_backup_info", FunctionDescriptor.of(ValueLayout.ADDRESS, ValueLayout.ADDRESS)
);
/// `int rocksdb_backup_engine_info_count(const rocksdb_backup_engine_info_t* info);`
private static final MethodHandle MH_INFO_COUNT;
private static final MethodHandle MH_INFO_COUNT = NativeLibrary.downcall(
"rocksdb_backup_engine_info_count", FunctionDescriptor.of(ValueLayout.JAVA_INT, ValueLayout.ADDRESS)
);
/// `int64_t rocksdb_backup_engine_info_timestamp(const rocksdb_backup_engine_info_t* info, int index);`
private static final MethodHandle MH_INFO_TIMESTAMP;
private static final MethodHandle MH_INFO_TIMESTAMP = NativeLibrary.downcall(
"rocksdb_backup_engine_info_timestamp",
FunctionDescriptor.of(ValueLayout.JAVA_LONG,
ValueLayout.ADDRESS, // info
ValueLayout.JAVA_INT) // index
);
/// `uint32_t rocksdb_backup_engine_info_backup_id(const rocksdb_backup_engine_info_t* info, int index);`
private static final MethodHandle MH_INFO_BACKUP_ID;
private static final MethodHandle MH_INFO_BACKUP_ID = NativeLibrary.downcall(
"rocksdb_backup_engine_info_backup_id",
FunctionDescriptor.of(ValueLayout.JAVA_INT,
ValueLayout.ADDRESS, // info
ValueLayout.JAVA_INT) // index
);
/// `uint64_t rocksdb_backup_engine_info_size(const rocksdb_backup_engine_info_t* info, int index);`
private static final MethodHandle MH_INFO_SIZE;
private static final MethodHandle MH_INFO_SIZE = NativeLibrary.downcall(
"rocksdb_backup_engine_info_size",
FunctionDescriptor.of(ValueLayout.JAVA_LONG,
ValueLayout.ADDRESS, // info
ValueLayout.JAVA_INT) // index
);
/// `uint32_t rocksdb_backup_engine_info_number_files(const rocksdb_backup_engine_info_t* info, int index);`
private static final MethodHandle MH_INFO_NUMBER_FILES;
private static final MethodHandle MH_INFO_NUMBER_FILES = NativeLibrary.downcall(
"rocksdb_backup_engine_info_number_files",
FunctionDescriptor.of(ValueLayout.JAVA_INT,
ValueLayout.ADDRESS, // info
ValueLayout.JAVA_INT) // index
);
/// `void rocksdb_backup_engine_info_destroy(const rocksdb_backup_engine_info_t* info);`
private static final MethodHandle MH_INFO_DESTROY;
private static final MethodHandle MH_INFO_DESTROY = NativeLibrary.downcall(
"rocksdb_backup_engine_info_destroy", FunctionDescriptor.ofVoid(ValueLayout.ADDRESS)
);
/// `void rocksdb_backup_engine_close(rocksdb_backup_engine_t* be);`
private static final MethodHandle MH_CLOSE;

static {
MH_OPEN = NativeLibrary.downcall("rocksdb_backup_engine_open",
FunctionDescriptor.of(ValueLayout.ADDRESS,
ValueLayout.ADDRESS, // options
ValueLayout.ADDRESS, // path
ValueLayout.ADDRESS)); // errptr

MH_OPEN_OPTS = NativeLibrary.downcall("rocksdb_backup_engine_open_opts",
FunctionDescriptor.of(ValueLayout.ADDRESS,
ValueLayout.ADDRESS, // options
ValueLayout.ADDRESS, // env
ValueLayout.ADDRESS)); // errptr

MH_CREATE_NEW_BACKUP_FLUSH = NativeLibrary.downcall(
"rocksdb_backup_engine_create_new_backup_flush",
FunctionDescriptor.ofVoid(
ValueLayout.ADDRESS, // be
ValueLayout.ADDRESS, // db
ValueLayout.JAVA_BYTE, // flush_before_backup
ValueLayout.ADDRESS)); // errptr

MH_PURGE_OLD_BACKUPS = NativeLibrary.downcall("rocksdb_backup_engine_purge_old_backups",
FunctionDescriptor.ofVoid(
ValueLayout.ADDRESS, // be
ValueLayout.JAVA_INT, // num_backups_to_keep (uint32_t)
ValueLayout.ADDRESS)); // errptr

MH_VERIFY_BACKUP = NativeLibrary.downcall("rocksdb_backup_engine_verify_backup",
FunctionDescriptor.ofVoid(
ValueLayout.ADDRESS, // be
ValueLayout.JAVA_INT, // backup_id (uint32_t)
ValueLayout.ADDRESS)); // errptr

MH_RESTORE_FROM_LATEST = NativeLibrary.downcall(
"rocksdb_backup_engine_restore_db_from_latest_backup",
FunctionDescriptor.ofVoid(
ValueLayout.ADDRESS, // be
ValueLayout.ADDRESS, // db_dir
ValueLayout.ADDRESS, // wal_dir
ValueLayout.ADDRESS, // restore_options
ValueLayout.ADDRESS)); // errptr

MH_RESTORE_FROM_BACKUP = NativeLibrary.downcall(
"rocksdb_backup_engine_restore_db_from_backup",
FunctionDescriptor.ofVoid(
ValueLayout.ADDRESS, // be
ValueLayout.ADDRESS, // db_dir
ValueLayout.ADDRESS, // wal_dir
ValueLayout.ADDRESS, // restore_options
ValueLayout.JAVA_INT, // backup_id (uint32_t)
ValueLayout.ADDRESS)); // errptr

MH_GET_BACKUP_INFO = NativeLibrary.downcall("rocksdb_backup_engine_get_backup_info",
FunctionDescriptor.of(ValueLayout.ADDRESS, ValueLayout.ADDRESS));

MH_INFO_COUNT = NativeLibrary.downcall("rocksdb_backup_engine_info_count",
FunctionDescriptor.of(ValueLayout.JAVA_INT, ValueLayout.ADDRESS));

MH_INFO_TIMESTAMP = NativeLibrary.downcall("rocksdb_backup_engine_info_timestamp",
FunctionDescriptor.of(ValueLayout.JAVA_LONG,
ValueLayout.ADDRESS, // info
ValueLayout.JAVA_INT)); // index

MH_INFO_BACKUP_ID = NativeLibrary.downcall("rocksdb_backup_engine_info_backup_id",
FunctionDescriptor.of(ValueLayout.JAVA_INT,
ValueLayout.ADDRESS, // info
ValueLayout.JAVA_INT)); // index

MH_INFO_SIZE = NativeLibrary.downcall("rocksdb_backup_engine_info_size",
FunctionDescriptor.of(ValueLayout.JAVA_LONG,
ValueLayout.ADDRESS, // info
ValueLayout.JAVA_INT)); // index

MH_INFO_NUMBER_FILES = NativeLibrary.downcall("rocksdb_backup_engine_info_number_files",
FunctionDescriptor.of(ValueLayout.JAVA_INT,
ValueLayout.ADDRESS, // info
ValueLayout.JAVA_INT)); // index

MH_INFO_DESTROY = NativeLibrary.downcall("rocksdb_backup_engine_info_destroy",
FunctionDescriptor.ofVoid(ValueLayout.ADDRESS));

MH_CLOSE = NativeLibrary.downcall("rocksdb_backup_engine_close",
FunctionDescriptor.ofVoid(ValueLayout.ADDRESS));
}
private static final MethodHandle MH_CLOSE = NativeLibrary.downcall(
"rocksdb_backup_engine_close", FunctionDescriptor.ofVoid(ValueLayout.ADDRESS)
);

private BackupEngine(MemorySegment ptr) {
super(ptr);
Expand Down
Loading
Loading