feat(storage): add outbound storage abstractions and memory lifecycle by lykakis · Pull Request #351 · cytechmobile/sendium

lykakis · 2026-10-08T12:07:12Z

Summary

Introduce backend-neutral pending, selected-router, routed-work and coordinator contracts with typed ownership identities.
Implement bounded, non-durable memory storage and activate the standalone outbound pipeline with fail-fast backend profile validation.
Connect HTTP/SMPP admission, multipart ownership, single-destination routing, provider completion, required DLR handoff, retry policies and graceful shutdown.
Preserve embedding extension points and legacy entry points; document the supported lifecycle and guarantees.

Verification

Includes focused storage, coordinator, protocol, retry, shutdown, standalone and embedded-consumer tests.
Latest origin/main integrated; clean working tree and diff whitespace checks passed.
Tests were not rerun during PR creation; CI should validate the integrated branch.

Refs #338
Related epic: #337

Task 1.1 - Define stage contracts, ownership transitions, and the composition boundary. Add single-destination lifecycle interfaces, typed work identities, and embedding contracts. Document stage-consistent recovery and verify public contract boundaries. Refs #338 Signed-off-by: pavlos <pavlos@cytech.gr>

Task 1.2 - Add standalone backend selection, startup validation, and profile logging. Reject unavailable profiles before worker startup, preserve core embedding independence, and verify configuration and startup behavior. Refs #338 Signed-off-by: pavlos <pavlos@cytech.gr>

Task 2.1 - Implement memory pending ownership and bounded router selection. Retain accepted sources through routing, select indexed eligible work within bounded queued and in-flight capacity, and preserve retry ownership through failures. Refs #338 Signed-off-by: pavlos <pavlos@cytech.gr>

Task 2.1a - Implement single-destination memory routed-work storage. Track destination takes, retry-safe forwarding, and terminal cleanup while retaining source ownership and rejecting conflicting assignments. Refs #338 Signed-off-by: pavlos <pavlos@cytech.gr>

Task 2.1b - Implement the concrete outbound lifecycle coordinator. Coordinate owned routing, required handoff, retryable cleanup, and shutdown. Remove downstream-only worker forwarding and document scope ownership rules. Refs #338 Signed-off-by: pavlos <pavlos@cytech.gr>

Signed-off-by: pavlos <pavlos@cytech.gr>

…rage-abstractions Signed-off-by: pavlos <pavlos@cytech.gr>

lykakis · 2026-10-09T13:40:24Z

+
+    private MessageStorageProfile profileAtStartup;
+
+    void start(@Observes @Priority(Interceptor.Priority.APPLICATION) StartupEvent event) {


This parameter is required by CDI: @observes StartupEvent registers the startup callback and @priority establishes its ordering. The event payload is intentionally unused. Retaining the parameter; an explanatory source comment is prepared locally.

lykakis · 2026-10-09T13:40:26Z

+        }
+    }
+
+    void start(@Observes @Priority(Interceptor.Priority.PLATFORM_BEFORE) StartupEvent event, MessageStorageProfile profile) {


The @observes StartupEvent parameter registers early startup logging and profile resolution in CDI. Removing it would remove the observer registration even though the payload is unused. Retaining it; an explanatory source comment is prepared locally.

lykakis · 2026-10-09T13:40:28Z

+        });
+    }
+
+    void stop(@Observes @Priority(Interceptor.Priority.PLATFORM_BEFORE) ShutdownEvent event) throws InterruptedException {


The @observes ShutdownEvent parameter registers provider draining before worker shutdown. Its payload is intentionally unused, but the parameter and annotation are required for CDI lifecycle invocation. Retaining it; an explanatory source comment is prepared locally.

lykakis · 2026-10-09T13:40:30Z

+        return result;
+    }
+
+    void start(@Observes @Priority(Interceptor.Priority.APPLICATION + 100) StartupEvent event) {


The @observes StartupEvent parameter registers pipeline activation after worker startup. Its payload is intentionally unused, but removing the parameter would remove the CDI observer registration. Retaining it; an explanatory source comment is prepared locally.

lykakis · 2026-10-09T13:40:32Z

+    @Produces
+    @Singleton
+    @DefaultBean
+    static OutboundCoordinator<StandardMessage> coordinator(MessageStorageProfile profile, Config config) {


The MessageStorageProfile producer parameter is an intentional CDI dependency: resolving it validates the configured backend profile before constructing or starting the outbound coordinator/stages. It must remain even though its value is not read in the body. An explanatory source comment is prepared locally; focused memory/unsupported-profile startup tests pass.

lykakis · 2026-10-09T13:40:34Z

+    }
+
+    public record Completed(SelectionId selection, Set<SourceId> sources) {
+        public Completed {


Completed defensively snapshots sources through copySources(), which calls Set.copyOf(). The sources.clear() line cited by the alert intentionally mutates the caller collection in a regression test. Subsequent assertions prove Completed retains both source IDs and completed.sources().clear() throws UnsupportedOperationException. OutboundWorkTest passes; no additional copy is needed.

Signed-off-by: pavlos <pavlos@cytech.gr>

Empty legacy worker loops acquired rate-limit permits before checking their queues. With lifecycle traffic sharing the same worker limiter, idle polling consumed send capacity and delayed actual submissions. Acquire a permit only after obtaining a message, preserving one shared per-worker TPS budget for legacy and coordinated attempts. Avoid rate waiting after shutdown and retain the post-wait check that restores taken work. Verify idle polling, shutdown during dequeue and rate acquisition, and real shared-limiter pacing across legacy and lifecycle sends. The focused SMPP, dispatch and shutdown suite passes 48 tests. Signed-off-by: pavlos <pavlos@cytech.gr>

Taken and routed selections have already left the ready deque. Unconditional removal searched the unrelated router backlog on every provider completion while holding shared storage and coordinator locks. Remove from the ready deque only for queued selections, retaining source-first cleanup, routing capacity accounting and idempotence. Verify queued, taken and routed cleanup preserves unrelated work ordering and capacity; all 80 focused storage and coordinator tests pass. Signed-off-by: pavlos <pavlos@cytech.gr>

Blocking rate waits, worker filters and SMPP sends occupied the shared two-thread scheduler, allowing a slow destination to delay healthy destinations and limiting effective sending concurrency. Reserve worker slots before submitting preparation to bounded dispatcher-owned pools sized by threadCount. Preserve the shared per-worker TPS limiter and parent-owned provider retries and handoffs. Retire stopped-worker pools and shut down owned pools after drain without closing application-owned timer schedulers. Verify destination and timer isolation, three occupied slots with a fourth attempt waiting, pool shutdown and worker replacement. Full reactor verify passes 393 core unit tests, 39 integration tests and 43 app tests. Signed-off-by: pavlos <pavlos@cytech.gr>

A mandatory 100 ms pause after every bounded processing cycle imposed a global staging ceiling even when ready messages and provider capacity were available. Schedule the next bounded cycle immediately after successful routing or provider scheduling. Retain idle, pause and failure backoff so unroutable work cannot trigger a busy loop. Fence queued cycles and rescheduling during shutdown, preserving per-worker TPS and execution limits. Add seven deterministic scheduling cases and retain real HTTP pipeline and shutdown coverage. Full reactor verify passes 393 core unit tests, 39 integration tests and 50 app tests with zero Checkstyle violations. Signed-off-by: pavlos <pavlos@cytech.gr>

Match the credential fixture's synchronized getter contract and annotate multipart listener overrides. Explain why CDI observer parameters and the validated-profile producer dependency remain necessary despite unused payload values. Preserve lifecycle ordering and the existing immutable source snapshots rather than removing dependencies or adding redundant copies to silence analyzer comments. All 42 focused ownership, HTTP and startup-profile tests pass with zero Checkstyle violations. Signed-off-by: pavlos <pavlos@cytech.gr>

lykakis added 14 commits September 28, 2026 12:19

feat(storage): connect held multipart and protocol admission

05fa118

Signed-off-by: pavlos <pavlos@cytech.gr>

feat(storage): connect lifecycle routing and provider completion

06ee2dc

Signed-off-by: pavlos <pavlos@cytech.gr>

feat(storage): schedule parent-owned provider retries

220f86b

Signed-off-by: pavlos <pavlos@cytech.gr>

test(storage): preserve startup logs under CI log levels

2b5b7f9

Signed-off-by: pavlos <pavlos@cytech.gr>

feat(storage): drain outbound executions before graceful shutdown

c21d17f

Signed-off-by: pavlos <pavlos@cytech.gr>

feat(storage): activate standalone message outbound pipeline

f5a7dc9

Signed-off-by: pavlos <pavlos@cytech.gr>

test(storage): verify combined outbound and embedded lifecycles

d58c173

Signed-off-by: pavlos <pavlos@cytech.gr>

refactor(storage): clarify outbound lifecycle API and documentation

4fda6b3

Signed-off-by: pavlos <pavlos@cytech.gr>

Merge remote-tracking branch 'origin/main' into feat/338-outbound-sto…

6451ba3

…rage-abstractions Signed-off-by: pavlos <pavlos@cytech.gr>

github-actions Bot added the documentation Improvements or additions to documentation label Oct 8, 2026

github-code-quality Bot found potential problems Oct 8, 2026

View reviewed changes

lykakis added 6 commits October 8, 2026 15:53

test(storage): exclude routing fixture from CDI discovery

f88ef50

Signed-off-by: pavlos <pavlos@cytech.gr>

github-code-quality Bot found potential problems Oct 9, 2026

View reviewed changes

Comment thread sendium-core/src/test/java/gr/cytech/sendium/core/storage/memory/MemoryStorageTest.java Dismissed

Comment thread sendium-core/src/test/java/gr/cytech/sendium/core/storage/memory/MemoryStorageTest.java Dismissed

Navigation Menu

Repository navigation

feat(storage): add outbound storage abstractions and memory lifecycle - #351

feat(storage): add outbound storage abstractions and memory lifecycle#351
lykakis wants to merge 20 commits into
mainfrom
feat/338-outbound-storage-abstractions

lykakis commented Oct 8, 2026

Uh oh!

Uh oh!

lykakis Oct 9, 2026

Uh oh!

lykakis Oct 9, 2026

Uh oh!

lykakis Oct 9, 2026

Uh oh!

lykakis Oct 9, 2026

Uh oh!

lykakis Oct 9, 2026

Uh oh!

lykakis Oct 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant


		private MessageStorageProfile profileAtStartup;

		void start(@Observes @Priority(Interceptor.Priority.APPLICATION) StartupEvent event) {

Conversation

lykakis commented Oct 8, 2026

Summary

Verification

Uh oh!

Uh oh!

lykakis Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

lykakis Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

lykakis Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

lykakis Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

lykakis Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

lykakis Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant