diff --git a/CHANGELOG.md b/CHANGELOG.md index a1625ca8..72284afb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **Migration hook for shared memory segments.** `lintel::shm::SegmentLayoutPolicy` lets the + application decide what happens when a segment was written with another object layout: refuse + (default), recreate, or run a registered migrator that sees the untouched segment file and can + export its payloads before it is replaced. `LINTEL_SHM_RECREATE=1` selects "recreate" without + recompiling. See `docs/shm-migration-strategy.md`. +- **Admin API client and CLI component.** The two `/admin/*` routes (`reload-tls`, `log-level`) had + no `AdminApi` and no CLI command; `reload_tls` and `log_level --level` fill that gap. +- **`POST /shm/recover`.** `SharedMemoryService::recover()` reclaims remap locks whose holder is + gone but had no caller. The route (admin only) answers `{"RECOVERED":n}`, with + `SharedMemoryApi::recoverOf()` and the CLI command `recover_segments`. +- **`SharedMemoryRepository::of()`.** The typed repositories can be created through a + process-wide factory that deduplicates by `segment:name` through + `SharedMemoryRepositoryRegistry` instead of building a second cell for the same name. +- **`Publisher-EventBus` group.** Publishing no longer rides on the read group. +- **EventBus over HTTP and in the CLI.** Seven routes (`/eventBus/buses`, `/eventBus/bus/{name}`, + `/eventBus/publish/{name}`, `POST`/`DELETE` `/eventBus/subscribe/{name}`, + `/eventBus/depth/{name}/{subscriber}`, `/eventBus/reset/{name}`) with an `EventBusApi` client and + an `EventBusCliComponent` (`show_event_buses`, `publish_event`, `subscribe_event`, + `unsubscribe_event`, `event_queue_depth`, `reset_event_bus`). Mutating routes need + `Admin-EventBus`, reading and publishing `User-EventBus`. The subsystem had no HTTP, API or CLI + surface before. +- **Shared memory and message queue operations gained endpoints.** `GET /shm/stats/{segment}` + (`SharedMemoryService::statsOf`, now part of `ISharedMemoryService`), `POST /shm/cleanup` + (`SharedMemoryCleanupService`, now registered in the container) and `POST /messageQueue/recover` + (`MessageQueueService::recoverOrphans`), each with its API method and CLI command + (`show_stats`, `cleanup`, `recover_message_queues`). +- **`/messageQueue/stats/...` and `/messageQueue/purge/...` documented** in `swagger.yaml` and + `swagger.md`; both routes existed in the controller but not in the published contract. - **`${ENV_VAR}` expansion for the bootstrap configuration.** `EnvironmentConfiguration::expand()` replaces every `${NAME}` (`NAME` = `[A-Za-z_][A-Za-z0-9_]*`) with the environment variable of that name; an unset variable stays literal and `$${NAME}` escapes to `${NAME}`. @@ -163,6 +191,44 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 the `postgres:17` service. ### Changed +- **Publishing on the event bus needs its own right (breaking).** `POST /eventBus/publish/{name}` + accepts a token of `Publisher-EventBus` or `Admin-EventBus`; `User-EventBus` is read-only and + gets 401. The group is a virtual group like the other two, so it is seeded automatically. +- **The shared memory process model is cross-process now (breaking).** `ShmSegmentAccessor` + carries the *name* of its segment (40 bytes instead of one pointer) and resolves it to the + allocator of the calling process, so an accessor copied into a segment is usable from every + process. `MessageQueueShmCore` keeps its header and ring in the segment under a stable name + instead of a process-local map, which means two processes now really share one queue. Segments + written by an older build are recognised by a layout marker; the previous layout cannot be + reinterpreted, so the operator decides what happens (`docs/shm-migration-strategy.md`): the + default refuses to open the segment and names the file, `LINTEL_SHM_RECREATE=1` discards it, and + a migrator registered through `shm::SegmentLayoutPolicy::setMigrator()` runs **before** anything + is removed and may export the payloads first. Data that only lives in a segment - the payloads of + `shm::SharedMemoryRepository` and the containers it stores - is exactly what such a migrator is + for; properties (database or bootstrap XML, the shared memory repository is a mirror that is + rebuilt), users, groups and history live outside the segment and stay untouched. +- **`SharedMemoryRepositoryRegistry.h` moved** from `features/base/services/` to `core/shm/`; the + typed repository header may not depend on a feature header. +- **Logging and property registration use `std::source_location` (breaking).** + `LoggerService::log()`/`logImpl()` take a `std::source_location` instead of + `(file, line, function)`, and `PropertyRegistration::registerProperty()` replaces its + `fileName`/`position` parameters with a `std::source_location` defaulted to + `std::source_location::current()`. The reported file, line and function, and the persisted + `":"` `DataStorage` identifier are unchanged; only callers that pass the two + arguments explicitly have to drop them. `LINTEL_FUNCTION` is gone. +- **`PropertyRepository::allOf()` takes a `PropertyFilter` (breaking).** The four regular + expressions (`processName`, `className`, `instanceName`, `name`) are one aggregate with the + previous `".*"` defaults, so `allOf()` and short calls keep their meaning. Follows the + `MessageQueueFilter`/`EventBusCapacities` pattern from `a4c1c59`. +- **`Connection::prepareStatement()` takes the statement name as `std::string_view`.** + `Connection::execute()` runs its script through `sqlite3_prepare_v2`/`sqlite3_step` instead of + `sqlite3_exec` and follows the `pzTail` remainder, so multi-statement scripts keep working. +- **`MemorySize` rejects leading whitespace and a sign.** `std::from_chars` replaced `std::stoul`; + `" 1024"` and `"-1"` now throw instead of being accepted. +- **All `NOLINT` suppressions are gone.** `handleCommandInput()`/`handleTabInput()` return an + `InputOutcome` instead of two `bool &` out-parameters, `parseAndScale()` takes a unit table, + `startsWith()` takes the menu as a `std::string_view`, `HistoryController` dropped its unused + `GroupRepository` parameter, and the shared-memory address arithmetic uses `std::bit_cast`. - **Leak detection is a blocking check.** The `sanitizers` job (ASan+UBSan+LSan, `detect_leaks=1`) is no longer `continue-on-error`: a full run including the `[pg]` cases is green, so a leak regression stops the merge instead of being reported and ignored. @@ -231,6 +297,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 Hypodermic was an unmaintained dependency; `lintel::di` replaces it. ### Fixed +- **The SHM message queue was not an IPC mechanism.** Every process built its own ring in the + segment, so `MessageQueueShmCore` only worked inside one process. A fork-based test now proves + that a child process finds the queue by name and that a message crosses the boundary. +- **The worker processes outlived `bin/main`.** Two defects added up: + `VirtualGroupBootstrapPlugin` is a singleton *inside* the DI container and held a `shared_ptr` + to that container, so container → plugin → container never dropped to zero - the container was + never destroyed and no service destructor ever ran. And `ProcessService` only stopped its + children in its destructor. The plugin (and the worker example's own service) hold the container + weakly now, and `ProcessService::onShutdown()` stops and, if needed, terminates every process it + started, so a shutdown no longer depends on destruction order. +- **The examples could not be started.** `Feature::m_name` was a `std::string_view` initialised from + a temporary `std::string`, so `getName()` returned freed stack bytes as soon as the constructor + returned. `StartupBuilder::verifyFeatureDependencies()` compared that garbage against the + configured names and aborted every process at startup with `terminate called ... feature '\x01'` + - `scripts/run-example.sh` never got past `bin/main`. The name is stored by value now, and a + regression test scribbles over the stack before it reads the name back. +- **`export_repository --file` expanded every tilde, not just a leading one.** A Windows short name + (`C:\Users\RUNNER~1\...`) was rewritten into `C:\Users\RUNNERC:\Users\...`, so the CLI + failed to create the directory. The new `lintel::expandHomeDirectory()` only replaces `~` at the + start of the path. +- **`RingBufferShm` addressed its slots with an absolute pointer.** Commit `a4c1c59` replaced the + byte offset with a `std::byte *` to drop an int-to-pointer cast, which only works while the ring + lives at the same virtual address in every process - a repository payload inside a shared memory + segment does not. Back to the offset, materialised with `std::bit_cast`; a regression test copies + the region holding the ring and its slots and reuses the ring at the second address. +- **CLI fixes.** `grow_segment` was described as "Shrink segment"; `export_repository` never bound + `--file` although it writes the file (`m_type` was dead and is gone); `PropertyCliComponent` + mixed `--process-name` and `--process_name`; `UserManagementCliComponent` declared + `AddGroup`/`RemoveGroup`/`UpdateGroup` without a command or handler (the HTTP API cannot create + groups, so the enum slots are gone); `HistoryCliComponent` had an `n_name` typo and a stray `)` + in its help text. - **Review fixes on the pre-1.0 hardening items (A8/U5/A6/A5).** - The shared-memory lock recovery could reclaim the same orphan twice: two processes that both found a dead holder ran `remove()` + `open_or_create()` on the same semaphore and each kept a diff --git a/docs/compile-time-measurements.md b/docs/compile-time-measurements.md index eca96ec5..de0e0b5c 100644 --- a/docs/compile-time-measurements.md +++ b/docs/compile-time-measurements.md @@ -135,7 +135,7 @@ header that feature consumers include. |---------|--------|-----------| | `ProcessResourceReader`, `ProcessInfo`, `ProcessInfoDto`, `ProcessService` | `boost::process::v1::pid_t` → `pid_t` (``), `child` only forward-declared | `getProcessId()` now returns `pid_t` by value | | `MessageQueue` | non-template core `MessageQueueCore` (Pimpl), Boost in `MessageQueueCore.cpp` | no (template API unchanged) | -| `EventBus` | `ShmSegmentAccessor` (Boost-free, 1 pointer) + `ShmSegmentAccessor::Allocator` interface; Boost-bound in `BoostSegmentAllocator.h` | constructor takes `ShmSegmentAccessor` instead of `segment_manager*` | +| `EventBus` | `ShmSegmentAccessor` (Boost-free, segment name + allocator pointer, 40 bytes) + `ShmSegmentAccessor::Allocator` interface; Boost-bound in `BoostSegmentAllocator.h` | constructor takes `ShmSegmentAccessor` instead of `segment_manager*` | | `SingleInstanceBootstrapPlugin` | `Lock` class (Boost `file_lock`) as private Pimpl in the `.cpp` | no | | `SharedMemoryService` | Boost-bound free functions in `shm::` (`ShmConstructors.h`), segment state in `SharedMemorySegmentHandle` | **yes**: `constructX()` → `shm::constructX(service, …)`, `ShmString` → `shm::String`, `getSegmentManager()` → `shm::segmentManagerOf()` | diff --git a/docs/nolint-integration-plan.md b/docs/nolint-integration-plan.md new file mode 100644 index 00000000..99700707 --- /dev/null +++ b/docs/nolint-integration-plan.md @@ -0,0 +1,761 @@ +# Plan: NOLINT-Abbau, HTTP/CLI-Integrationslücken und `std::source_location` + +Stand: 21. Oktober 2026 · Basis: `master` @ `1333386` · Bibliotheksversion 0.2.0 + +Dieses Dokument war der Arbeitsplan für den Branch `refactor/nolint-cleanup`. Die +Teile A und C sowie die Punkte B.2 (1–4) und B.3 sind umgesetzt; Abschnitt +[7. Umsetzungsstand](#7-umsetzungsstand) ordnet jedem Punkt den Commit zu und listet, +was bewusst offen bleibt. + +| Teil | Thema | Umfang | Ergebnis | +|------|-------|--------|----------| +| **A** | NOLINT-Marker vollständig entfernen | 13 Marker in 10 Dateien, 4 Check-Klassen | 0 Marker, keine neuen CodeQL-Alerts | +| **B** | Fehlende HTTP-/CLI-Integration | 11 Lücken + 7 Defekte | priorisierter Backlog | +| **C** | `__FILE__`/`__LINE__` → `std::source_location` | 2 Makro-Familien, ~50 Aufrufstellen | eine API-Änderung (Breaking) | + +--- + +## 1. Warum die NOLINTs überhaupt nötig sind + +### 1.1 Der `--checks`-Override ist die Ursache, nicht die Regel + +`.clang-tidy` schaltet zwei der betroffenen Checks explizit ab: + +```yaml + -bugprone-easily-swappable-parameters, + -performance-unnecessary-value-param, +``` + +`scripts/check-tidy.sh` überschreibt diese Liste aber auf der Kommandozeile: + +```bash +CHECKS="${TIDY_CHECKS:--*,clang-analyzer-*,bugprone-*,cert-*,performance-*,portability-*}" +``` + +Der CLI-Wert ersetzt `Checks:` aus der Konfigurationsdatei vollständig. Damit sind +beide Checks im Gate **wieder an** – für geänderte Dateien, obwohl die Projektkonfiguration +sie als nicht zielführend eingestuft hat. Die NOLINTs sind die Folge. + +Es gibt zwei grundsätzlich verschiedene Auswege, und der Plan verfolgt bewusst den +ersten: + +1. **Code reparieren**, sodass der Check nichts mehr findet (dieses Dokument). +2. Den Override entschärfen, z. B. `-bugprone-easily-swappable-parameters` in + `TIDY_CHECKS` aufnehmen. Das wäre eine Policy-Änderung am Gate und wird hier + **nicht** empfohlen: die Prüfung hat in diesem Repo mehrfach echte Fehlerklassen + aufgedeckt (siehe `a4c1c59`), und ein pauschales Abschalten würde sie wieder + verlieren. + +### 1.2 Drei Werkzeuge, die sich widersprechen + +Bei den vier `performance-no-int-to-ptr`-Stellen kollidieren drei Prüfungen, die alle +Teil der CI sind: + +| Werkzeug | Fordert | Warum | +|----------|---------|-------| +| `clang-tidy` (`performance-no-int-to-ptr`) | **kein** `IntegralToPointer`-Cast | Check matcht `castExpr(hasCastKind(CK_IntegralToPointer))` – jede Ganzzahl→Zeiger-Konvertierung außer aus einem Integer-Literal | +| `CodeQL` (`cpp/suspicious-pointer-scaling`) | **keine** Byte-Zeiger-Arithmetik auf einem `reinterpret_cast` aus einem breiteren Objektzeiger | Meldung: *"This pointer might have type ShmRing (size 160), but this pointer arithmetic is done with type byte \* (size 1)."* | +| `GCC` (`-Wstringop-overflow`, `-Werror`) | Zeiger darf **nicht** auf das umgebende Objekt zurückführbar sein | Sonst meldet der Optimierer einen Überlauf, weil der Slot-Puffer im Segment und nicht im Objekt liegt | + +Deshalb ist die aktuelle Lösung „Rechnung in der Integer-Domäne, Rückführung per +`reinterpret_cast`“ entstanden: sie erfüllt CodeQL und GCC und verletzt clang-tidy. + +--- + +## 2. Teil A — NOLINT-Marker + +### A.0 Bestandsaufnahme + +```bash +grep -rn "NOLINT" --include="*.cpp" --include="*.h" src/ examples/ tests/ +``` + +| # | Datei:Zeile | Check | betroffene Funktion | +|---|-------------|-------|---------------------| +| 1 | `src/features/base/events/EventBus.cpp:156` | `performance-no-int-to-ptr` | `ShmRing::buffer()` | +| 2 | `src/features/base/events/EventBus.cpp:163` | `performance-no-int-to-ptr` | `ShmRing::buffer() const` | +| 3 | `src/features/base/services/SharedMemoryRepository.cpp:96` | `performance-no-int-to-ptr` | `ShmCellRecord::payload()` | +| 4 | `src/features/base/msg/MessageQueueShmCore.cpp:254` | `performance-no-int-to-ptr` | `ShmRing::slotAt()` | +| 5 | `src/features/property/repositories/SharedMemoryPropertyRepository.cpp:92` | `bugprone-easily-swappable-parameters` | `allOf(process, class, instance, name)` | +| 6 | `src/features/cli/services/CommandLineService.cpp:140` | `bugprone-easily-swappable-parameters` | `handleCommandInput(ev, bool&, bool&)` | +| 7 | `src/features/cli/services/CommandLineService.cpp:171` | `bugprone-easily-swappable-parameters` | `handleTabInput(ev, bool&, bool&)` | +| 8 | `src/features/cli/services/CommandLineHistoryService.cpp:67` | `bugprone-easily-swappable-parameters` | `startsWith(command, menu)` | +| 9 | `src/core/persistence/sqlite3/Connection.cpp:140` | `bugprone-easily-swappable-parameters` | `prepareStatement(queryName, query)` | +| 10 | `src/core/persistence/sqlite3/Connection.cpp:257` | `bugprone-easily-swappable-parameters` | `execute()`-Callback (`char**, char**`) | +| 11 | `src/core/utils/MemorySize.cpp:19` | `bugprone-easily-swappable-parameters` | `parseAndScale(input, prefix, multiplier)` | +| 12 | `src/features/http/controllers/HistoryController.cpp:41` | `performance-unnecessary-value-param` | unbenutzter `groupRepository`-Parameter | +| 13 | `examples/main.cpp:128` | `bugprone-exception-escape` | impliziter Move-`operator=` von `TestDataDto` | + +Dazu kommt `SharedMemoryPropertyRepository.cpp:91` – kein Marker, sondern der Kommentar, +der erklärt, warum kein Zeilenumbruch möglich ist. Er entfällt mit dem Umbau in A.2. + +**Reihenfolge der Arbeitspakete** (jedes ist einzeln mergebar): + +| Paket | Marker | Aufwand | Risiko | +|-------|--------|---------|--------| +| A.1 | 4 × `performance-no-int-to-ptr` | klein | mittel (Werkzeugkonflikt) | +| A.3 | 1 × `performance-unnecessary-value-param` | klein | keins | +| A.4 | 1 × `bugprone-exception-escape` | klein | keins | +| A.2a | 3 × swappable (`startsWith`, `prepareStatement`, `parseAndScale`) | klein | keins (keine Aufrufstellen-Änderung) | +| A.2b | 2 × swappable (`handleCommandInput`, `handleTabInput`) | klein | keins (privat) | +| A.2c | 1 × swappable (`execute()`-Callback) | mittel | niedrig | +| A.2d | 1 × swappable (`PropertyRepository::allOf`) | groß | API-Bruch, ~27 Aufrufstellen | + +--- + +### A.1 `performance-no-int-to-ptr` (4 Marker) + +#### Befund + +Alle vier Stellen adressieren einen Puffer, der **im Shared-Memory-Segment** liegt, über +einen **Byte-Offset relativ zum umgebenden Objekt**: + +```cpp +// EventBus.cpp:131 – im Konstruktor +m_bufferOffset = aligned - reinterpret_cast(this); +``` + +Das Offset-Muster ist korrekt und muss so bleiben: das Objekt liegt selbst im Segment und +wird in jedem Prozess an einer **anderen** virtuellen Adresse eingeblendet, nur die +Relative bleibt stabil. Ein absoluter Zeiger wäre prozesslokal und damit falsch (siehe +A.1.4). + +#### Gemessene Varianten + +Alle Varianten wurden in `src/` bzw. einer isolierten Probe gegen die echten +Projektflags (`-Werror -Wall -Wextra -Wcast-align …`) und die CI-Checkliste geprüft: + +| Variante | GCC-Build | clang-tidy (CI-Checks) | CodeQL | +|----------|-----------|------------------------|--------| +| V1 Integer-Domäne + `reinterpret_cast` **(Status quo)** | ok | **Finding** | ok | +| V2 `std::byte *base = …; return reinterpret_cast(base + offset);` | **Fehler**: `writing 345 bytes into a region of size 152 [-Werror=stringop-overflow=]` (`slot.value = event` in `EventBus::publish`) | ok | **Alert** `cpp/suspicious-pointer-scaling` | +| V2b V2 + `// codeql[cpp/suspicious-pointer-scaling]` | Fehler | ok | unterdrückt | +| **V3 `std::bit_cast(address)`** | **ok** | **ok** | kein Alert zu erwarten (kein Cast-Ausdruck, keine Zeigerarithmetik) | +| V4 absoluter `std::byte *m_buffer`-Member (Muster aus `RingBufferShm`) | ok | ok | ok – **aber fachlich falsch**, siehe A.1.4 | + +V2 scheitert an GCC: sobald der Zeiger über eine `std::byte *`-Basis zurückführbar ist, +sieht der Optimierer das umgebende Objekt (152 Byte) und den 345 Byte großen Schreibvorgang. +Das ist ein **False Positive** – der Puffer liegt im Segment –, aber mit `-Werror` +buildbrechend. `std::bit_cast` (V3) unterbricht die Rückführbarkeit und ist damit die +einzige Variante, die alle drei Werkzeuge gleichzeitig erfüllt. + +#### Änderung (V3) + +```cpp +// src/features/base/events/EventBus.cpp (+ #include ) +EventBus::ShmRing::Slot *EventBus::ShmRing::buffer() +{ + // Der Puffer liegt im Segment, nicht in diesem Objekt, und wird über den Byte-Offset + // m_bufferOffset adressiert. Die Adresse entsteht in der Integer-Domäne; std::bit_cast + // stellt sie wieder her: ein reinterpret_cast aus der Ganzzahl meldet clang-tidy als + // performance-no-int-to-ptr, Byte-Zeiger-Arithmetik meldet CodeQL als + // cpp/suspicious-pointer-scaling. + const std::uintptr_t address = reinterpret_cast(this) + static_cast(m_bufferOffset); + return std::bit_cast(address); +} + +const EventBus::ShmRing::Slot *EventBus::ShmRing::buffer() const +{ + const std::uintptr_t address = reinterpret_cast(this) + static_cast(m_bufferOffset); + return std::bit_cast(address); +} +``` + +```cpp +// src/features/base/services/SharedMemoryRepository.cpp (+ #include ) +[[nodiscard]] void *payload() const +{ + const std::uintptr_t address = + reinterpret_cast(this) + static_cast(m_payloadOffset); + return std::bit_cast(address); +} +``` + +```cpp +// src/features/base/msg/MessageQueueShmCore.cpp (+ #include ) +[[nodiscard]] SlotHeader *slotAt(std::uint64_t position) const +{ + const std::uintptr_t address = reinterpret_cast(this) + + static_cast(m_bufferOffset) + + (position % m_capacity) * m_stride; // kein Cast: size_t == uintptr_t + return std::bit_cast(address); +} +``` + +**Fallstricke, die beim Nachbauen auftreten:** + +- `static_cast((position % m_capacity) * m_stride)` ist ein + `-Wuseless-cast`-Fehler (das Produkt ist bereits `std::size_t`, auf LP64 identisch mit + `std::uintptr_t`). Der Cast muss weg. +- In `const`-Memberfunktionen ist `reinterpret_cast(this)` unzulässig; die + Integer-Variante umgeht das, weil eine Ganzzahl keine Constness trägt. Genau deshalb + bleibt der Cast `reinterpret_cast(this)` (ohne `const_cast`). +- `-Wcast-align` schlägt bei keiner der Varianten zu (mit den Projektflags geprüft). + +#### Verifikation + +```bash +cmake --build build/build/Release --parallel "$(nproc)" +cd build/build/Release && ctest --output-on-failure --label-regex unit # 1194 Tests, grün +scripts/check-tidy.sh master build/tidy # 0 findings +``` + +Zusätzlich muss der CodeQL-Lauf im PR grün sein; lokal ist CodeQL nicht installiert +(`docs/code-scanning.md`, Abschnitt *Reproducing the analysis locally*). + +#### A.1.4 Warum der „Präzedenzfall“ `RingBufferShm` nicht kopiert werden darf + +`src/include/lintel/core/shm/containers/RingBufferShm.h` hat den NOLINT in `a4c1c59` +durch einen absoluten Zeiger ersetzt: + +```cpp +m_buffer = aligned; // statt m_bufferOffset = aligned - (byte*)this +[[nodiscard]] Slot *buffer() { return reinterpret_cast(m_buffer); } +``` + +Das ist **kein gültiges Muster** für Objekte im Segment: `m_buffer` enthält die Adresse +aus dem Mapping des *erzeugenden* Prozesses. Der Klassenkommentar (Zeilen 27–32) +beschreibt weiterhin korrekt das Offset-Layout – Code und Dokumentation sind hier +auseinandergelaufen. Der Test `SharedMemoryRepositoryTest.cpp:392` +("QueueShm in shared memory is FIFO") deckt nur den Ein-Prozess-Fall ab, deshalb ist der +Fehler bisher nicht aufgefallen. + +**Konsequenz für den Plan:** `RingBufferShm` wird in einem eigenen Schritt auf das +Offset-Muster (mit `std::bit_cast`) zurückgeführt, sobald A.1 gemergt ist. Bis dahin +darf es nicht als Vorbild für weitere Stellen dienen. + +#### A.1.5 Strukturelle Folgelösung (optional, nach A.1) + +Die saubere Langfristlösung adressiert relativ zur **Segmentbasis** statt zum Objekt: + +1. `ShmSegmentAccessor::Allocator` um `virtual std::byte *baseAddress() const noexcept` + erweitern (Default `nullptr`), `ShmSegmentAccessor::baseAddress()` durchreichen. +2. `BoostSegmentAllocator` implementiert das über die Adresse der gemappten Datei + (`basic_managed_memory_impl::get_address()`, siehe + `boost/interprocess/detail/managed_memory_impl.hpp:228`); der Allocator braucht dafür + zusätzlich die Basisadresse im Konstruktor. +3. Offsets relativ zur Segmentbasis speichern und mit + `reinterpret_cast(segmentBase + offset)` adressieren – Byte-Arithmetik auf einem + `std::byte *`, dessen statischer Typ bereits `std::byte *` ist. Damit entfallen + `std::bit_cast` **und** jede Unterdrückung. + +**Blocker, die vorher gelöst werden müssen** (eigene Arbeitspakete, siehe B.4): + +- `ShmSegmentAccessor` wird **im Segment** gespeichert und hält einen Zeiger auf den + Allocator des *erzeugenden* Prozesses. In einem zweiten Prozess ist dieser Zeiger + ungültig; `baseAddress()` würde die falsche Basis liefern. Der Accessor muss pro Prozess + neu gebunden werden (z. B. über einen prozesslokalen Registry-Eintrag je Segmentname, + analog zu `mqshm_detail::g_registry`). +- `ShmConstructors.h` (`constructObjectWith`) ruft den Konstruktor nur beim *Erzeugen*; + ein anhängender Prozess bekommt das Objekt ohne Rebinding. + +--- + +### A.2 `bugprone-easily-swappable-parameters` (7 Marker) + +#### Regel + +Der Check meldet Nachbarparameter **gleichen Typs**, deren Namen nicht durch ein +gemeinsames Präfix/Suffix „geschützt“ sind (`NamePrefixSuffixSilenceDissimilarityTreshold`, +Default 1). Empirisch gilt für dieses Repo: + +- **Umbenennen allein hilft nicht** (außer bei fast identischen Namen wie `name`/`names`). +- Die Typen müssen sich **unterscheiden** – zwei verschiedene Strong Types oder z. B. + `std::string` vs. `std::string_view`. +- Ein gemeinsames Aggregat für *beide* Parameter hilft nicht, nur ein einzelner + Aggregat-Parameter (wie in `a4c1c59` mit `MessageQueueFilter` geschehen). + +#### A.2a Drei Stellen ohne Änderung der Aufrufstellen + +| Stelle | Änderung | Warum ohne Aufrufstellen-Änderung | +|--------|----------|-----------------------------------| +| `CommandLineHistoryService::startsWith(command, menu)` | `menu` → `std::string_view` | wird nur verglichen (`iter.getMenu() != menu`) | +| `Connection::prepareStatement(queryName, query)` | `queryName` → `std::string_view` | Schlüssel wird einmalig als `std::string key(queryName)` materialisiert; `std::string` und `const char *` konvertieren implizit | +| `MemorySize::parseAndScale(input, prefix, multiplier)` | beide → `std::string_view`, `std::stoul` → `std::from_chars` | anonymer Namespace, zwei Aufrufe in derselben Datei | + +Für `parseAndScale` fällt dabei ein Nebengewinn an: `std::from_chars` ist `noexcept` und +allokiert nicht, die Fehlermeldungen laufen über `std::format` (im TU bereits eingebunden): + +```cpp +std::size_t parseAndScale(std::string_view input, std::string_view prefix, std::size_t multiplier) +{ + unsigned long value = 0; + const auto [end, error] = std::from_chars(prefix.data(), prefix.data() + prefix.size(), value); + if (error != std::errc{} || end != prefix.data() + prefix.size()) { + throw std::invalid_argument(std::format("invalid memory size: '{}'", input)); + } + ... +} +``` + +#### A.2b `CommandLineService`: Out-Parameter → Rückgabewert + +```cpp +// CommandLineService.h +struct InputOutcome +{ + bool m_tabPressed = false; + bool m_printPrompt = true; +}; + +InputOutcome handleCommandInput(const SymbolEvent &symbolEvent); +InputOutcome handleTabInput(const SymbolEvent &symbolEvent); +``` + +Einziger Aufrufer ist `CommandLineService.cpp:264/267`. Damit verschwinden beide Marker +und die beiden `bool &`-Parameter, die den Prompt-Zustand durch die Funktion tragen. + +#### A.2c `Connection::execute()`: Callback entfernen + +Der Callback hat die von der C-API vorgegebene Signatur +`(void *, int, char **, char **)` – zwei `char **` lassen sich nicht durch Typen +unterscheiden. Deshalb nicht umbenennen, sondern **die API wechseln**: `execute()` auf +`sqlite3_prepare_v2` + `sqlite3_step` + `sqlite3_column_*` umstellen, so wie +`executeParameters()` (Zeilen ~225–251) es bereits tut. + +Gewinn: `Connection::callBack` (Zeile 279), der `std::function`-Wrapper und der +`static_cast(&func)`-Trick entfallen; der Marker verschwindet ohne Ersatz. + +#### A.2d `PropertyRepository::allOf` – Aggregat (Breaking Change) + +```cpp +// src/include/lintel/features/property/repositories/PropertyRepository.h +struct PropertyFilter +{ + std::string m_processName{ ".*" }; + std::string m_className{ ".*" }; + std::string m_instanceName{ ".*" }; + std::string m_name{ ".*" }; +}; + +virtual std::vector> allOf(const PropertyFilter &filter = {}) = 0; +``` + +Muster: `MessageQueueFilter` / `EventBusCapacities` aus `a4c1c59`. + +Betroffen: + +- `PropertyRepository.h` (Interface), `SharedMemoryPropertyRepository`, + `DatabasePropertyRepository`, Mocks in `tests/` +- ~27 Aufrufstellen, u. a. `PropertyService.cpp:190/203/220/241/379`, + `PropertyController.cpp:43`, `PropertyCliComponent.cpp:44`, + `PropertyChangeTest.cpp:299`, `DatabasePropertyRepositoryTest.cpp:196–202` + +Das ist der einzige echte API-Bruch in Teil A und gehört in `CHANGELOG.md` unter +`[Unreleased]` (siehe `docs/versioning.md`: vor 1.0.0 ist ein Minor-Bruch erlaubt, muss +aber ausgewiesen werden). + +*Alternative, falls der Umbau verschoben wird:* Interface unverändert lassen und den +Marker stehen lassen – dann bleibt A.2 offen. Nicht empfohlen, weil der Marker dann +dauerhaft als „bekannte Schuld“ im Code steht. + +--- + +### A.3 `performance-unnecessary-value-param` (1 Marker) + +`HistoryController` bekommt einen `std::shared_ptr`, den der Konstruktor +nie verwendet (die Gruppen sind fest verdrahtet: `m_adminGroup("Admin-History")`, +`m_userGroup("User-History")`). Der Kommentar begründet die Beibehaltung mit der +DI-Auflösung – das ist nicht nötig, der Container löst nach Typ auf und kommt mit zwei +Parametern aus. + +```diff +- explicit HistoryController(const std::shared_ptr &authService, +- std::shared_ptr groupRepository, +- std::shared_ptr historyRepository); ++ explicit HistoryController(const std::shared_ptr &authService, ++ std::shared_ptr historyRepository); +``` + +Mit anzupassen: + +- `src/features/http/controllers/HistoryController.cpp` (Definition + irreführender Kommentar) +- `tests/http/HistoryControllerTest.cpp` (`makeController()`, unbenutztes `m_groupRepository` + samt Include entfernen) +- `#include "…/GroupRepository.h"` in `HistoryController.h` entfällt + +Die DI-Registrierung `builder.registerType()…` in +`src/app/HttpFeature.cpp:34` bleibt unverändert. + +--- + +### A.4 `bugprone-exception-escape` (1 Marker) + +Betroffen ist **nicht** der Kopier-Zuweisungsoperator, sondern der **implizit erzeugte +Move-`operator=`** von `TestDataDto`: `bugprone-exception-escape` prüft Move-Konstruktoren, +Move-Zuweisungen, Destruktoren, `swap`, `main` und alles mit `noexcept` – und der +implizite Move-`operator=` der `shm::String`-Member kann `bad_alloc` werfen. + +**Nicht** funktioniert (jeweils geprüft): + +- `operator=(const TestDataDto &) = delete;` → `boost::container::vector::push_back` + braucht die Zuweisung (`copy_move_algo.hpp:889`), Build bricht. +- Move-Operationen explizit `= default` setzen → `TestDataDto` ist dann kein Aggregat + mehr, die Initialisierung mit fünf Argumenten (`examples/main.cpp`) bricht. + +**Funktioniert:** den Destruktor explizit defaulten. Das unterdrückt die impliziten +Move-Operationen; Züge fallen auf die Kopieroperationen zurück (was im Beispiel ohnehin +passiert), und der Check hat kein Move mehr zu prüfen: + +```cpp +struct TestDataDto +{ + // Die shm::String-Member allokieren aus dem Segment, also kann auch ein Move das + // Werfen nicht versprechen. Der explizit gedefaultete Destruktor unterdrückt die + // impliziten Move-Operationen; Kopien laufen über die Kopieroperationen, was jede + // Verwendung in diesem Beispiel ohnehin tut. + ~TestDataDto() = default; + ... +}; +``` + +Aggregat-Initialisierung bleibt möglich (ein Destruktor zählt nicht zu den +„user-declared constructors“). + +--- + +### A.5 Definition of Done für Teil A + +```bash +grep -rn "NOLINT" --include="*.cpp" --include="*.h" src/ examples/ tests/ # leer +cmake --build build/build/Release --parallel "$(nproc)" # grün +cd build/build/Release && ctest --output-on-failure --label-regex unit # grün +scripts/check-tidy.sh master build/tidy # 0 findings +``` + +Zusätzlich: `CHANGELOG.md` unter `[Unreleased]` um die API-Änderung +(`PropertyRepository::allOf` samt Aufrufern) ergänzen. + +--- + +## 3. Teil B — fehlende HTTP-/CLI-Integration + +### B.1 Methodik + +Verglichen wurde die Menge der öffentlichen Service-Methoden (`src/include/lintel/**/services/*.h`, +`*Repository.h`, `*Api.h`) mit der Menge der HTTP-Routen (`ADD_HANDLER_METHOD` in +`controllers/`) und der CLI-Kommandos (`CommandParser::addCommand` in `components/`). + +Wesentliche Strukturerkenntnis: **die CLI reitet auf HTTP**. Jede CLI-Komponente hält eine +`*Api`-Klasse, die den HTTP-Client des gleichen Endpunkts aufruft. CLI-Abdeckung ist damit +immer eine Teilmenge der HTTP-Abdeckung – eine Lücke im HTTP ist automatisch eine Lücke in +der CLI, umgekehrt nicht. + +### B.2 Lücken, nach Nutzen sortiert + +| # | Lücke | Ist-Zustand | Vorschlag | Nutzen | +|---|-------|-------------|-----------|--------| +| 1 | **EventBus** (publish/subscribe/stats/dropped) | **kein** Controller, **keine** Api, **kein** CLI-Kommando – `grep EventBus src/features/http src/features/cli` ist leer | `EventBusController` (`/eventBus/…`: publish, subscribe, stats, purge) + `EventBusApi` + CLI-Komponente + swagger | hoch – das Subsystem ist komplett unsichtbar | +| 2 | `SharedMemoryCleanupService` | Klasse existiert, ist aber **nicht in der DI** (`src/app/BaseFeature.cpp:117–158`) und hat keinen Endpoint | Registrierung + `/sharedMemory/cleanup/…` + CLI | hoch – sonst nur programmatisch erreichbar | +| 3 | `MessageQueueService::recoverOrphans(staleThreshold)` | nur Service (`MessageQueueService.cpp:95`) | `POST /messageQueue/recover` + CLI | hoch (Betrieb) | +| 4 | `SharedMemoryService::statsOf(segment)` | nur auf der **konkreten** Klasse, nicht auf `ISharedMemoryService` | Methode ins Interface, `/sharedMemory/stats/…` | hoch | +| 5 | `SharedMemoryService::recover(...)` | **null Aufrufer**, Doc-Kommentar beschreibt ein anderes Verhalten | Endpoint bauen **oder** entfernen (toter Code) | mittel | +| 6 | MQ-Überlauf-/Dead-Letter-Policy (`setOverflowPolicy`, `setOverflowTimeout`, `setDeadLetterQueueName`) | wird von `MessageQueue` und `MessageQueueService` **nicht** durchgereicht | Service-API + `PUT /messageQueue/policy/…` | mittel | +| 7 | `MessageQueueRequestReply` | nirgends exponiert | optional API | mittel | +| 8 | `ProcessService`: die `ProcessGroup`-Überladungen (`startOf/restartOf/stopOf/terminateOf/detachOf(const ProcessGroup &)`, Zeilen 264–281) | HTTP kennt nur die `Process`-Variante (`ProcessController.h:21–31`) | Gruppen-Endpoints | mittel | +| 9 | `HistoryService`-Abfrage-Überladungen | `HistoryController` greift **direkt** auf `HistoryRepository` zu, umgeht den Service | Controller auf den Service umstellen | mittel (Architektur) | +| 10 | `SharedMemoryRepositoryRegistry` | Singleton (`instance()`), **kein einziger Aufrufer** im Repo | verdrahten oder löschen | niedrig | +| 11 | `ShmView`/`ShmSlice`, SHM-Lifecycle-Hooks, typisierte SHM-Payload-API | nicht exponiert | optional, erst nach A.1.5 | niedrig | + +### B.3 Konkrete Defekte in bestehender CLI/Swagger + +Alle im Code verifiziert, unabhängig von den Lücken oben: + +| Datei:Zeile | Defekt | +|-------------|--------| +| `src/features/cli/components/SharedMemoryCliComponent.cpp:26` | `grow_segment` hat die Beschreibung **„Shrink segment“** | +| `src/features/cli/components/SharedMemoryCliComponent.cpp:30–33` | `export_repository` bindet `m_type` und `m_file` nicht als Argumente – die Flags fehlen, die Member sind toter Zustand | +| `src/features/cli/components/PropertyCliComponent.cpp:12/18/25` | `--process-name` und `--process_name` gemischt (gleiche Option, zwei Schreibweisen) | +| `src/features/cli/components/UserManagementCliComponent.cpp:19/84` | Enum-Slots `AddGroup`, `RemoveGroup`, `UpdateGroup` haben **weder** `addCommand` **noch** `case` | +| `src/features/cli/components/HistoryCliComponent.cpp:9` | Parametername `n_name` (Tippfehler) | +| `src/features/cli/components/HistoryCliComponent.cpp` | Hilfe-Text mit überzähliger `)` | +| `swagger.yaml` | `/messageQueue/stats/{processName}/{messageQueueName}` und `/messageQueue/purge/{processName}` sind implementiert (`MessageQueueController.h:39–40`), fehlen aber in `swagger.yaml` **und** `swagger.md` – `scripts/check-swagger.sh` prüft nur die Gegenrichtung | + +### B.4 Nebenbefunde (SHM-Korrektheit, blockieren A.1.5) + +1. **`MessageQueueShmCore` ist nicht prozessübergreifend.** `openOrCreateState()` legt den + `QueueState` in einem prozesslokalen `std::map`-Registry ab + (`MessageQueueShmCore.cpp:306–357`); jeder Prozess erzeugt damit seinen **eigenen** Ring + im Segment. Der Test „is visible to a second core of the same segment“ prüft nur zwei + Cores **im selben Prozess**. Der SHM-Backend liefert damit faktisch keine IPC. +2. **`ShmSegmentAccessor` im Segment zeigt auf den erzeugenden Prozess.** Der Accessor wird + als Member im Segmentobjekt kopiert (`ShmSegmentAccessor.h:16–19`) und dereferenziert + `m_allocator` lazy – in einem zweiten Prozess ist das ein fremder Prozesszeiger. + `~ShmRing` ruft `m_segment.deallocate(...)`. +3. **`RingBufferShm` speichert eine absolute Adresse** (siehe A.1.4). + +Diese drei Punkte gehören in ein eigenes Arbeitspaket „SHM-Prozessmodell“, bevor +B.2 #2/#4/#11 sinnvoll werden. + +--- + +## 4. Teil C — `__FILE__`/`__LINE__` → `std::source_location` + +### C.1 Bestandsaufnahme + +```bash +grep -rn "__FILE__" src/ tests/ examples/ benchmarks/ # 51 Treffer +``` + +| Ort | Vorkommen | Ersetzbar? | +|-----|-----------|------------| +| `src/include/lintel/core/services/LoggerService.h:201–213` | 6 `LOG_*`-Makros, jedes mit `__FILE__, __LINE__, LINTEL_FUNCTION` | **ja** | +| `src/include/lintel/core/services/PropertyRegistration.h:26–31` | `registerProperty(name, defaultValue, description, runtime, fileName, position)`, `DataStorage`-Kennung `":"` | **ja** | +| `src/include/lintel/features/property/services/PropertyService.h:22–24` | `DEFINE_PROPERTY`-Makro | **ja** | +| 19 direkte Aufrufe `registerProperty(…, __FILE__, __LINE__)` in `AuthService` (6), `HistoryService` (5), `ProcessService` (2), `SharedMemoryService` (3), `SchedulerService` (1), `CommandLineHistoryService` (2) | Aufrufstellen | **ja** | +| `tests/core/LoggerServiceTest.cpp` | 25 Aufrufe `service.log(__FILE__, __LINE__, "test", …)` | **ja** | +| `src/include/lintel/core/utils/TypeName.h` | `__PRETTY_FUNCTION__` zur **Compile-Zeit** | **nein** – `std::source_location` ist ein Laufzeitobjekt und kann den Typnamen nicht ersetzen. Bleibt unverändert und wird hier nur abgegrenzt. | + +Die 351 `LOG_*`-Aufrufe im Repo bleiben **unverändert** – die Makros kapseln die Signatur. + +Verhaltensprüfung (GCC 14, `-O0` und `-O2`, in Makros, Default-Argumenten und Lambdas): +`std::source_location::current()` liefert exakt dieselben `file_name()`/`line()`-Werte wie +`__FILE__`/`__LINE__`, `function_name()` entspricht `__PRETTY_FUNCTION__` (inkl. +Lambda-Schreibweise). Die Migration ist damit **verhaltensneutral**, auch für die +`DataStorage`-Kennung `":"`. + +### C.2 Ziel-API + +```cpp +// LoggerService.h – Signatur von log()/logImpl() umstellen, Makros bleiben in der Form +void log(const std::source_location &location, LogLevel lvl, std::string_view message, Args &&...args) +{ + logImpl(location, lvl, message, std::make_format_args(args...)); +} + +#define LOG_INFO(message, ...) \ + lintel::LoggerService::get().log(std::source_location::current(), lintel::LogLevel::info, message, ##__VA_ARGS__) +``` + +```cpp +// PropertyRegistration.h +template +std::shared_ptr> registerProperty(std::string name, + type defaultValue, + std::string description, + bool runtime, + const std::source_location &location = std::source_location::current()) +{ + ... + const std::filesystem::path path(location.file_name()); + property->setDataStorage(DataStorage(PropertyRepositoryType::DEFAULT, + path.filename().generic_string() + ":" + std::to_string(location.line()))); + ... +} +``` + +```cpp +// PropertyService.h +#define DEFINE_PROPERTY(name, type, defaultValue, description, runtime) \ + std::shared_ptr> name = \ + registerProperty(std::string(#name), defaultValue, description, runtime) +``` + +`logImpl()` in `src/core/services/LoggerService.cpp:263–287` baut heute +`spdlog::source_loc{file, line, function}`; danach: + +```cpp +const spdlog::source_loc source{ location.file_name(), + static_cast(location.line()), + location.function_name() }; +``` + +`log()` bleibt ein Template (`std::make_format_args`), deshalb kann `logImpl()` die +`source_location` nicht selbst erzeugen – das `current()` muss im Makro stehen. Genau +deshalb ist die Umstellung verhaltensneutral: der Aufrufpunkt ist derselbe wie bei +`__FILE__`/`__LINE__`. + +### C.3 Migrationsschritte + +1. `LoggerService.h` (6 Makros + `log()`) und `LoggerService.cpp` (`logImpl()`, `spdlog::source_loc`) + umstellen. Die 351 `LOG_*`-Aufrufstellen bleiben unberührt. +2. `PropertyRegistration.h` (`registerProperty`, `DataStorage`-Kennung) und + `PropertyService.h` (`DEFINE_PROPERTY`) umstellen. +3. Die 19 direkten `registerProperty`-Aufrufe: `, __FILE__, __LINE__` **entfernen** + (Default-Argument greift). Das ist der einzige Punkt, an dem Aufrufstellen angefasst + werden – ein rein mechanischer, zeilenweiser Edit. +4. `tests/core/LoggerServiceTest.cpp` (25 Stellen): + `service.log(__FILE__, __LINE__, "test", …)` → + `service.log(std::source_location::current(), …)`; das dritte Argument (Funktionsname) + entfällt, weil `function_name()` jetzt aus der Location kommt. +5. Verifikation: `ctest -R Logger`, `ctest -R Property`, plus Sichtprüfung einer Logzeile + und einer persistierten Property (`DataStorage`-Kennung `":"` muss + identisch bleiben). + +### C.4 Bruchstelle + +`PropertyRegistration::registerProperty` ist **öffentliche API** +(`src/include/lintel/core/services/PropertyRegistration.h`). Der Wegfall von +`fileName`/`position` ist ein Signaturbruch und muss in `CHANGELOG.md` unter +`[Unreleased]` stehen (siehe `docs/versioning.md`: vor 1.0.0 erlaubt, aber auszuweisen). + +`LoggerService::log()` ist ebenfalls öffentlich, wird aber praktisch nur über die Makros +benutzt; die Umstellung ist dort unkritisch, weil die Makros die Signatur kapseln. + +--- + +## 5. Risiken und offene Entscheidungen + +| Risiko | Auswirkung | Umgang | +|--------|-----------|--------| +| `std::bit_cast` zwischen Ganzzahl und Zeiger ist formal nicht garantiert (der Standard sichert nur die Objektrepräsentation zu) | theoretisch; auf allen Zielplattformen (LP64, flache Adressräume) korrekt | in A.1.5 durch Segmentbasis-Adressierung ersetzen; bis dahin im Kommentar benennen | +| CodeQL-Verhalten von `std::bit_cast` ist lokal nicht prüfbar | ein Alert könnte im PR auftauchen | PR-Lauf abwarten; Fallback ist `// codeql[cpp/suspicious-pointer-scaling]` (dokumentierter Mechanismus, `docs/code-scanning.md`) | +| A.2d bricht die `PropertyRepository`-API | ~27 Aufrufstellen | eigener PR, `CHANGELOG.md`, ggf. `[[deprecated]]`-Überladung für eine Minor-Version | +| `RingBufferShm`-Adressfehler ist unentdeckt | Cross-Process-Korruption | eigener Fix **vor** allen Erweiterungen in B.2 #11 | +| `MessageQueueShmCore` ist nicht prozessübergreifend | SHM-Backend erfüllt MQ-6 aus `docs/ipc-driver-unit-plan.md` nicht | dortigen Status korrigieren; eigenes Arbeitspaket | + +**Zu entscheiden** (Stand nach der Umsetzung, Details im Backlog in §7): + +1. ~~A.2d (Aggregat) jetzt oder später?~~ → umgesetzt (`cbd7c36`). +2. `SharedMemoryService::recover()` – Endpoint bauen oder löschen? +3. `SharedMemoryRepositoryRegistry` – verdrahten oder entfernen? +4. Reihenfolge B.4 (SHM-Prozessmodell) gegen B.2 – ohne B.4 sind #2/#4/#11 nur Fassade. + +--- + +## 6. Anhang + +### 6.1 Verifikationskommandos + +```bash +# Build (CI-Layout) +cmake --build build/build/Release --parallel "$(nproc)" + +# Unit-Tests (PostgreSQL-Tests werden ohne laufenden Server übersprungen) +cd build/build/Release && ctest --output-on-failure --label-regex unit + +# clang-tidy-Gate (braucht build/tidy mit -DENABLE_UNITY_BUILD=OFF -DENABLE_PCH=OFF) +scripts/check-tidy.sh master build/tidy + +# Format / Namespace / Secrets +scripts/check-format.sh master +scripts/check-namespace.sh +scripts/check-secrets.sh +``` + +### 6.2 Vorab geprüfte Varianten + +Die Variantenmatrix aus A.1 und die beiden verworfenen Ansätze aus A.4 wurden vor dem +Umbau isoliert gegen die echten Projektflags und die CI-Checkliste gemessen; die +Ergebnisse stehen dort. Die daraus abgeleiteten Änderungen sind als Commits `01e2856` +(A.1), `c3717fd` (A.3) und `c0356d2` (A.4) enthalten. + +--- + +## 7. Umsetzungsstand + +Branch `refactor/nolint-cleanup`, aufgesetzt auf `master @ 1333386`, 20 Commits. + +### Teil A — NOLINT-Marker: vollständig + +`grep -rn NOLINT --include="*.cpp" --include="*.h" src/ examples/ tests/` ist leer. + +| Paket | Commit | Anmerkung | +|-------|--------|-----------| +| A.1 `performance-no-int-to-ptr` (4) | `01e2856` | `std::bit_cast`; zusätzlich `RingBufferShm` auf das Offset-Muster zurückgeführt (`dff0ae7`) | +| A.2a `startsWith`, `prepareStatement`, `parseAndScale` | `2f060d6` | `parseAndScale` nimmt jetzt eine `MemoryUnit`-Tabelle statt eines zweiten Strings | +| A.2b `handleCommandInput`/`handleTabInput` | `2f060d6` | Rückgabe `InputOutcome` | +| A.2c `sqlite3_exec`-Callback | `095b3e1` | `execute()` nutzt `prepare_v2`/`step` mit `pzTail`-Schleife; Regressionstest für mehrstellige Skripte | +| A.2d `PropertyRepository::allOf` | `cbd7c36` | `PropertyFilter`-Aggregat, 21 Aufrufstellen + Mock | +| A.3 `performance-unnecessary-value-param` | `c3717fd` | unbenutzter `GroupRepository`-Parameter entfernt | +| A.4 `bugprone-exception-escape` | `c0356d2` | explizit gedefaulteter Destruktor | + +### Teil C — `std::source_location`: vollständig + +`grep -rn __FILE__ src/ tests/ examples/ benchmarks/` ist leer. + +| Punkt | Commit | +|-------|--------| +| `LOG_*` + `log()`/`logImpl()` auf `std::source_location` | `80c1bdd` | +| `registerProperty` mit Default-Argument `std::source_location::current()` (19 Aufrufstellen) | `80c1bdd` | +| `DEFINE_PROPERTY` | `80c1bdd` | +| `tests/core/LoggerServiceTest.cpp` (25 Aufrufe) | `80c1bdd` | +| `CHANGELOG.md` (`registerProperty`, `PropertyFilter`, `prepareStatement`, `MemorySize`) | `8938853` | + +`LINTEL_FUNCTION` ist entfallen; `TypeName.h` behält `__PRETTY_FUNCTION__`, weil ein +Compile-Zeit-Typname nicht aus einem Laufzeitobjekt kommen kann. + +### Teil B — Integration: 9 von 14 Punkten umgesetzt + +| Punkt | Status | Commit | +|-------|--------|--------| +| B.2 #1 EventBus (Controller, Api, CLI, swagger) | **umgesetzt**, 7 Routen | `61e3b0f` | +| B.2 #2 `SharedMemoryCleanupService` in der DI + `POST /shm/cleanup` | **umgesetzt** | `8a7a604` | +| B.2 #3 `recoverOrphans` → `POST /messageQueue/recover` | **umgesetzt** | `8a7a604` | +| B.2 #4 `statsOf` im `ISharedMemoryService` + `GET /shm/stats/{segment}` | **umgesetzt** | `8a7a604` | +| B.2 #5 `SharedMemoryService::recover()` | **umgesetzt**: `POST /shm/recover` (nur Admin), `SharedMemoryApi::recoverOf()`, CLI `recover_segments`, Doc-Kommentar korrigiert | `d1c8253` | +| B.2 #10 `SharedMemoryRepositoryRegistry` | **umgesetzt**: `SharedMemoryRepository::of()` dedupliziert über die Registry; Header nach `core/shm/` verschoben | `7419464` | +| B.3 sieben CLI-/Swagger-Defekte | **umgesetzt** | `6a3f960`, `fc6ad98` | +| B.4 #2 Accessor-Rebinding | **umgesetzt**: `ShmSegmentAccessor` trägt den Segmentnamen (40 B) und löst ihn prozesslokal auf | `475c822` | +| B.4 #1 `MessageQueueShmCore` prozessübergreifend | **umgesetzt**: Header + Ring liegen unter einem Namen im Segment, Claim-Protokoll, Fork-Test | `475c822` | +| B.4 #3 `RingBufferShm` absolute Adresse | **umgesetzt**, mit Regressionstest | `dff0ae7` | +| übrige Punkte | offen | siehe Backlog | + +Entscheidungen aus §5, wie umgesetzt: + +1. `SharedMemoryService::recover()` → **Endpoint gebaut** (`POST /shm/recover`). +2. `SharedMemoryRepositoryRegistry` → **verdrahtet** über `SharedMemoryRepository::of()`. +3. EventBus-Publish → **eigenes Recht** `Publisher-EventBus`; `User-EventBus` ist nur noch lesend. +4. B.4 → **behoben** (Accessor-Bindung + geteilter Queue-Header). + +Zusätzlich, weil es im Plan fehlte: `AdminController` hatte weder Api noch CLI. Neu sind +`AdminApi` und `AdminCliComponent` (`reload_tls`, `log_level --level`), Commit `4da3f2e`. +Das Publish-Recht ist `2b04cbf`. + +### Nicht im ursprünglichen Plan: drei Fehler, die beim Testen auffielen + +Alle drei blockierten `scripts/run-example.sh` bzw. hinterließen Prozesse und sind behoben: + +| Fehler | Ursache | Commit | +|--------|---------|--------| +| `bin/main` startete gar nicht (`terminate called … feature ''`) | `Feature::m_name` war ein `std::string_view` auf ein temporäres `std::string`; `verifyFeatureDependencies()` verglich danach freigegebene Stack-Bytes | `9db4db7` | +| `export_repository --file` scheiterte auf Windows | jede `~` wurde durch das Home-Verzeichnis ersetzt, auch in Kurznamen wie `RUNNER~1` | `de3be99` | +| beide `worker` überlebten das Beenden von `main` | Container-Zyklus (`VirtualGroupBootstrapPlugin` hielt die Container-`shared_ptr`, in der sie selbst als Singleton lebt) → kein Destruktor lief; `ProcessService` stoppte seine Kinder nur im Destruktor | `d0aff6c`, `e283f41`, `fbc17c6` | + +### Backlog — die verbleibenden Punkte + +| Prio | Punkt | Ziel | Aufwand | Voraussetzung | +|------|-------|------|---------|---------------| +| 1 | **B.2 #6** MQ-Überlauf-/Dead-Letter-Policy | `MessageQueue` und `MessageQueueService` reichen `setOverflowPolicy`/`setOverflowTimeout`/`setDeadLetterQueueName` an den Core durch; `PUT /messageQueue/policy/{name}` + CLI | mittel | – | +| 2 | **B.2 #8** `ProcessGroup`-Lifecycle | fünf Routen (`start`, `restart`, `stop`, `terminate`, `detach`) analog zu den `Process`-Routen; `ProcessService` hat alle Methoden, `ProcessApi`/`ProcessCliComponent` ziehen nach | mittel | – | +| 3 | **B.2 #9** `HistoryController` → `HistoryService` | Controller fragt den Service statt des Repositories | klein | – | +| 4 | **B.2 #7** `MessageQueueRequestReply` | API + Route (`POST /messageQueue/requestReply/{name}`), Timeout als Property | mittel | – | +| 5 | **B.2 #11 + A.1.5** `ShmView`/`ShmSlice`, Lifecycle-Hooks, typisierte Payloads, Segmentbasis-Adressierung | exponiert die restlichen SHM-Bausteine; ersetzt `std::bit_cast` durch `reinterpret_cast(segmentBase + offset)` und entfernt die formale Unsicherheit aus A.1 | groß | B.4 (erledigt) | +| – | *Kandidat, nicht im Plan* | Paging-Items von `/messageQueue` liefern `MESSAGES = 0` (`MessageQueuesPageDto` reicht `0` durch), obwohl `/messageQueue/stats` dasselbe Payload mit echten Werten verspricht | klein | – | + +### Layout-Marker: Verwerfen statt Migrieren + +Der Marker (`lintel_segment_layout`, Wert 2) verhindert, dass ein Segment aus einem älteren +Build mit dem neuen Objektlayout fehlinterpretiert wird. Die Entscheidung liegt beim +Betreiber (`docs/shm-migration-strategy.md`): + +- **Default: Startabbruch.** `std::runtime_error` mit Segmentnamen und Pfad, die Datei + bleibt unangetastet. +- **`LINTEL_SHM_RECREATE=1`** oder `SegmentLayoutPolicy::set(Recreate)`: Inhalt verwerfen. +- **`SegmentLayoutPolicy::setMigrator(...)`**: der Hook läuft, *bevor* etwas gelöscht wird, + sieht die unveränderte Datei und entscheidet per Rückgabewert; er kann die Payloads + vorher exportieren. + +Betroffen ist nur, was ausschließlich im Segment lebt (Payloads von +`shm::SharedMemoryRepository`, EventBus-Ringe, SHM-Queue-Inhalte). Properties (DB bzw. +XML — der SHM-Repository ist ein Spiegel), Users, Groups und History liegen außerhalb. +Vier Tests pinnen das Verhalten: Abbruch, Recreate per Env-Variable, Migrator aufgerufen +mit Pfad/Versionen, Migrator verweigert (Datei bleibt erhalten). + +### Offene Entscheidungen + +Keine aus §5 mehr offen. Neu zu entscheiden, sobald #5 angegangen wird: ob die +Segmentbasis-Adressierung den Accessor-Namen ersetzt oder nur ergänzt. + +### Verifikationsstand + +| Prüfung | Ergebnis | +|---------|----------| +| Build `build/build/Release` (GCC 14, `-Werror`) | grün | +| Build `build/debug-nonunity` (Debug, Unity aus) | grün | +| `ctest --label-regex unit` | 1276/1276 grün (22 PostgreSQL-Tests übersprungen) | +| `ctest` im TSan-Build (`build/tsan`) | 1247/1247 grün (Stand vor den letzten Fixes) | +| `clang-tidy` mit CI-Checkliste auf allen geänderten Quellen | 0 Findings | +| `scripts/check-format.sh master`, `check-namespace.sh`, `check-swagger.sh` (48 Pfade), `check-secrets.sh` | grün | +| `scripts/run-example.sh` (main, worker) | startet, beendet Kinder sauber | +| Alte Segmentdateien | werden über den Layout-Marker erkannt und neu angelegt (im Log verifiziert) | diff --git a/docs/shm-migration-strategy.md b/docs/shm-migration-strategy.md new file mode 100644 index 00000000..796840b0 --- /dev/null +++ b/docs/shm-migration-strategy.md @@ -0,0 +1,102 @@ +# Migrationsstrategie für Shared-Memory-Segmente + +Stand: 22. Oktober 2026 · gilt ab Layout-Version 2 (`lintel_segment_layout`) + +Ein Segment ist eine Datei mit einem Layout, das die Bibliothek definiert. Ändert sich +dieses Layout (in 0.2 zum Beispiel: `ShmSegmentAccessor` von einem Zeiger auf 40 Byte, +neuer Queue-Header), können die Bytes eines alten Segments nicht mehr gedeutet werden. +Dieses Dokument beschreibt, welche Daten das trifft, wer entscheidet und wie eine +Migration aussieht. + +## 1. Welche Daten liegen im Segment? + +| Objekt | Autoritativ? | Verlust beim Neuanlegen | +|--------|--------------|-------------------------| +| `SharedMemoryPropertyRepository` (Spiegel) | **nein** – Werte stehen in DB bzw. Bootstrap-XML, der SHM-Repository ist eine Kopie für prozessübergreifende Änderungen | wird beim Start neu aufgebaut (`PropertyService::init`) | +| `shm::SharedMemoryRepository` (typisierte Repositories) | **ja** – das ist Nutzdaten-API | **Datenverlust** | +| `RingBufferShm` / `QueueShm` / `ArrayShm` als Payload | **ja**, wie oben | **Datenverlust** | +| `EventBus` (Ringe, Sticky-Werte, Subscriptions) | nein, flüchtig | verloren, per Definition | +| `MessageQueueShmCore` (Header + Ring) | Nachrichten sind flüchtig | verloren | +| Users, Groups, Properties, History | liegen **außerhalb** (DB/XML) | unberührt | + +Damit ist die Sorge berechtigt: alles, was über `SharedMemoryRepository` abgelegt +wird, existiert nur im Segment. Die Bibliothek darf das nicht eigenmächtig löschen. + +## 2. Wer entscheidet? + +`SharedMemoryService` fragt beim Öffnen den Layout-Marker ab. Bei einer Abweichung gilt +folgende Reihenfolge (`SharedMemoryService.cpp`, `create()`): + +1. **Migrator registriert** (`lintel::shm::SegmentLayoutPolicy::setMigrator`) → er + entscheidet. `true` = Inhalte dürfen weg, `false` = Start abbrechen. +2. **Policy `Recreate`** (`set(LayoutMismatchPolicy::Recreate)`) oder + `LINTEL_SHM_RECREATE=1` → Inhalt wird verworfen. +3. Sonst **`Fail`** (Default): `std::runtime_error` mit Segmentname und Dateipfad, die + Datei bleibt unangetastet. + +Der Migrator läuft, **bevor** irgendetwas gelöscht wird, und bekommt den Pfad der +aktuellen Datei sowie die gefundene und die erwartete Layout-Version. + +## 3. Einen Migrator schreiben + +```cpp +#include + +// einmalig beim Start, vor dem DI-Container +lintel::shm::SegmentLayoutPolicy::setMigrator( + [](const std::filesystem::path &stale, std::uint64_t found, std::uint64_t current) { + // 1) alten Stand lesen - die Datei ist noch da und noch nicht verändert + boost::interprocess::managed_mapped_file old(boost::interprocess::open_only, stale.c_str()); + for (const auto &name : old.get_named_objects_handles()) { /* ... */ } + // 2) was wichtig ist, dauerhaft ablegen (JSON, DB, eigene Datei) + // z. B. die Payloads der typisierten Repositories über ihre serialize() + exportToJson(old, "shm-migration-" + stale.filename().string() + "-v" + std::to_string(found) + ".json"); + // 3) true = der Neuanfang ist erlaubt, false = Start abbrechen + return true; + }); +``` + +Regeln für den Migrator: + +- Er darf die Datei **lesen**, nicht schreiben; die Bibliothek hat sie zu diesem + Zeitpunkt bereits gemappt. +- Er muss idempotent sein: schlägt der Start danach fehl, läuft er beim nächsten + Versuch erneut. +- Er sollte protokollieren, was übernommen und was verworfen wurde (der Report gehört + ins Log, nicht in die Ausnahme). +- Ohne Migrator und ohne `LINTEL_SHM_RECREATE` passiert **nichts** – das ist die + sichere Voreinstellung für Produktion. + +## 4. Grenze der aktuellen Umsetzung + +Die Entscheidung gilt **pro Segment, nicht pro Objekt**: die Objekte im Segment tragen +keine eigene Version, also kann der Migrator nur „alles retten" oder „alles aufgeben". +Für die typisierten Repositories ist das ausreichend, weil ihr Payload ohnehin über +`serialize()`/`deserialize()` der DTOs geht. + +## 5. Nächste Stufen (Backlog) + +| Stufe | Inhalt | Nutzen | +|-------|--------|--------| +| **2** | `ShmObjectHeader { magic, version, payloadBytes }` vor jedem benannten Objekt; pro Typ eine Migrationsfunktion `v(n) → v(n+1)` | echte Teil-Migration, ohne Export/Import | +| **3** | Persistenzklasse pro Objekt beim Registrieren: `Volatile` (darf weg), `Derived` (aus DB/XML rekonstruierbar), `Persistent` (muss migriert werden, sonst Startabbruch) | die Bibliothek weiß selbst, was sie verwerfen darf | +| **4** | stabile Payload-ABI: Nutzdaten als versionierte Blobs, interne Buchhaltung (Accessor, Header) versioniert getrennt | Layout-Änderungen der Bibliothek invalidieren Nutzdaten nie wieder | + +Stufe 2 ist die Voraussetzung für 3 und 4; ohne sie bleibt es beim Segment als +Ganzes. + +## 6. Betrieb + +```bash +# Produktion: nichts anfassen, Start bricht mit Pfad und Anleitung ab +./bin/main + +# Entwicklung: Segmente dürfen neu angelegt werden +LINTEL_SHM_RECREATE=1 ./bin/main + +# Migration: Migrator im Programm registrieren (siehe oben), dann ohne Env-Variable starten +``` + +Die Segmentdateien liegen im konfigurierten Verzeichnis (``, +Standard `~/dev/shm`). Ein `POST /shm/cleanup` entfernt verwaiste *Repositories*, keine +Segmente – Segmente werden nur über den Layout-Pfad oder manuell ersetzt. diff --git a/examples/main.cpp b/examples/main.cpp index 94abe770..2d1385bb 100644 --- a/examples/main.cpp +++ b/examples/main.cpp @@ -121,11 +121,6 @@ class ArrayDao : public lintel::JsonSerializable bool deserialize(const rapidjson::Value &obj) override { return hasAllRecordKeys(obj, m_shape); } }; -// The implicitly generated assignment can theoretically throw for shm::String -// (bad_alloc). In this example the DTOs are filled exactly once and only read / -// serialized afterwards - a dedicated operator= with noexcept signature would -// only document a falsehood here. -// NOLINTNEXTLINE(bugprone-exception-escape) struct TestDataDto { lintel::shm::String m_name; @@ -134,6 +129,13 @@ struct TestDataDto lintel::shm::String m_location; int32_t m_age = 0; + // The shm::String members allocate from the segment, so a move cannot promise + // not to throw either. Declaring the destructor suppresses the implicitly + // generated move constructor / move assignment (moving would then silently + // terminate on bad_alloc); copies fall back to the copy operations, which is + // what every use in this example does anyway. + ~TestDataDto() = default; + bool operator==(const TestDataDto &rhs) const { return m_name == rhs.m_name && m_addr == rhs.m_addr && m_plz == rhs.m_plz && m_location == rhs.m_location diff --git a/examples/worker.cpp b/examples/worker.cpp index daa38064..0f75525f 100644 --- a/examples/worker.cpp +++ b/examples/worker.cpp @@ -36,23 +36,31 @@ class WorkerService : public lintel::AbstractService { private: - std::shared_ptr m_container; + /** Weak on purpose: the container owns this service, so a shared_ptr here + * would close a cycle and the container would never be destroyed. */ + std::weak_ptr m_container; std::shared_ptr m_processName; std::atomic_bool m_running = true; std::thread m_listener; std::thread m_historyThread; public: - WorkerService(std::shared_ptr container, std::shared_ptr processName) - : AbstractService(processName->getProcessName()), m_container(std::move(container)), + WorkerService(const std::shared_ptr &container, + std::shared_ptr processName) + : AbstractService(processName->getProcessName()), m_container(container), m_processName(std::move(processName)) {} void onInitialize() override { - auto propertyService = m_container->resolve(); - auto sharedMemoryRepository = m_container->resolve(); - auto eventBusService = m_container->resolve(); + const std::shared_ptr container = m_container.lock(); + if (container == nullptr) { + LOG_ERROR("container is gone - worker cannot initialize"); + return; + } + auto propertyService = container->resolve(); + auto sharedMemoryRepository = container->resolve(); + auto eventBusService = container->resolve(); // unique subscriber name per worker instance (pid based) const std::string subscriber = m_processName->getProcessName() + "-" + std::to_string(lintel::currentProcessId()); @@ -62,7 +70,8 @@ class WorkerService : public lintel::AbstractService // property is registered, so that a stale default is never written back int32_t initialValue = 2; try { - auto mirrored = sharedMemoryRepository->allOf("main", "SchedulerService", "__DEFAULT", "numberOfThreads"); + auto mirrored = sharedMemoryRepository->allOf( + lintel::PropertyFilter{ "main", "SchedulerService", "__DEFAULT", "numberOfThreads" }); if (!mirrored.empty()) { initialValue = std::stoi(mirrored.front()->toString()); } } catch (const std::exception &exception) { LOG_WARN("cannot read numberOfThreads from shared memory: {}", exception.what()); @@ -75,7 +84,8 @@ class WorkerService : public lintel::AbstractService // change made over HTTP is applied by the worker through the event bus int32_t loginTimeout = 5; try { - auto mirrored = sharedMemoryRepository->allOf("main", "AuthService", "__DEFAULT", "m_timeoutLogin"); + auto mirrored = + sharedMemoryRepository->allOf(lintel::PropertyFilter{ "main", "AuthService", "__DEFAULT", "m_timeoutLogin" }); if (!mirrored.empty()) { loginTimeout = std::stoi(mirrored.front()->toString()); } } catch (const std::exception &exception) { LOG_WARN("cannot read m_timeoutLogin from shared memory: {}", exception.what()); @@ -112,7 +122,7 @@ class WorkerService : public lintel::AbstractService // produce a few test history entries every 3 seconds so that main // picks them up from the IPC queue and persists them - auto historyService = m_container->resolve(); + auto historyService = container->resolve(); const std::string workerPid = std::to_string(lintel::currentProcessId()); m_historyThread = std::thread([this, historyService, subscriber, workerPid]() { int counter = 0; diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 4a5d5780..4e5f6403 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -128,7 +128,6 @@ set(headers include/lintel/core/services/ISharedMemoryService.h include/lintel/core/services/PropertyRegistration.h include/lintel/features/base/services/SharedMemoryService.h - include/lintel/features/base/services/SharedMemoryRepositoryRegistry.h include/lintel/features/base/services/ShmConstructors.h include/lintel/core/services/SignalService.h include/lintel/core/services/StopWatchService.h @@ -173,6 +172,7 @@ set(headers include/lintel/features/base/configuration/SharedMemorySegmentComponent.h include/lintel/features/base/configuration/SharedMemorySegmentEntry.h # features - base - controller + include/lintel/features/base/controller/EventBusDtos.h include/lintel/features/base/controller/GroupDto.h include/lintel/features/base/controller/HistoryDto.h include/lintel/features/base/controller/HistoryDtos.h @@ -219,8 +219,10 @@ set(headers include/lintel/features/base/events/EventBus.h include/lintel/core/shm/BoostSegmentAllocator.h include/lintel/core/shm/ShmSegmentAccessor.h + include/lintel/core/shm/SegmentLayoutPolicy.h include/lintel/core/shm/ShmView.h include/lintel/core/shm/SharedMemoryRepository.h + include/lintel/core/shm/SharedMemoryRepositoryRegistry.h include/lintel/core/shm/containers/ArrayShm.h include/lintel/core/shm/containers/QueueShm.h include/lintel/core/shm/containers/RingBufferShm.h @@ -266,6 +268,8 @@ set(headers include/lintel/features/cli/services/InputService.h include/lintel/features/cli/services/TerminalService.h include/lintel/features/cli/utils/CommandLineUtils.h + include/lintel/features/cli/components/AdminCliComponent.h + include/lintel/features/cli/components/EventBusCliComponent.h include/lintel/features/cli/components/HistoryCliComponent.h include/lintel/features/cli/components/MessageQueueCliComponent.h include/lintel/features/cli/components/ProcessCliComponent.h @@ -293,7 +297,10 @@ set(headers include/lintel/features/http/service/HttpClientHelper.h include/lintel/features/http/service/Server.h include/lintel/features/http/service/TlsReloadRegistry.h + include/lintel/features/http/controllers/AdminApi.h include/lintel/features/http/controllers/AdminController.h + include/lintel/features/http/controllers/EventBusApi.h + include/lintel/features/http/controllers/EventBusController.h include/lintel/features/http/controllers/HistoryApi.h include/lintel/features/http/controllers/HistoryController.h include/lintel/features/http/controllers/MessageQueueApi.h @@ -425,6 +432,7 @@ set(sources features/base/configuration/SharedMemorySegmentComponent.cpp features/base/configuration/SharedMemorySegmentEntry.cpp # features - base - controller + features/base/controller/EventBusDtos.cpp features/base/controller/GroupDto.cpp features/base/controller/HistoryDto.cpp features/base/controller/HistoryDtos.cpp @@ -488,6 +496,8 @@ set(sources features/cli/services/InputService.cpp features/cli/services/TerminalService.cpp features/cli/utils/CommandLineUtils.cpp + features/cli/components/AdminCliComponent.cpp + features/cli/components/EventBusCliComponent.cpp features/cli/components/HistoryCliComponent.cpp features/cli/components/MessageQueueCliComponent.cpp features/cli/components/ProcessCliComponent.cpp @@ -510,7 +520,10 @@ set(sources features/http/service/HttpStatusCode.cpp features/http/service/Server.cpp features/http/service/TlsReloadRegistry.cpp + features/http/controllers/AdminApi.cpp features/http/controllers/AdminController.cpp + features/http/controllers/EventBusApi.cpp + features/http/controllers/EventBusController.cpp features/http/controllers/HistoryApi.cpp features/http/controllers/HistoryController.cpp features/http/controllers/MessageQueueApi.cpp diff --git a/src/app/BaseFeature.cpp b/src/app/BaseFeature.cpp index 95cd0d67..e5963b75 100644 --- a/src/app/BaseFeature.cpp +++ b/src/app/BaseFeature.cpp @@ -36,6 +36,7 @@ #include "lintel/features/base/services/ProcessArgumentService.h" #include "lintel/features/base/services/ProcessService.h" #include "lintel/features/base/services/SchedulerService.h" +#include "lintel/features/base/services/SharedMemoryCleanupService.h" #include "lintel/features/base/services/SharedMemorySegmentManager.h" #include "lintel/features/base/services/SharedMemoryService.h" @@ -152,6 +153,11 @@ void BaseFeature::registerTypes(lintel::di::ContainerBuilder &builder, .as() .asSelf() .singleInstance(); + // The cleanup service only needs the scheduler and the shared memory service; + // its optional database connection stays unset (the container cannot inject + // DatabaseConnectionEntry), so an orphan is removed from the segment but a + // matching row in shared_memory_repositories is not reconciled. + builder.registerType().as().asSelf().singleInstance(); builder.registerType().as().asSelf().singleInstance(); builder.registerType().singleInstance(); builder.registerType().as().asSelf().singleInstance(); diff --git a/src/app/CommandLineFeature.cpp b/src/app/CommandLineFeature.cpp index 1030813c..6182c0a1 100644 --- a/src/app/CommandLineFeature.cpp +++ b/src/app/CommandLineFeature.cpp @@ -2,6 +2,8 @@ #include "lintel/core/di/Container.h" +#include "lintel/features/cli/components/AdminCliComponent.h" +#include "lintel/features/cli/components/EventBusCliComponent.h" #include "lintel/features/cli/components/HistoryCliComponent.h" #include "lintel/features/cli/components/MessageQueueCliComponent.h" #include "lintel/features/cli/components/ProcessCliComponent.h" @@ -30,7 +32,9 @@ void CommandLineFeature::registerTypes(lintel::di::ContainerBuilder &builder) builder.registerType().as().asSelf().singleInstance(); builder.registerType().as().asSelf().singleInstance(); builder.registerType().as().asSelf().singleInstance(); + builder.registerType().as().asSelf().singleInstance(); builder.registerType().as().asSelf().singleInstance(); + builder.registerType().as().asSelf().singleInstance(); } void CommandLineFeature::initialize(std::shared_ptr container) diff --git a/src/app/HttpFeature.cpp b/src/app/HttpFeature.cpp index bff66eaa..9cc43936 100644 --- a/src/app/HttpFeature.cpp +++ b/src/app/HttpFeature.cpp @@ -2,7 +2,10 @@ #include "lintel/core/di/Container.h" +#include "lintel/features/http/controllers/AdminApi.h" #include "lintel/features/http/controllers/AdminController.h" +#include "lintel/features/http/controllers/EventBusApi.h" +#include "lintel/features/http/controllers/EventBusController.h" #include "lintel/features/http/controllers/HistoryApi.h" #include "lintel/features/http/controllers/HistoryController.h" #include "lintel/features/http/controllers/MessageQueueApi.h" @@ -23,6 +26,7 @@ void HttpFeature::registerTypes(lintel::di::ContainerBuilder &builder) { builder.registerType().singleInstance(); builder.registerType().singleInstance(); + builder.registerType().singleInstance(); builder.registerType().as().as().asSelf().singleInstance(); builder.registerType().singleInstance(); builder.registerType().as().as().asSelf().singleInstance(); @@ -34,6 +38,8 @@ void HttpFeature::registerTypes(lintel::di::ContainerBuilder &builder) builder.registerType().as().as().asSelf().singleInstance(); builder.registerType().singleInstance(); builder.registerType().as().as().asSelf().singleInstance(); + builder.registerType().singleInstance(); + builder.registerType().as().as().asSelf().singleInstance(); } void HttpFeature::initialize(std::shared_ptr container) diff --git a/src/app/plugins/VirtualGroupBootstrapPlugin.cpp b/src/app/plugins/VirtualGroupBootstrapPlugin.cpp index af4249e8..81c86e65 100644 --- a/src/app/plugins/VirtualGroupBootstrapPlugin.cpp +++ b/src/app/plugins/VirtualGroupBootstrapPlugin.cpp @@ -11,8 +11,8 @@ namespace lintel { VirtualGroupBootstrapPlugin::VirtualGroupBootstrapPlugin( const std::shared_ptr &connectionConfigurations, - std::shared_ptr container) - : m_container(std::move(container)), m_connectionEntry(connectionConfigurations->ofDefault()) + const std::shared_ptr &container) + : m_container(container), m_connectionEntry(connectionConfigurations->ofDefault()) {} BootstrapSequence VirtualGroupBootstrapPlugin::getPriority() { return BootstrapSequence::VirtualGroups; } @@ -23,7 +23,12 @@ void VirtualGroupBootstrapPlugin::onStart() LOG_INFO("no default database connection - skip virtual group bootstrap"); return; } - auto groupProviders = m_container->resolveAll(); + const std::shared_ptr container = m_container.lock(); + if (container == nullptr) { + LOG_ERROR("container is gone - skip virtual group bootstrap"); + return; + } + auto groupProviders = container->resolveAll(); std::set groups; for (const auto &groupProvider : groupProviders) { std::vector temp = groupProvider->provide(); diff --git a/src/core/persistence/sqlite3/Connection.cpp b/src/core/persistence/sqlite3/Connection.cpp index f83f4dc0..22e991bf 100644 --- a/src/core/persistence/sqlite3/Connection.cpp +++ b/src/core/persistence/sqlite3/Connection.cpp @@ -12,8 +12,8 @@ #include "lintel/core/persistence/sqlite3/Connection.h" -#include #include +#include #include "lintel/core/configuration/DatabaseConnectionEntry.h" #include "lintel/core/exceptions/SQLException.h" @@ -28,6 +28,12 @@ namespace { /// In-memory databases (and empty paths) must not switch to WAL. constexpr const char *kInMemoryDatabase = ":memory:"; + /// Releases a prepared statement; used as unique_ptr deleter so every path finalizes. + struct StatementFinalizer + { + void operator()(sqlite3_stmt *stmt) const noexcept { sqlite3_finalize(stmt); } + }; + void execPragma(sqlite3 *db, const std::string &pragma) { char *errorMessage = nullptr; @@ -137,10 +143,10 @@ namespace sqlite { } } - // NOLINTNEXTLINE(bugprone-easily-swappable-parameters) — distinct names (queryName, query); signature is public API. - std::shared_ptr Connection::prepareStatement(const std::string &queryName, const std::string &query) + std::shared_ptr Connection::prepareStatement(std::string_view queryName, const std::string &query) { - auto found = m_preparedStatements.find(queryName); + std::string const name(queryName); + auto found = m_preparedStatements.find(name); if (found != m_preparedStatements.end()) { throw db::SQLException("SQLite sqlite3_stmt is initialized => abort"); } sqlite3_stmt *stmt = nullptr; @@ -148,7 +154,7 @@ namespace sqlite { if (rc != SQLITE_OK) { throw db::SQLException("SQLite prepare exception: " + getErrorMessage()); } - m_preparedStatements.insert({ queryName, stmt }); + m_preparedStatements.insert({ name, stmt }); return std::make_shared(); } @@ -252,35 +258,45 @@ namespace sqlite { std::shared_ptr Connection::execute(const std::string &statement) const { - char *errorMessage = nullptr; std::shared_ptr result = std::make_shared(); - // NOLINTNEXTLINE(bugprone-easily-swappable-parameters) — sqlite3_exec callback signature is fixed by the C API. - std::function func = [&](int argc, char **argv, char **column) { - db::Arguments arguments(db::ConnectionType::SQLite); - for (int i = 0; i < argc; i++) { - std::string value; - if (argv[i] != nullptr) { value = argv[i]; } - db::Argument argument(value, column[i]); - arguments.add(argument); + + // sqlite3_exec ran every statement of the script. sqlite3_prepare_v2 only prepares the first + // one and returns the rest of the script in `tail`, so the script has to be walked statement + // by statement until no input is left. + const char *tail = statement.c_str(); + while (tail != nullptr && *tail != '\0') { + const char *const current = tail; + sqlite3_stmt *stmt = nullptr; + const int prepareRc = sqlite3_prepare_v2(m_db, current, -1, &stmt, &tail); + if (prepareRc != SQLITE_OK) { throw db::SQLException("SQLite execution error: " + getErrorMessage()); } + // A successful prepare yields no statement for trailing whitespace or comments only. + if (stmt == nullptr) { break; } + // finalize on every path, including the throwing ones + std::unique_ptr const guard(stmt); + + int step = SQLITE_ROW; + while (step == SQLITE_ROW) { + step = sqlite3_step(stmt); + if (step != SQLITE_ROW) { continue; } + db::Arguments arguments(db::ConnectionType::SQLite); + const int count = sqlite3_column_count(stmt); + for (int i = 0; i < count; i++) { + const unsigned char *text = sqlite3_column_text(stmt, i); + const char *name = sqlite3_column_name(stmt, i); + std::string value; + if (text != nullptr) { value = std::string(reinterpret_cast(text)); } + if (name == nullptr) { continue; } + db::Argument argument(value, name); + arguments.add(argument); + } + result->add(arguments); } - result->add(arguments); - }; - void *funcPtr = static_cast(&func); - const int rc = sqlite3_exec(m_db, statement.c_str(), callBack, funcPtr, &errorMessage); + if (step != SQLITE_DONE) { throw db::SQLException("SQLite execution error: " + getErrorMessage()); } - if (rc != SQLITE_OK) { - std::string const errMsg = (errorMessage); - sqlite3_free(errorMessage); - throw db::SQLException("SQLite execution error: " + errMsg); + // Guard against a prepare that reports success without consuming any input. + if (tail <= current) { break; } } return result; } - - int Connection::callBack(void *funcPtr, int argc, char **argv, char **column) - { - auto *func = static_cast *>(funcPtr); - func->operator()(argc, argv, column); - return 0; - } }// namespace sqlite }// namespace lintel \ No newline at end of file diff --git a/src/core/services/LoggerService.cpp b/src/core/services/LoggerService.cpp index b15dea76..77eea9b9 100644 --- a/src/core/services/LoggerService.cpp +++ b/src/core/services/LoggerService.cpp @@ -260,9 +260,7 @@ void LoggerService::initSingleton(const std::shared_ptr &processInf void LoggerService::initSingleton2() { m_instance = std::make_unique(); } -void LoggerService::logImpl(const char *file, - int line, - const char *function, +void LoggerService::logImpl(const std::source_location &location, LogLevel lvl, std::string_view message, std::format_args args) @@ -274,15 +272,16 @@ void LoggerService::logImpl(const char *file, try { text = std::vformat(message, args); } catch (const std::exception &ex) { - text = std::format("log formatting failed: {} [{}:{}]", ex.what(), file, line); + text = std::format("log formatting failed: {} [{}:{}]", ex.what(), location.file_name(), location.line()); } + const spdlog::source_loc source{ location.file_name(), static_cast(location.line()), location.function_name() }; if (g_requestId.empty()) { - m_logger->log(spdlog::source_loc{ file, line, function }, level, std::string_view(text)); + m_logger->log(source, level, std::string_view(text)); return; } // the correlation id of the request the calling thread is serving: it ties the log lines // of one request together and matches the X-Request-Id header the server answered with text = std::format("[{}] {}", g_requestId, text); - m_logger->log(spdlog::source_loc{ file, line, function }, level, std::string_view(text)); + m_logger->log(source, level, std::string_view(text)); } }// namespace lintel \ No newline at end of file diff --git a/src/core/utils/MemorySize.cpp b/src/core/utils/MemorySize.cpp index b98844bd..9a447fe7 100644 --- a/src/core/utils/MemorySize.cpp +++ b/src/core/utils/MemorySize.cpp @@ -1,7 +1,12 @@ #include "lintel/core/utils/MemorySize.h" +#include +#include #include #include +#include +#include +#include #include "lintel/core/utils/StringUtils.h" @@ -13,42 +18,49 @@ namespace { constexpr std::size_t kBytesPerTera = 1000000000000; constexpr std::size_t kBytesPerPeta = 1000000000000000; - /** Parses a numeric prefix and validates it, optionally scaling it by a multiplier. */ - // Anonymous-namespace helper: the two string parameters have distinct roles (full input for the - // error message vs. the numeric part to parse) and every call site passes them in this order. - // NOLINTNEXTLINE(bugprone-easily-swappable-parameters) - std::size_t parseAndScale(const std::string &input, const std::string &prefix, std::size_t multiplier) + /** Unit suffix of a memory size and the number of bytes it stands for. */ + struct MemoryUnit { - std::size_t pos = 0; + std::string_view m_suffix; + std::size_t m_bytesPerUnit; + }; + + /** Longer suffixes first, so that "1kB" matches kB instead of the trailing B. */ + constexpr std::array kUnits{ { + { "kB", kBytesPerKilo }, + { "MB", kBytesPerMega }, + { "GB", kBytesPerGiga }, + { "TB", kBytesPerTera }, + { "PB", kBytesPerPeta }, + { "B", 1 }, + } }; + + /** Parses the number in front of the unit suffix and scales it to bytes. */ + std::size_t parseAndScale(std::string_view input, const MemoryUnit &unit) + { + std::string_view const number = input.substr(0, input.size() - unit.m_suffix.size()); unsigned long value = 0; - try { - value = std::stoul(prefix, &pos); - } catch (const std::exception &) { - throw std::invalid_argument("invalid memory size: '" + input + "'"); + const auto [end, error] = std::from_chars(number.data(), number.data() + number.size(), value); + if (error == std::errc::result_out_of_range) { + throw std::invalid_argument(std::format("memory size overflow: '{}'", input)); } - if (pos != prefix.size()) { throw std::invalid_argument("invalid memory size: '" + input + "'"); } - if (value > (std::numeric_limits::max() / multiplier)) { - throw std::invalid_argument("memory size overflow: '" + input + "'"); + // Reject unparsable input as well as a parsed number followed by trailing garbage. + if (error != std::errc{} || end != number.data() + number.size()) { + throw std::invalid_argument(std::format("invalid memory size: '{}'", input)); } - return static_cast(value) * multiplier; + if (value > (std::numeric_limits::max() / unit.m_bytesPerUnit)) { + throw std::invalid_argument(std::format("memory size overflow: '{}'", input)); + } + return static_cast(value) * unit.m_bytesPerUnit; } }// namespace std::size_t MemorySize::deserialize(const std::string &size) { - bool isByte = StringUtils::endsWith(size, "B"); - bool isKiloByte = StringUtils::endsWith(size, "kB"); - bool isMegaByte = StringUtils::endsWith(size, "MB"); - bool isGigaByte = StringUtils::endsWith(size, "GB"); - bool isTeraByte = StringUtils::endsWith(size, "TB"); - bool isPetaByte = StringUtils::endsWith(size, "PB"); - if (isKiloByte) { return parseAndScale(size, size.substr(0, size.length() - 2), kBytesPerKilo); } - if (isMegaByte) { return parseAndScale(size, size.substr(0, size.length() - 2), kBytesPerMega); } - if (isGigaByte) { return parseAndScale(size, size.substr(0, size.length() - 2), kBytesPerGiga); } - if (isTeraByte) { return parseAndScale(size, size.substr(0, size.length() - 2), kBytesPerTera); } - if (isPetaByte) { return parseAndScale(size, size.substr(0, size.length() - 2), kBytesPerPeta); } - if (isByte) { return parseAndScale(size, size.substr(0, size.length() - 1), 1); } - return parseAndScale(size, size, 1); + for (const auto &unit : kUnits) { + if (StringUtils::endsWith(size, unit.m_suffix)) { return parseAndScale(size, unit); } + } + return parseAndScale(size, MemoryUnit{ "", 1 }); } std::string MemorySize::serialize(std::size_t byte) diff --git a/src/features/base/controller/EventBusDtos.cpp b/src/features/base/controller/EventBusDtos.cpp new file mode 100644 index 00000000..6388cf5e --- /dev/null +++ b/src/features/base/controller/EventBusDtos.cpp @@ -0,0 +1,190 @@ +#include "lintel/features/base/controller/EventBusDtos.h" + +#include "lintel/core/exceptions/HttpBadRequestException.h" +#include "lintel/core/utils/JsonUtils.h" + +#include +#include + +#include +#include + +namespace lintel { +namespace { + /** @param policy the drop policy + * @return the name used on the wire */ + const char *dropPolicyNameOf(DropPolicy policy) + { + switch (policy) { + case DropPolicy::Fail: + return "Fail"; + case DropPolicy::DropOldest: + return "DropOldest"; + case DropPolicy::DropNewest: + return "DropNewest"; + case DropPolicy::Block: + return "Block"; + } + return "Fail"; + } + + /** @param name the name used on the wire + * @return the drop policy, empty for an unknown name */ + std::optional dropPolicyOf(const std::string &name) + { + if (name == "Fail") { return DropPolicy::Fail; } + if (name == "DropOldest") { return DropPolicy::DropOldest; } + if (name == "DropNewest") { return DropPolicy::DropNewest; } + if (name == "Block") { return DropPolicy::Block; } + return std::nullopt; + } +}// namespace + +EventBusDto::EventBusDto(std::string name, const EventBusConfig &config) : m_name(std::move(name)), m_config(config) {} + +EventBusDto::EventBusDto(std::string name) : m_name(std::move(name)) {} + +void EventBusDto::markAvailable(const EventBusCounters &counters) +{ + m_available = true; + m_dropped = counters.dropped; + m_topics = counters.topics; + m_subscribers = counters.subscribers; + m_busDepth = counters.busDepth; + m_sequence = counters.sequence; +} + +const std::string &EventBusDto::getName() const { return m_name; } + +const std::optional &EventBusDto::getConfig() const { return m_config; } + +bool EventBusDto::isAvailable() const { return m_available; } + +std::uint64_t EventBusDto::getDropped() const { return m_dropped; } + +std::size_t EventBusDto::getTopics() const { return m_topics; } + +std::size_t EventBusDto::getSubscribers() const { return m_subscribers; } + +std::size_t EventBusDto::getBusDepth() const { return m_busDepth; } + +std::int64_t EventBusDto::getSequence() const { return m_sequence; } + +void EventBusDto::serialize(rapidjson::Writer *writer) const +{ + writer->StartObject(); + writer->String("NAME"); + writer->String(m_name.c_str()); + writer->String("AVAILABLE"); + writer->Bool(m_available); + // the configuration is omitted for a bus whose bootstrap entry is invalid + if (m_config.has_value()) { + const EventBusConfig &config = m_config.value(); + writer->String("BUS_CAPACITY"); + writer->Uint64(static_cast(config.busCapacity)); + writer->String("SUBSCRIBER_CAPACITY"); + writer->Uint64(static_cast(config.subscriberCapacity)); + writer->String("MAX_TOPICS"); + writer->Uint64(static_cast(config.maxTopics)); + writer->String("MAX_SUBSCRIBERS"); + writer->Uint64(static_cast(config.maxSubscribers)); + writer->String("MAX_PRIORITIES"); + writer->Uint64(static_cast(config.maxPriorities)); + writer->String("PUB_POLICY"); + writer->String(dropPolicyNameOf(config.pubPolicy)); + writer->String("STICKY"); + writer->Bool(config.sticky); + writer->String("STICKY_TTL_MS"); + writer->Uint(config.stickyTtlMs); + } + // the runtime counters are only known while the bus can be opened + if (m_available) { + writer->String("DROPPED"); + writer->Uint64(m_dropped); + writer->String("TOPIC_COUNT"); + writer->Uint64(static_cast(m_topics)); + writer->String("SUBSCRIBER_COUNT"); + writer->Uint64(static_cast(m_subscribers)); + writer->String("BUS_DEPTH"); + writer->Uint64(static_cast(m_busDepth)); + writer->String("SEQUENCE"); + writer->Int64(m_sequence); + } + writer->EndObject(); +} + +bool EventBusDto::deserialize(const rapidjson::Value &obj) +{ + if (!obj.IsObject()) { return false; } + std::string name; + if (!json::getString(obj, "NAME", name)) { return false; } + m_name = std::move(name); + json::getBool(obj, "AVAILABLE", m_available); + EventBusConfig config; + bool hasConfig = false; + std::uint64_t number = 0; + if (json::getUint64(obj, "BUS_CAPACITY", number)) { + config.busCapacity = static_cast(number); + hasConfig = true; + } + if (json::getUint64(obj, "SUBSCRIBER_CAPACITY", number)) { + config.subscriberCapacity = static_cast(number); + hasConfig = true; + } + if (json::getUint64(obj, "MAX_TOPICS", number)) { + config.maxTopics = static_cast(number); + hasConfig = true; + } + if (json::getUint64(obj, "MAX_SUBSCRIBERS", number)) { + config.maxSubscribers = static_cast(number); + hasConfig = true; + } + if (json::getUint64(obj, "MAX_PRIORITIES", number)) { + config.maxPriorities = static_cast(number); + hasConfig = true; + } + std::string policyName; + if (json::getString(obj, "PUB_POLICY", policyName)) { + const std::optional policy = dropPolicyOf(policyName); + if (!policy.has_value()) { return false; } + config.pubPolicy = policy.value(); + hasConfig = true; + } + if (json::getBool(obj, "STICKY", config.sticky)) { hasConfig = true; } + if (json::getUint64(obj, "STICKY_TTL_MS", number)) { + config.stickyTtlMs = static_cast(number); + hasConfig = true; + } + m_config = hasConfig ? std::optional(config) : std::nullopt; + json::getUint64(obj, "DROPPED", m_dropped); + if (json::getUint64(obj, "TOPIC_COUNT", number)) { m_topics = static_cast(number); } + if (json::getUint64(obj, "SUBSCRIBER_COUNT", number)) { m_subscribers = static_cast(number); } + if (json::getUint64(obj, "BUS_DEPTH", number)) { m_busDepth = static_cast(number); } + json::getInt64(obj, "SEQUENCE", m_sequence); + return true; +} + +EventBusesDto::EventBusesDto(std::vector eventBusDtos) : m_eventBusDtos(std::move(eventBusDtos)) {} + +const std::vector &EventBusesDto::getEventBusDtos() const { return m_eventBusDtos; } + +void EventBusesDto::serialize(rapidjson::Writer *writer) const +{ + writer->StartArray(); + for (const auto &eventBusDto : m_eventBusDtos) { eventBusDto.serialize(writer); } + writer->EndArray(); +} + +void EventBusesDto::deserialize(const std::string &json) +{ + rapidjson::Document document; + document.Parse(json.c_str()); + if (!document.IsArray()) { throw HttpBadRequestException(); } + m_eventBusDtos.clear(); + for (const auto &entry : document.GetArray()) { + if (!entry.IsObject()) { continue; } + EventBusDto eventBusDto; + if (eventBusDto.deserialize(entry)) { m_eventBusDtos.push_back(std::move(eventBusDto)); } + } +} +}// namespace lintel diff --git a/src/features/base/events/EventBus.cpp b/src/features/base/events/EventBus.cpp index 223a2901..728eca69 100644 --- a/src/features/base/events/EventBus.cpp +++ b/src/features/base/events/EventBus.cpp @@ -4,6 +4,7 @@ #include "lintel/core/shm/ShmSegmentAccessor.h" #include "lintel/features/base/events/TopicMatcher.h" +#include #include #include #include @@ -147,21 +148,19 @@ EventBus::ShmRing::~ShmRing() EventBus::ShmRing::Slot *EventBus::ShmRing::buffer() { // The node buffer lives in the shared segment, not inside this object (see the ShmRing - // documentation), and is addressed by the byte offset m_bufferOffset. The addition runs in the - // integer domain instead of as `reinterpret_cast(this) + m_bufferOffset`: that byte - // pointer arithmetic is exactly the intent, yet CodeQL reports it as cpp/suspicious-pointer-scaling - // (a ShmRing * scaled by bytes). The aligned buffer is not necessarily a multiple of sizeof(Slot), - // so element arithmetic is not an option either. - const std::intptr_t bufferAddress = reinterpret_cast(this) + m_bufferOffset; - // NOLINTNEXTLINE(performance-no-int-to-ptr) — address in this process' mapping, restored verbatim. - return reinterpret_cast(bufferAddress); + // documentation), and is addressed by the byte offset m_bufferOffset. The aligned buffer is not + // necessarily a multiple of sizeof(Slot), so element arithmetic is not an option; the address is + // therefore materialised in the integer domain. std::bit_cast restores it: a plain + // reinterpret_cast from the integer would be reported as performance-no-int-to-ptr, byte pointer + // arithmetic as cpp/suspicious-pointer-scaling. + const std::uintptr_t address = reinterpret_cast(this) + static_cast(m_bufferOffset); + return std::bit_cast(address); } const EventBus::ShmRing::Slot *EventBus::ShmRing::buffer() const { - const std::intptr_t bufferAddress = reinterpret_cast(this) + m_bufferOffset; - // NOLINTNEXTLINE(performance-no-int-to-ptr) — see the non-const overload. - return reinterpret_cast(bufferAddress); + const std::uintptr_t address = reinterpret_cast(this) + static_cast(m_bufferOffset); + return std::bit_cast(address); } bool EventBus::ShmRing::push(const Event &event) diff --git a/src/features/base/msg/MessageQueueShmCore.cpp b/src/features/base/msg/MessageQueueShmCore.cpp index 3d00ab99..8f9ac544 100644 --- a/src/features/base/msg/MessageQueueShmCore.cpp +++ b/src/features/base/msg/MessageQueueShmCore.cpp @@ -2,6 +2,7 @@ #include #include +#include #include #include #include @@ -31,6 +32,16 @@ namespace mqshm_detail { constexpr std::size_t kSlotHeaderSize = 16; /** Granularity of the wait used by the blocking operations. */ constexpr std::chrono::milliseconds kWaitSlice{ 1 }; + /** Prefix of the segment object name that holds a queue's header. */ + constexpr const char *kHeaderNamePrefix = "lintel_mq_"; + /** Upper bound for that object name, including the terminating NUL. */ + constexpr std::size_t kHeaderNameSize = 32; + /** Marks an initialised header; "LMQSHM01" in ASCII. */ + constexpr std::uint64_t kQueueMagic = 0x4c4d5153484d3031ull; + /** Magic value while one process is constructing the ring behind a fresh header. */ + constexpr std::uint64_t kQueueMagicInitialising = 1; + /** Upper bound for waiting until another process finished that construction. */ + constexpr std::chrono::milliseconds kInitWaitTimeout{ 5000 }; [[nodiscard]] std::size_t alignUp(std::size_t value, std::size_t alignment) { @@ -215,6 +226,9 @@ namespace mqshm_detail { [[nodiscard]] std::size_t slotSize() const { return m_slotSize; } + /** @return distance in bytes between two consecutive slots */ + [[nodiscard]] std::size_t stride() const { return m_stride; } + [[nodiscard]] static bool isLockFree() { return std::atomic::is_always_lock_free; } private: @@ -241,18 +255,17 @@ namespace mqshm_detail { /** Slot at a logical ring position. * * The slot array lives in the shared segment, not inside this object, and - * is addressed by the byte offset m_bufferOffset: a plain `std::byte *` - * scaled by bytes is the intent, yet CodeQL reports the `reinterpret_cast` - * base as cpp/suspicious-pointer-scaling (a ShmRing * scaled by bytes). The - * stride is not a multiple of sizeof(SlotHeader) either, so element - * arithmetic is not an option. The addition therefore runs in the integer - * domain and the result is restored verbatim. */ + * is addressed by the byte offset m_bufferOffset: the stride is not a + * multiple of sizeof(SlotHeader) either, so element arithmetic is not an + * option and the address is materialised in the integer domain. + * std::bit_cast restores it: a plain reinterpret_cast from the integer + * would be reported as performance-no-int-to-ptr, byte pointer arithmetic + * as cpp/suspicious-pointer-scaling. */ [[nodiscard]] SlotHeader *slotAt(std::uint64_t position) const { - const std::intptr_t address = reinterpret_cast(this) + m_bufferOffset - + static_cast((position % m_capacity) * m_stride); - // NOLINTNEXTLINE(performance-no-int-to-ptr) — relative offset into this process' mapping. - return reinterpret_cast(address); + const std::uintptr_t address = reinterpret_cast(this) + + static_cast(m_bufferOffset) + (position % m_capacity) * m_stride; + return std::bit_cast(address); } /** Payload of a slot lives directly behind its header. `slot + 1` advances @@ -261,15 +274,54 @@ namespace mqshm_detail { [[nodiscard]] static std::byte *payloadOf(SlotHeader *slot) { return reinterpret_cast(slot + 1); } }; + // The block comes from find_or_construct, whose allocation is aligned + // to the segment's memory alignment (max_align_t), and the ring is placed at + // a multiple of its own alignment behind the header - which only works while + // that alignment does not exceed the block alignment. + static_assert(alignof(ShmRing) <= alignof(std::max_align_t), + "the named queue block must be aligned enough for the ring"); + + /** + * Cross-process anchor of one queue inside the segment. + * + * The header is a named object of the segment ("lintel_mq_" + queue name) and + * is what a second process finds first: it describes where the ring sits and + * which layout it has. Everything in here is either a scalar or a byte + * offset, never a pointer - the segment is mapped at a different virtual + * address in every process. + * + * The header is never constructed as a C++ object: the allocator hands out a + * zero initialised block (magic == 0), which is exactly the "not + * initialised" state. A writer claims the header with a compare exchange + * (magic == kQueueMagicInitialising), fills the fields and publishes them by + * storing kQueueMagic with release semantics; a reader waits until the claim + * is resolved and then loads the magic with acquire, after which the fields + * are valid. + */ + struct QueueHeader + { + /** 0 = not initialised, kQueueMagicInitialising = being initialised, + * kQueueMagic = the fields below are valid */ + std::uint64_t magic; + int32_t capacity; + std::size_t slotSize; + std::size_t stride; + /** Byte offset of the ShmRing relative to this header */ + std::intptr_t ringOffset; + }; + /** State shared by every core of one process that opens the same queue. */ struct QueueState { ShmRing *ring = nullptr; - void *ringAllocation = nullptr; + /** Header of this queue in the segment; valid in every process that opened it. */ + QueueHeader *header = nullptr; ShmSegmentAccessor accessor; std::string name; int32_t capacity = 0; std::size_t slotSize = 0; + /** True only for the core that constructed the ring (see the destructor). */ + bool owner = false; std::mutex waitMutex; std::condition_variable waitCondition; @@ -277,18 +329,35 @@ namespace mqshm_detail { ~QueueState() { + // Only the process that constructed the ring may tear it down: the ring + // and its slot buffer belong to the segment, and the process local + // pointer to the slot buffer inside the ring means nothing in any other + // process. An adopting core leaves both alone. + if (!owner) { return; } + if (header != nullptr) { + // Mark the header as uninitialised *before* the ring disappears, so a + // later open re-creates the ring instead of adopting a destroyed one. + // The header block itself stays in the segment: it is a named object, + // and a plain deallocate would leave the name in the segment's index + // pointing at freed memory. + std::atomic_ref(header->magic).store(0u, std::memory_order_release); + } if (ring != nullptr) { ring->~ShmRing(); ring = nullptr; } - if (ringAllocation != nullptr) { - accessor.deallocate(ringAllocation); - ringAllocation = nullptr; - } } void notifyWaiters() { waitCondition.notify_all(); } + /** Wait a slice for progress on the queue. + * + * This is a poll, not a wakeup: the condition variable only knows the + * waiters of this process, and a producer in another process has no way to + * signal it. Every waiter therefore re-checks the ring after kWaitSlice. + * A cross-process wakeup (named semaphore or a futex inside the segment) + * is deliberately left out; it would add a second synchronisation object + * to the queue for a latency gain of at most one slice. */ void waitForProgress(const std::chrono::steady_clock::time_point &deadline, bool bounded) { std::unique_lock lock(waitMutex); @@ -304,34 +373,61 @@ namespace mqshm_detail { }; std::mutex g_registryMutex; + /** Process local cache of the open queues; the segment header is the source of truth. */ std::map> g_registry; - [[nodiscard]] std::shared_ptr - createState(const std::string &name, int32_t msgCount, std::size_t msgSize, ShmSegmentAccessor segment) + /** Name of the segment object that holds a queue's header. + * @param name of the queue + * @return name of the header object inside the segment + * @throws std::invalid_argument if the composed name does not fit into kHeaderNameSize bytes */ + [[nodiscard]] std::string headerNameOf(const std::string &name) { - auto state = std::make_shared(segment); - state->name = name; - state->capacity = msgCount; - state->slotSize = msgSize; - - const std::size_t allocationSize = sizeof(ShmRing) + kCacheLine - 1u; - void *raw = segment.allocate(allocationSize); - if (raw == nullptr) { throw std::bad_alloc(); } - state->ringAllocation = raw; - - void *alignedRaw = raw; - std::size_t alignmentSpace = allocationSize; - if (std::align(alignof(ShmRing), sizeof(ShmRing), alignedRaw, alignmentSpace) == nullptr) { - throw std::bad_alloc(); + std::string headerName = std::string(kHeaderNamePrefix) + name; + if (headerName.size() >= kHeaderNameSize) { + throw std::invalid_argument("MessageQueueShmCore queue name '" + name + + "' is too long: the shared memory object name '" + headerName + "' exceeds " + + std::to_string(kHeaderNameSize - 1) + " characters"); } + return headerName; + } - try { - state->ring = new (alignedRaw) ShmRing(static_cast(msgCount), msgSize, segment); - } catch (...) { - state->ring = nullptr; - throw; + /** @return the ring at a byte offset from the header block */ + [[nodiscard]] ShmRing *ringAt(void *block, std::size_t offset) + { + // The ring follows the header inside the same segment allocation, so its + // address is block + offset. The offset is not a multiple of sizeof(ShmRing), + // element arithmetic is not an option, and the address is therefore + // materialised in the integer domain. std::bit_cast restores it: a plain + // reinterpret_cast from the integer would be reported as + // performance-no-int-to-ptr. + const std::uintptr_t address = reinterpret_cast(block) + static_cast(offset); + return std::bit_cast(address); + } + + /** Wait until another process resolved the initialisation claim of a header. + * + * A header whose magic is kQueueMagicInitialising belongs to a process that + * is constructing the ring right now. The wait is bounded: a process that + * died in that window leaves the claim behind, and taking it over would + * mean touching a ring somebody else may still be building. Failing loudly + * is the safe direction - the segment has to be removed by an operator. + * + * @param name of the queue, for the error message + * @param header the header to watch + * @return the magic the header settled on + * @throws std::runtime_error if it did not settle within kInitWaitTimeout */ + [[nodiscard]] std::uint64_t waitForHeader(const std::string &name, QueueHeader &header) + { + const auto deadline = std::chrono::steady_clock::now() + kInitWaitTimeout; + for (;;) { + const std::uint64_t magic = std::atomic_ref(header.magic).load(std::memory_order_acquire); + if (magic != kQueueMagicInitialising) { return magic; } + if (std::chrono::steady_clock::now() >= deadline) { + throw std::runtime_error("timed out waiting for the initialisation of the shared memory queue '" + name + + "': the process that started to create it did not finish"); + } + std::this_thread::sleep_for(kWaitSlice); } - return state; } [[nodiscard]] std::shared_ptr @@ -339,6 +435,11 @@ namespace mqshm_detail { { if (msgCount <= 0) { throw std::invalid_argument("MessageQueueShmCore message count must be positive"); } if (msgSize == 0) { throw std::invalid_argument("MessageQueueShmCore message size must be positive"); } + if (segment.segmentName().empty()) { + throw std::invalid_argument( + "MessageQueueShmCore needs a named segment accessor so the queue is found across " + "processes; use SharedMemoryService::getSegmentAccessor()"); + } std::lock_guard lock(g_registryMutex); const auto found = g_registry.find(name); @@ -352,7 +453,77 @@ namespace mqshm_detail { g_registry.erase(found); } - auto state = createState(name, msgCount, msgSize, segment); + // The header is the cross-process anchor of the queue: it lives in the + // segment under a name derived from the queue name, so a second process + // finds the very same ring instead of creating one of its own. The ring + // and its slot array follow the header inside the same segment, which is + // why only byte offsets are stored - a pointer would be process local. + const std::size_t ringOffset = alignUp(sizeof(QueueHeader), alignof(ShmRing)); + const std::size_t blockSize = ringOffset + sizeof(ShmRing); + const std::string headerName = headerNameOf(name); + void *block = segment.findOrConstructArray(headerName.c_str(), blockSize); + if (block == nullptr) { + throw std::runtime_error("cannot create the queue header '" + headerName + + "' in the shared memory segment: the allocator does not support named objects"); + } + + auto *header = static_cast(block); + auto state = std::make_shared(segment); + state->name = name; + state->capacity = msgCount; + state->slotSize = msgSize; + state->header = header; + + const std::uint64_t magic = std::atomic_ref(header->magic).load(std::memory_order_acquire); + if (magic == 0u) { + // Fresh header: the allocator zero initialised the block, so no ring has + // been placed behind it yet. Claim the initialisation with a compare + // exchange, so a second process cannot build a second ring at the same + // address, then construct the ring, publish the layout and set the magic + // last - a reader that sees the magic sees the fields too. + std::uint64_t expected = 0u; + if (std::atomic_ref(header->magic) + .compare_exchange_strong(expected, kQueueMagicInitialising, std::memory_order_acq_rel)) { + try { + state->ring = new (ringAt(block, ringOffset)) ShmRing(static_cast(msgCount), msgSize, segment); + } catch (...) { + // Release the claim: a header stuck in "initialising" would make + // every later open wait for a ring that will never be published. + std::atomic_ref(header->magic).store(0u, std::memory_order_release); + throw; + } + header->capacity = msgCount; + header->slotSize = msgSize; + header->stride = state->ring->stride(); + header->ringOffset = static_cast(ringOffset); + std::atomic_ref(header->magic).store(kQueueMagic, std::memory_order_release); + state->owner = true; + g_registry.emplace(name, state); + return state; + } + // Another process claimed the header first: fall through and adopt the + // ring once it publishes it. + } + + const std::uint64_t settled = waitForHeader(name, *header); + if (settled != kQueueMagic) { + throw std::runtime_error("the shared memory queue '" + name + + "' is not initialised: its creating process removed it or the header is corrupt"); + } + const std::intptr_t storedOffset = header->ringOffset; + if (storedOffset < static_cast(sizeof(QueueHeader)) + || static_cast(storedOffset) + sizeof(ShmRing) > blockSize) { + throw std::runtime_error( + "the shared memory queue '" + name + "' has a corrupt header (ring offset out of range)"); + } + if (header->capacity != msgCount || header->slotSize != msgSize) { + throw std::invalid_argument("MessageQueueShmCore already exists with a different layout: " + name); + } + // An existing header: the ring is already constructed by the creating + // process, adopt it instead of constructing a second one. + state->ring = ringAt(block, static_cast(storedOffset)); + state->owner = false; + g_registry.emplace(name, state); return state; } diff --git a/src/features/base/services/AuthService.cpp b/src/features/base/services/AuthService.cpp index e5b8a091..98241a55 100644 --- a/src/features/base/services/AuthService.cpp +++ b/src/features/base/services/AuthService.cpp @@ -44,30 +44,18 @@ AuthService::AuthService(const std::shared_ptr &processName, : PropertyRegistration(processName->getProcessName()), m_userRepository(std::move(userRepository)), m_scheduler(std::move(schedulerService)) { - m_scheduleRate = registerProperty("m_scheduleRate", - std::chrono::seconds(2), - "schedule rate of user tokens checks in seconds", - true, - __FILE__, - __LINE__); + m_scheduleRate = registerProperty( + "m_scheduleRate", std::chrono::seconds(2), "schedule rate of user tokens checks in seconds", true); m_timeoutLogin = registerProperty( - "m_timeoutLogin", std::chrono::minutes(5), "timeout of user login in minutes", true, __FILE__, __LINE__); - m_maxLoginFailures = registerProperty( - "m_maxLoginFailures", 5, "maximum failed login attempts per IP before lockout", true, __FILE__, __LINE__); - m_loginLockout = registerProperty("m_loginLockout", - std::chrono::seconds(60), - "lockout duration after too many failed login attempts", - true, - __FILE__, - __LINE__); + "m_timeoutLogin", std::chrono::minutes(5), "timeout of user login in minutes", true); + m_maxLoginFailures = + registerProperty("m_maxLoginFailures", 5, "maximum failed login attempts per IP before lockout", true); + m_loginLockout = registerProperty( + "m_loginLockout", std::chrono::seconds(60), "lockout duration after too many failed login attempts", true); m_maxLoginAttempts = registerProperty( - "m_maxLoginAttempts", 20, "maximum login attempts per IP within the rate limit window", true, __FILE__, __LINE__); - m_rateLimitWindow = registerProperty("m_rateLimitWindow", - std::chrono::seconds(60), - "sliding window for login attempt rate limiting in seconds", - true, - __FILE__, - __LINE__); + "m_maxLoginAttempts", 20, "maximum login attempts per IP within the rate limit window", true); + m_rateLimitWindow = registerProperty( + "m_rateLimitWindow", std::chrono::seconds(60), "sliding window for login attempt rate limiting in seconds", true); } std::optional AuthService::onLoginOf(const UserLogin &userLogin) diff --git a/src/features/base/services/EventBusService.cpp b/src/features/base/services/EventBusService.cpp index 26dd8912..0d6d8977 100644 --- a/src/features/base/services/EventBusService.cpp +++ b/src/features/base/services/EventBusService.cpp @@ -30,7 +30,7 @@ std::map> EventBusService::init( return map; } -EventBusConfig EventBusService::configOf(const std::shared_ptr &entry) +EventBusConfig EventBusService::configOfEntry(const std::shared_ptr &entry) { EventBusConfig config; config.busCapacity = entry->getBusCapacity(); @@ -54,7 +54,7 @@ EventBus &EventBusService::resolve(const std::shared_ptr &entry) LOG_ERROR("segment '{}' not found for event bus '{}'", entry->getSegment(), entry->getName()); throw std::runtime_error("segment not found for event bus " + entry->getName()); } - const EventBusConfig config = configOf(entry); + const EventBusConfig config = configOfEntry(entry); const std::size_t required = EventBus::requiredSize(config); if (segment->getSize() < required) { LOG_WARN( @@ -73,17 +73,39 @@ EventBus &EventBusService::resolve(const std::shared_ptr &entry) return bus; } -std::shared_ptr EventBusService::of(const std::string &name) +const std::shared_ptr &EventBusService::entryOf(const std::string &name) const { - auto found = m_configurationMap.find(name); + const auto found = m_configurationMap.find(name); if (found == m_configurationMap.end()) { LOG_ERROR("no event bus found for {}", name); throw std::runtime_error("no event bus found for " + name); } - EventBus &bus = resolve(found->second); + return found->second; +} + +std::shared_ptr EventBusService::of(const std::string &name) +{ + EventBus &bus = resolve(entryOf(name)); return std::make_shared(bus); } +std::vector EventBusService::namesOf() const +{ + std::vector names; + names.reserve(m_configurationMap.size()); + for (const auto &entry : m_configurationMap) { names.push_back(entry.first); } + return names; +} + +EventBusConfig EventBusService::configOf(const std::string &name) const { return configOfEntry(entryOf(name)); } + +std::size_t EventBusService::busQueueDepthOf(const std::string &name) { return resolve(entryOf(name)).busQueueDepth(); } + +std::size_t EventBusService::subscriberQueueDepthOf(const std::string &name, const std::string &subscriber) +{ + return resolve(entryOf(name)).subscriberQueueDepth(subscriber); +} + void EventBusService::onInitialize() { for (const auto &entry : m_configurationMap) { diff --git a/src/features/base/services/HistoryService.cpp b/src/features/base/services/HistoryService.cpp index 978368b7..39110f10 100644 --- a/src/features/base/services/HistoryService.cpp +++ b/src/features/base/services/HistoryService.cpp @@ -17,19 +17,15 @@ HistoryService::HistoryService(std::shared_ptr processName, m_historyMessageQueueReceive( m_messageQueueSerivice->of(static_cast(MessageQueues::HISTORY).getMessageQueueName())) { - m_duration = registerProperty( - "m_duration", std::chrono::seconds(1), "Duration of scheduling", true, __FILE__, __LINE__); - m_period = registerProperty( - "m_period", std::chrono::seconds(1), "Period of scheduling", true, __FILE__, __LINE__); - m_forceQueue = registerProperty("m_forceQueue", false, "Force usage of queue", true, __FILE__, __LINE__); - m_commitRate = registerProperty( - "m_commitRate", static_cast(100), "Commit Rate", true, __FILE__, __LINE__); + m_duration = + registerProperty("m_duration", std::chrono::seconds(1), "Duration of scheduling", true); + m_period = registerProperty("m_period", std::chrono::seconds(1), "Period of scheduling", true); + m_forceQueue = registerProperty("m_forceQueue", false, "Force usage of queue", true); + m_commitRate = registerProperty("m_commitRate", static_cast(100), "Commit Rate", true); m_sendTimeout = registerProperty("m_sendTimeout", std::chrono::milliseconds(100), "max time to wait for queue space before dropping a history message", - true, - __FILE__, - __LINE__); + true); } const std::string &HistoryService::getProcessName() const { return m_ownProcessName; } diff --git a/src/features/base/services/ProcessService.cpp b/src/features/base/services/ProcessService.cpp index 7258dbe2..b5e2809a 100644 --- a/src/features/base/services/ProcessService.cpp +++ b/src/features/base/services/ProcessService.cpp @@ -64,13 +64,9 @@ ProcessService::ProcessService(std::shared_ptr processName, m_historyService(std::move(historyService)) { m_monitorWaitTime = registerProperty( - "m_monitorWaitTime", std::chrono::seconds(1), "Wait time after monitor cycle", true, __FILE__, __LINE__); - m_processStopWaitTime = registerProperty("m_processStopWaitTime", - std::chrono::milliseconds(200), - "Wait time for stopping a process", - true, - __FILE__, - __LINE__); + "m_monitorWaitTime", std::chrono::seconds(1), "Wait time after monitor cycle", true); + m_processStopWaitTime = registerProperty( + "m_processStopWaitTime", std::chrono::milliseconds(200), "Wait time for stopping a process", true); // make sure that all variables are initialized for starting the thread m_process = std::make_shared(m_processName->getPath(), m_processName->getArgs()); std::vector> entries = m_configuration->configurationOf(); @@ -561,30 +557,56 @@ void ProcessService::monitorProcessGroups() } } -ProcessService::~ProcessService() +void ProcessService::stopAllProcesses() { - try { - m_running.store(false); - m_condition.notify_all(); - if (m_monitorThread.joinable()) { m_monitorThread.join(); } - std::vector runningProcesses; - { - std::lock_guard locker(m_processesMutex); - for (auto &[id, process] : m_processes) { - if (!process.getChild()->running()) { continue; } - runningProcesses.push_back(process); - } + // stop the monitor first, otherwise an autoRestart child that just exited + // would be spawned again between the stop and the terminate pass + m_running.store(false); + m_condition.notify_all(); + if (m_monitorThread.joinable()) { m_monitorThread.join(); } + std::vector runningProcesses; + { + std::lock_guard locker(m_processesMutex); + for (auto &[id, process] : m_processes) { + if (!process.getChild()->running()) { continue; } + runningProcesses.push_back(process); } - for (auto &item : runningProcesses) { - if (!item.getChild()->running()) { continue; } - stopOf(*item.getProcess()); + } + // one failing process must not keep the others alive, so every call is + // guarded on its own (stopOf() writes a history entry, and the history + // service may already be shut down when this runs last) + for (auto &item : runningProcesses) { + try { + if (item.getChild()->running()) { stopOf(*item.getProcess()); } + } catch (const std::exception &exception) { + LOG_WARN("stop of {}/{} failed: {}", + item.getProcess()->getId(), + item.getProcess()->getPath().filename().string(), + exception.what()); } - for (auto &item : runningProcesses) { - if (!item.getChild()->running()) { continue; } - terminateOf(*item.getProcess()); + } + for (auto &item : runningProcesses) { + try { + if (item.getChild()->running()) { terminateOf(*item.getProcess()); } + } catch (const std::exception &exception) { + LOG_WARN("terminate of {}/{} failed: {}", + item.getProcess()->getId(), + item.getProcess()->getPath().filename().string(), + exception.what()); } - } catch (const std::exception &e) { - static_cast(e); + } +} + +void ProcessService::onShutdown() { stopAllProcesses(); } + +ProcessService::~ProcessService() +{ + // a destructor must not throw: stopAllProcesses() logs and locks, and both + // can fail when the logger or the history service is already gone + try { + stopAllProcesses(); + } catch (const std::exception &exception) { + static_cast(exception); } } diff --git a/src/features/base/services/SchedulerService.cpp b/src/features/base/services/SchedulerService.cpp index 165dace3..3d0adc01 100644 --- a/src/features/base/services/SchedulerService.cpp +++ b/src/features/base/services/SchedulerService.cpp @@ -72,8 +72,7 @@ void SchedulerService::onShutdown() SchedulerService::SchedulerService(const std::shared_ptr &processName) : PropertyRegistration(processName->getProcessName()), m_exit(false) { - numberOfThreads = - registerProperty("numberOfThreads", 40, "Number of Scheduler Threads", false, __FILE__, __LINE__); + numberOfThreads = registerProperty("numberOfThreads", 40, "Number of Scheduler Threads", false); } SchedulerService::~SchedulerService() diff --git a/src/features/base/services/SharedMemoryRepository.cpp b/src/features/base/services/SharedMemoryRepository.cpp index bba13b5f..8dc72370 100644 --- a/src/features/base/services/SharedMemoryRepository.cpp +++ b/src/features/base/services/SharedMemoryRepository.cpp @@ -6,6 +6,7 @@ #include "lintel/features/base/services/SharedMemoryService.h" #include "lintel/features/base/services/ShmConstructors.h" +#include #include #include #include @@ -86,15 +87,16 @@ namespace { [[nodiscard]] void *payload() const { // The payload offset is stored in the record inside the segment (its - // address differs per mapping), so it has to be materialised here. A - // std::byte * base scaled by bytes is the intent, yet CodeQL reports the - // reinterpret_cast base as cpp/suspicious-pointer-scaling (a ShmCellRecord - // * scaled by bytes); the payload alignment is not a multiple of - // sizeof(ShmCellRecord), so element arithmetic is not an option. The - // addition therefore runs in the integer domain and is restored verbatim. - const std::intptr_t address = reinterpret_cast(this) + m_payloadOffset; - // NOLINTNEXTLINE(performance-no-int-to-ptr) — relative offset into this process' mapping. - return reinterpret_cast(address); + // address differs per mapping), so it has to be materialised here. The + // payload alignment is not a multiple of sizeof(ShmCellRecord), so + // element arithmetic is not an option; the address is therefore + // materialised in the integer domain. std::bit_cast restores it: a plain + // reinterpret_cast from the integer would be reported as + // performance-no-int-to-ptr, byte pointer arithmetic as + // cpp/suspicious-pointer-scaling. + const std::uintptr_t address = + reinterpret_cast(this) + static_cast(m_payloadOffset); + return std::bit_cast(address); } [[nodiscard]] std::uint64_t magic() const { return m_magic; } diff --git a/src/features/base/services/SharedMemoryRepositoryRegistry.cpp b/src/features/base/services/SharedMemoryRepositoryRegistry.cpp index 05b479ce..eb9ba19b 100644 --- a/src/features/base/services/SharedMemoryRepositoryRegistry.cpp +++ b/src/features/base/services/SharedMemoryRepositoryRegistry.cpp @@ -1,4 +1,4 @@ -#include "lintel/features/base/services/SharedMemoryRepositoryRegistry.h" +#include "lintel/core/shm/SharedMemoryRepositoryRegistry.h" namespace lintel { diff --git a/src/features/base/services/SharedMemoryService.cpp b/src/features/base/services/SharedMemoryService.cpp index d4f20dd4..be3f92d8 100644 --- a/src/features/base/services/SharedMemoryService.cpp +++ b/src/features/base/services/SharedMemoryService.cpp @@ -5,28 +5,67 @@ #include #include #include +#include #include namespace lintel { +namespace { + /** Layout marker of the objects stored inside a segment. + * + * Bumped whenever the in-segment layout of a lintel object changes (the + * ShmSegmentAccessor grew from one pointer to a named 40 byte value in 0.2, + * for example). A segment written by an older build is recreated instead of + * being reinterpreted with the new layout, which would corrupt it silently. + */ + constexpr std::uint64_t kSegmentLayoutVersion = 2; + constexpr char kSegmentLayoutName[] = "lintel_segment_layout"; + + /** @param file mapped segment + * @return the layout version stored in the segment; empty for a segment that + * was never used, 0 for one written before the marker existed */ + std::optional layoutVersionOf(boost::interprocess::managed_mapped_file &file) + { + const auto found = file.find(kSegmentLayoutName); + if (found.first != nullptr) { return *found.first; } + // No marker: either a segment that was never used, or one written before + // the marker existed (which always has named objects by then). + if (file.get_num_named_objects() == 0) { return std::nullopt; } + return 0; + } + + /** @return true when the operator allowed a segment with an older layout to be discarded + * + * Recreating a segment throws its contents away, and only the operator can + * decide whether that is acceptable. The default therefore refuses and names + * the file, so data loss never happens silently. + */ + bool recreateAllowed() + { + const char *value = std::getenv("LINTEL_SHM_RECREATE"); + if (value == nullptr) { return false; } + const std::string_view flag(value); + return flag == "1" || flag == "true" || flag == "TRUE" || flag == "yes" || flag == "YES"; + } + + /** Write the layout marker into a freshly created segment. */ + void stampLayout(boost::interprocess::managed_mapped_file &file) + { + if (file.find(kSegmentLayoutName).first != nullptr) { return; } + file.construct(kSegmentLayoutName)(kSegmentLayoutVersion); + } +}// namespace + SharedMemoryService::SharedMemoryService(const std::shared_ptr &sharedMemorySegmentManager, std::shared_ptr schedulerService, const std::shared_ptr &processName) : PropertyRegistration(processName->getProcessName()), m_segments(create(sharedMemorySegmentManager->allOf())), m_schedulerService(std::move(schedulerService)) { - m_scheduleRate = registerProperty("m_scheduleRate", - std::chrono::seconds(2), - "schedule rate of user tokens checks in seconds", - true, - __FILE__, - __LINE__); - m_autoExtend = registerProperty("m_autoExtend", true, "auto extend shared memory", true, __FILE__, __LINE__); - m_autoExtendEpsilon = registerProperty("m_autoExtendEpsilon", - static_cast(1000), - "epsilon when auto extend gets triggered", - true, - __FILE__, - __LINE__); + m_scheduleRate = registerProperty( + "m_scheduleRate", std::chrono::seconds(2), "schedule rate of user tokens checks in seconds", true); + m_autoExtend = registerProperty("m_autoExtend", true, "auto extend shared memory", true); + m_autoExtendEpsilon = registerProperty( + "m_autoExtendEpsilon", static_cast(1000), "epsilon when auto extend gets triggered", true); } void SharedMemoryService::growOf(const std::shared_ptr &segment, std::size_t grow) @@ -135,6 +174,50 @@ std::map> SharedMemorySe const std::string filePath = segment->getPath().string(); handle->m_managedMappedFile = std::make_shared( boost::interprocess::open_or_create, filePath.c_str(), segment->getSize()); + const std::optional storedVersion = layoutVersionOf(*handle->m_managedMappedFile); + if (storedVersion.has_value() && *storedVersion != kSegmentLayoutVersion) { + // The objects in a segment have no layout version of their own, so + // their bytes cannot be reinterpreted safely - but only the operator + // knows whether the contents can be recomputed. So the operator + // decides: an explicit migrator, an explicit policy, or + // LINTEL_SHM_RECREATE=1. Without either of them nothing is discarded. + // See docs/shm-migration-strategy.md. + bool mayDiscard = false; + if (shm::SegmentLayoutPolicy::hasMigrator()) { + LOG_WARN( + "shared memory segment '{}' was written with layout {} (current {}) - asking the registered " + "migrator for '{}'", + segment->getName(), + *storedVersion, + kSegmentLayoutVersion, + filePath); + mayDiscard = shm::SegmentLayoutPolicy::migrate(segment->getPath(), *storedVersion, kSegmentLayoutVersion); + if (!mayDiscard) { + throw std::runtime_error("the migrator refused to hand over shared memory segment '" + segment->getName() + + "' at '" + filePath + "'"); + } + } else if (shm::SegmentLayoutPolicy::policy() == shm::LayoutMismatchPolicy::Recreate || recreateAllowed()) { + mayDiscard = true; + } + if (!mayDiscard) { + throw std::runtime_error("shared memory segment '" + segment->getName() + "' at '" + filePath + + "' was written with an older object layout and cannot be opened. Delete the " + "file to start with an empty segment, register a migrator " + "(lintel::shm::SegmentLayoutPolicy) or set LINTEL_SHM_RECREATE=1 to have it " + "discarded automatically."); + } + LOG_WARN( + "shared memory segment '{}' was written with an older object layout - recreating '{}' because " + "LINTEL_SHM_RECREATE is set (its contents are discarded, not migrated)", + segment->getName(), + filePath); + handle->m_managedMappedFile.reset(); + std::error_code removeError; + std::filesystem::remove(segment->getPath(), removeError); + handle->m_managedMappedFile = std::make_shared( + boost::interprocess::open_or_create, filePath.c_str(), segment->getSize()); + } + stampLayout(*handle->m_managedMappedFile); // Holder record of the remap semaphore, plus the automatic recovery: a // previous run that crashed while holding the lock is visible here as a // stuck semaphore whose recorded holder is gone. The record is created @@ -203,7 +286,13 @@ ShmSegmentAccessor SharedMemoryService::getSegmentAccessor(const std::shared_ptr handle.m_segmentAllocator = std::make_shared(*handle.m_managedMappedFile->get_segment_manager()); } - return ShmSegmentAccessor(*handle.m_segmentAllocator); + // Bind the name to *this* process' allocator and hand out an accessor that + // carries the name: a copy of it that is stored inside the segment resolves + // to the allocator of whichever process reads it, instead of pointing at the + // heap of the process that created the object (see ShmSegmentAccessor). + ShmSegmentAccessor accessor(segment->getName(), *handle.m_segmentAllocator); + ShmSegmentAccessor::bind(segment->getName(), *handle.m_segmentAllocator); + return accessor; } void SharedMemoryService::onInitialize() diff --git a/src/features/cli/components/AdminCliComponent.cpp b/src/features/cli/components/AdminCliComponent.cpp new file mode 100644 index 00000000..9042f1ff --- /dev/null +++ b/src/features/cli/components/AdminCliComponent.cpp @@ -0,0 +1,102 @@ +#include "lintel/features/cli/components/AdminCliComponent.h" + +#include +#include +#include +#include +#include + +namespace lintel { +namespace { + /** @param response the answer of the controller + * @return the number of resources the reload reported as failed */ + std::size_t failedCountOf(const AdminTlsReloadResponse &response) + { + std::size_t failed = 0; + for (const auto &result : response.results) { + if (!result.reloaded) { ++failed; } + } + return failed; + } +}// namespace + +AdminCliComponent::AdminCliComponent(std::shared_ptr adminApi) + : CommandLineComponent(m_name, m_alias), m_adminApi(std::move(adminApi)) +{ + m_commandParser.addCommand(Command("reload_tls", "Reload the TLS certificate of the running server"), ReloadTls); + m_commandParser.addCommand( + Command("log_level", "Change the log level of the running process") + .addArgument({ "--level", "-l" }, &m_level, "Level: trace, debug, info, warn, err (or error), critical or off"), + LogLevel); +} + +const std::string &AdminCliComponent::requiredOf(const std::optional &value, const char *flag) +{ + if (!value.has_value()) { throw std::runtime_error(std::string(flag) + " is required"); } + return value.value(); +} + +void AdminCliComponent::onCommand(const UserDto & /*userDto*/, + const std::string &command, + const std::vector ¶meters) +{ + try { + const Commands cmd = m_commandParser.parse(command, parameters); + if (cmd == ReloadTls) { + const std::optional response = m_adminApi->reloadTlsOf(); + if (!response.has_value()) { + std::cout << "ERROR: TLS certificate could not be reloaded\n"; + return; + } + printReloadResults(response.value()); + } else if (cmd == LogLevel) { + const std::string &level = requiredOf(m_level, "--level"); + const std::optional response = m_adminApi->logLevelOf(level); + if (!response.has_value()) { + std::cout << "ERROR: log level " << level << " could not be changed\n"; + return; + } + if (response->status != "changed") { + std::cout << "ERROR: log level " << (response->level.empty() ? level : response->level) << " was rejected (" + << response->status << ")\n"; + return; + } + std::cout << "Log level changed to " << (response->level.empty() ? level : response->level) << "\n"; + } + } catch (const std::exception &exception) { + std::cerr << "ERROR: " << exception.what() << "\n"; + } +} + +void AdminCliComponent::printReloadResults(const AdminTlsReloadResponse &response) +{ + for (const auto &result : response.results) { std::cout << result.name << ": " << result.detail << "\n"; } + if (response.status == "reloaded") { + std::cout << "Reloaded the TLS certificate of " << response.results.size() << " resource(s)\n"; + return; + } + if (response.status == "nothing_to_reload") { + // no TLS server registered a handler, so the process serves clear text or + // is not the process that owns the server + std::cout << "ERROR: no TLS server is registered, nothing to reload\n"; + return; + } + std::cout << "ERROR: TLS certificate could not be reloaded on " << failedCountOf(response) << " of " + << response.results.size() << " resource(s)\n"; +} + +void AdminCliComponent::onHelp() { m_commandParser.printHelp(getName(), getAlias(), m_description); } + +bool AdminCliComponent::onMenu(const std::string & /*component*/) { return false; } + +void AdminCliComponent::onShowMenu() { std::cout << "No submenu available!\n"; } + +bool AdminCliComponent::onExit() { return true; } + +void AdminCliComponent::printCommandList(std::set menuAlias) +{ + m_commandParser.printCommandList(menuAlias); +} + +std::vector AdminCliComponent::allCommandsOf() { return m_commandParser.allCommandsOf(); } +}// namespace lintel diff --git a/src/features/cli/components/EventBusCliComponent.cpp b/src/features/cli/components/EventBusCliComponent.cpp new file mode 100644 index 00000000..3e969247 --- /dev/null +++ b/src/features/cli/components/EventBusCliComponent.cpp @@ -0,0 +1,219 @@ +#include "lintel/features/cli/components/EventBusCliComponent.h" + +#include "lintel/core/utils/TableBuilder.h" + +#include +#include +#include +#include +#include + +namespace lintel { +namespace { + /** @param policy the drop policy + * @return the name used on the wire */ + const char *dropPolicyNameOf(DropPolicy policy) + { + switch (policy) { + case DropPolicy::Fail: + return "Fail"; + case DropPolicy::DropOldest: + return "DropOldest"; + case DropPolicy::DropNewest: + return "DropNewest"; + case DropPolicy::Block: + return "Block"; + } + return "Fail"; + } +}// namespace + +EventBusCliComponent::EventBusCliComponent(std::shared_ptr eventBusApi) + : CommandLineComponent(m_name, m_alias), m_eventBusApi(std::move(eventBusApi)) +{ + m_commandParser.addCommand(Command("show_event_buses", "Show all configured event buses"), ShowEventBuses); + m_commandParser.addCommand(Command("show_event_bus", "Show one configured event bus") + .addArgument({ "--bus", "-b" }, &m_bus, "Name of the event bus"), + ShowEventBus); + m_commandParser.addCommand( + Command("publish_event", "Publish an event on an event bus") + .addArgument({ "--bus", "-b" }, &m_bus, "Name of the event bus") + .addArgument({ "--topic", "-t" }, &m_topic, "Topic of the event") + .addArgument({ "--type", "-ty" }, &m_type, "Type of the event") + .addArgument({ "--payload", "-p" }, &m_payload, "Payload of the event (at most 256 bytes)", true) + .addArgument({ "--priority", "-pr" }, &m_priority, "Priority: Critical, High, Normal or Low", true) + .addArgument({ "--to-subscribers", "-ts" }, + &m_toSubscribers, + "Broadcast to the subscribers instead of queueing on the bus", + true), + PublishEvent); + m_commandParser.addCommand(Command("subscribe_event", "Subscribe a named subscriber to a topic") + .addArgument({ "--bus", "-b" }, &m_bus, "Name of the event bus") + .addArgument({ "--topic", "-t" }, &m_topic, "Topic or wildcard pattern") + .addArgument({ "--subscriber", "-s" }, &m_subscriber, "Name of the subscriber"), + SubscribeEvent); + m_commandParser.addCommand(Command("unsubscribe_event", "Remove a subscription from a topic") + .addArgument({ "--bus", "-b" }, &m_bus, "Name of the event bus") + .addArgument({ "--topic", "-t" }, &m_topic, "Topic of the subscription") + .addArgument({ "--subscriber", "-s" }, &m_subscriber, "Name of the subscriber"), + UnsubscribeEvent); + m_commandParser.addCommand(Command("event_queue_depth", "Show the queue depth of one subscriber") + .addArgument({ "--bus", "-b" }, &m_bus, "Name of the event bus") + .addArgument({ "--subscriber", "-s" }, &m_subscriber, "Name of the subscriber"), + EventQueueDepth); + m_commandParser.addCommand(Command("reset_event_bus", "Drop every queued event of an event bus") + .addArgument({ "--bus", "-b" }, &m_bus, "Name of the event bus"), + ResetEventBus); +} + +const std::string &EventBusCliComponent::requiredOf(const std::optional &value, const char *flag) +{ + if (!value.has_value()) { throw std::runtime_error(std::string(flag) + " is required"); } + return value.value(); +} + +void EventBusCliComponent::onCommand(const UserDto & /*userDto*/, + const std::string &command, + const std::vector ¶meters) +{ + try { + const Commands cmd = m_commandParser.parse(command, parameters); + if (cmd == ShowEventBuses) { + printEventBuses(m_eventBusApi->allOf()); + } else if (cmd == ShowEventBus) { + const std::string &bus = requiredOf(m_bus, "--bus"); + const std::optional eventBus = m_eventBusApi->of(bus); + if (!eventBus.has_value()) { + std::cout << "ERROR: event bus " << bus << " not found\n"; + return; + } + printEventBuses({ eventBus.value() }); + } else if (cmd == PublishEvent) { + const std::string &bus = requiredOf(m_bus, "--bus"); + const std::string &topic = requiredOf(m_topic, "--topic"); + const std::string &type = requiredOf(m_type, "--type"); + const std::string payload = m_payload.has_value() ? m_payload.value() : std::string(); + const std::string priority = m_priority.has_value() ? m_priority.value() : std::string("Normal"); + EventPublishRequest request; + request.bus = bus; + request.topic = topic; + request.type = type; + request.payload = payload; + request.priority = priority; + request.toSubscribers = m_toSubscribers; + const std::optional published = m_eventBusApi->publishOf(request); + if (!published.has_value()) { + std::cout << "ERROR: event " << topic << "/" << type << " could not be published on event bus " << bus << "\n"; + return; + } + if (!published.value()) { + std::cout << "ERROR: event " << topic << "/" << type << " was dropped, every matching queue is full\n"; + return; + } + std::cout << "Published event " << topic << "/" << type << " on event bus " << bus << "\n"; + } else if (cmd == SubscribeEvent) { + const std::string &bus = requiredOf(m_bus, "--bus"); + const std::string &topic = requiredOf(m_topic, "--topic"); + const std::string &subscriber = requiredOf(m_subscriber, "--subscriber"); + const std::optional subscribed = m_eventBusApi->subscribeOf(EventSubscription{ bus, topic, subscriber }); + if (!subscribed.has_value() || !subscribed.value()) { + std::cout << "ERROR: subscriber " << subscriber << " could not subscribe to " << topic << " on event bus " + << bus << "\n"; + return; + } + std::cout << "Subscribed " << subscriber << " to " << topic << " on event bus " << bus << "\n"; + } else if (cmd == UnsubscribeEvent) { + const std::string &bus = requiredOf(m_bus, "--bus"); + const std::string &topic = requiredOf(m_topic, "--topic"); + const std::string &subscriber = requiredOf(m_subscriber, "--subscriber"); + const std::optional unsubscribed = + m_eventBusApi->unsubscribeOf(EventSubscription{ bus, topic, subscriber }); + if (!unsubscribed.has_value() || !unsubscribed.value()) { + std::cout << "ERROR: subscriber " << subscriber << " is not subscribed to " << topic << " on event bus " << bus + << "\n"; + return; + } + std::cout << "Unsubscribed " << subscriber << " from " << topic << " on event bus " << bus << "\n"; + } else if (cmd == EventQueueDepth) { + const std::string &bus = requiredOf(m_bus, "--bus"); + const std::string &subscriber = requiredOf(m_subscriber, "--subscriber"); + const std::optional depth = m_eventBusApi->depthOf(bus, subscriber); + if (!depth.has_value()) { + std::cout << "ERROR: queue depth of subscriber " << subscriber << " on event bus " << bus + << " could not be read\n"; + return; + } + std::cout << "Depth of subscriber " << subscriber << " on event bus " << bus << ": " << depth.value() << "\n"; + } else if (cmd == ResetEventBus) { + const std::string &bus = requiredOf(m_bus, "--bus"); + const std::optional reset = m_eventBusApi->resetOf(bus); + if (!reset.has_value() || !reset.value()) { + std::cout << "ERROR: event bus " << bus << " could not be reset\n"; + return; + } + std::cout << "Reset event bus " << bus << "\n"; + } + } catch (const std::exception &exception) { + std::cerr << "ERROR: " << exception.what() << "\n"; + } +} + +void EventBusCliComponent::printEventBuses(const std::vector &eventBusDtos) +{ + const auto configNumberOrDash = [](const std::optional &config, auto getter) { + return config.has_value() ? std::to_string(getter(config.value())) : std::string("-"); + }; + const auto runtimeNumberOrDash = [](const EventBusDto &eventBusDto, auto getter) { + return eventBusDto.isAvailable() ? std::to_string(getter(eventBusDto)) : std::string("-"); + }; + TableBuilder<14> builder; + builder.add({ "No.", + "Name", + "Available", + "BusCapacity", + "SubscriberCapacity", + "MaxTopics", + "MaxSubscribers", + "PubPolicy", + "Sticky", + "Topics", + "Subscribers", + "Dropped", + "Sequence", + "BusDepth" }); + std::size_t iter = 0; + for (const auto &eventBusDto : eventBusDtos) { + const std::optional &config = eventBusDto.getConfig(); + builder.add({ std::to_string(++iter), + eventBusDto.getName(), + eventBusDto.isAvailable() ? "yes" : "no", + configNumberOrDash(config, [](const EventBusConfig &entry) { return entry.busCapacity; }), + configNumberOrDash(config, [](const EventBusConfig &entry) { return entry.subscriberCapacity; }), + configNumberOrDash(config, [](const EventBusConfig &entry) { return entry.maxTopics; }), + configNumberOrDash(config, [](const EventBusConfig &entry) { return entry.maxSubscribers; }), + config.has_value() ? dropPolicyNameOf(config.value().pubPolicy) : std::string("-"), + config.has_value() ? (config.value().sticky ? "yes" : "no") : std::string("-"), + runtimeNumberOrDash(eventBusDto, [](const EventBusDto &entry) { return entry.getTopics(); }), + runtimeNumberOrDash(eventBusDto, [](const EventBusDto &entry) { return entry.getSubscribers(); }), + runtimeNumberOrDash(eventBusDto, [](const EventBusDto &entry) { return entry.getDropped(); }), + runtimeNumberOrDash(eventBusDto, [](const EventBusDto &entry) { return entry.getSequence(); }), + runtimeNumberOrDash(eventBusDto, [](const EventBusDto &entry) { return entry.getBusDepth(); }) }); + } + std::cout << builder.build() << "\n"; +} + +void EventBusCliComponent::onHelp() { m_commandParser.printHelp(getName(), getAlias(), m_description); } + +bool EventBusCliComponent::onMenu(const std::string & /*component*/) { return false; } + +void EventBusCliComponent::onShowMenu() { std::cout << "No submenu available!\n"; } + +bool EventBusCliComponent::onExit() { return true; } + +void EventBusCliComponent::printCommandList(std::set menuAlias) +{ + m_commandParser.printCommandList(menuAlias); +} + +std::vector EventBusCliComponent::allCommandsOf() { return m_commandParser.allCommandsOf(); } +}// namespace lintel diff --git a/src/features/cli/components/HistoryCliComponent.cpp b/src/features/cli/components/HistoryCliComponent.cpp index 512d0fd2..b1a5a0c4 100644 --- a/src/features/cli/components/HistoryCliComponent.cpp +++ b/src/features/cli/components/HistoryCliComponent.cpp @@ -6,12 +6,12 @@ namespace lintel { HistoryCliComponent::HistoryCliComponent(std::shared_ptr historyApi) - : CommandLineComponent(n_name, m_alias), m_historyApi(std::move(historyApi)) + : CommandLineComponent(m_name, m_alias), m_historyApi(std::move(historyApi)) { m_commandParser.addCommand( Command("show_histories", "Shows all histories for set parameters") .addArgument({ "--process-name", "-p" }, &m_processName, "Process Name of history entries") - .addArgument({ "--service-name", "-s" }, &m_serviceName, "Service Name of history entries)") + .addArgument({ "--service-name", "-s" }, &m_serviceName, "Service Name of history entries") .addArgument({ "--label", "-l" }, &m_label, "Label Name of history entries"), ShowHistories); } diff --git a/src/features/cli/components/MessageQueueCliComponent.cpp b/src/features/cli/components/MessageQueueCliComponent.cpp index 9400fd51..276dc9a8 100644 --- a/src/features/cli/components/MessageQueueCliComponent.cpp +++ b/src/features/cli/components/MessageQueueCliComponent.cpp @@ -30,6 +30,9 @@ MessageQueueCliComponent::MessageQueueCliComponent(std::shared_ptr recovered = m_messageQueueApi->recoverOf(); + if (!recovered.has_value()) { + std::cout << "ERROR: orphaned message queue entries could not be reclaimed\n"; + return; + } + std::cout << "Recovered " << recovered.value() << " orphaned message queue entries\n"; } } catch (const std::exception &exception) { std::cerr << "ERROR: " << exception.what() << "\n"; diff --git a/src/features/cli/components/PropertyCliComponent.cpp b/src/features/cli/components/PropertyCliComponent.cpp index e7ccd952..4e4e2c05 100644 --- a/src/features/cli/components/PropertyCliComponent.cpp +++ b/src/features/cli/components/PropertyCliComponent.cpp @@ -15,15 +15,15 @@ PropertyCliComponent::PropertyCliComponent(std::shared_ptr property ShowProperties); // Command: show property m_commandParser.addCommand(Command("show_property", "Show given property") - .addArgument({ "--process_name", "-p" }, &m_processName, "Process Name of property!") - .addArgument({ "--class_name", "-c" }, &m_className, "Class Name of property!") + .addArgument({ "--process-name", "-p" }, &m_processName, "Process Name of property!") + .addArgument({ "--class-name", "-c" }, &m_className, "Class Name of property!") .addArgument({ "--instance-name", "-i" }, &m_instanceName, "Instance Name of property") .addArgument({ "--name", "-n" }, &m_propertyName, "Name of property!"), ShowProperty); // Command: update property m_commandParser.addCommand(Command("update_property", "Update given property") - .addArgument({ "--process_name", "-p" }, &m_processName, "Process Name of property!") - .addArgument({ "--class_name", "-c" }, &m_className, "Class Name of property!") + .addArgument({ "--process-name", "-p" }, &m_processName, "Process Name of property!") + .addArgument({ "--class-name", "-c" }, &m_className, "Class Name of property!") .addArgument({ "--instance-name", "-i" }, &m_instanceName, "Instance Name of property") .addArgument({ "--value", "-v" }, &m_value, "Value of property!") .addArgument({ "--name", "-n" }, &m_propertyName, "Name of property!"), diff --git a/src/features/cli/components/SharedMemoryCliComponent.cpp b/src/features/cli/components/SharedMemoryCliComponent.cpp index 92106a79..72ace286 100644 --- a/src/features/cli/components/SharedMemoryCliComponent.cpp +++ b/src/features/cli/components/SharedMemoryCliComponent.cpp @@ -20,16 +20,24 @@ SharedMemoryCliComponent::SharedMemoryCliComponent(std::shared_ptrclear(); break; } + case ShowStats: { + if (!m_segmentName.has_value()) { + std::cout << "ERROR: please set segment name\n"; + return; + } + const std::optional stats = m_sharedMemoryApi->statsOf(m_segmentName.value()); + if (!stats.has_value()) { + std::cout << "ERROR: no statistics for segment " << m_segmentName.value() << "\n"; + m_segmentName->clear(); + return; + } + printStats(stats.value()); + m_segmentName->clear(); + break; + } + case Cleanup: { + const std::optional summary = m_sharedMemoryApi->cleanupOf(); + if (!summary.has_value()) { + std::cout << "ERROR: cleanup was rejected\n"; + return; + } + std::cout << "Cleanup scanned " << summary->scanned << " entries: " << summary->orphans << " orphans, " + << summary->removed << " removed, " << summary->failed << " failures\n"; + break; + } + case RecoverSegments: { + const std::optional recovered = m_sharedMemoryApi->recoverOf(); + if (!recovered.has_value()) { + std::cout << "ERROR: recover was rejected\n"; + return; + } + std::cout << "Recovered " << recovered.value() << " orphaned remap locks\n"; + break; + } case ShrinkSegment: { if (!m_segmentName.has_value()) { std::cout << "ERROR: please set segment name\n"; @@ -98,7 +140,7 @@ void SharedMemoryCliComponent::handleExportRepository() } std::string result = m_sharedMemoryApi->repositoryOf(m_uuid.value()); if (m_file.has_value()) { - std::string file = StringUtils::replaceAll(m_file.value(), "~", lintel::homeDirectory()); + const std::string file = lintel::expandHomeDirectory(m_file.value()); std::filesystem::path path(file); bool parentPathExits = std::filesystem::exists(path.parent_path()); if (!parentPathExits) { std::filesystem::create_directories(path.parent_path()); } @@ -116,7 +158,6 @@ void SharedMemoryCliComponent::handleExportRepository() if (m_file.has_value()) { m_file->clear(); } if (m_repositoryName.has_value()) { m_repositoryName->clear(); } if (m_segmentName.has_value()) { m_segmentName->clear(); } - if (m_type.has_value()) { m_type->clear(); } } void SharedMemoryCliComponent::printSegments(const std::vector &segments) @@ -180,5 +221,20 @@ void SharedMemoryCliComponent::printRepositories(const std::vector builder; + builder.add({ "Field", "Value" }); + builder.add({ "Name", stats.name }); + builder.add({ "AllocatedBytes", std::to_string(stats.allocatedBytes) }); + builder.add({ "CapacityBytes", std::to_string(stats.capacityBytes) }); + builder.add({ "FreeBytes", std::to_string(stats.freeBytes) }); + builder.add({ "GrowCount", std::to_string(stats.growCount) }); + builder.add({ "ShrinkCount", std::to_string(stats.shrinkCount) }); + builder.add({ "RecoveryCount", std::to_string(stats.recoveryCount) }); + builder.add({ "Generation", std::to_string(stats.generation) }); + std::cout << builder.build() << "\n"; +} + std::vector SharedMemoryCliComponent::allCommandsOf() { return m_commandParser.allCommandsOf(); } }// namespace lintel diff --git a/src/features/cli/components/UserManagementCliComponent.cpp b/src/features/cli/components/UserManagementCliComponent.cpp index a19becd4..c97d85fd 100644 --- a/src/features/cli/components/UserManagementCliComponent.cpp +++ b/src/features/cli/components/UserManagementCliComponent.cpp @@ -60,7 +60,7 @@ UserManagementCliComponent::UserManagementCliComponent(std::shared_ptr * Dispatches an entered CLI command to the UserApi. * * Flow: the parser splits the input into a command and its parameters, then a - * switch over the nine commands executes the matching UserApi operation and + * switch over the eight commands executes the matching UserApi operation and * prints the result. All filters (group name, virtual group) are taken from the * previously set session context, not from the parameters - the CLI session * carries them. diff --git a/src/features/cli/services/CommandLineHistoryService.cpp b/src/features/cli/services/CommandLineHistoryService.cpp index e20b9131..1219df66 100644 --- a/src/features/cli/services/CommandLineHistoryService.cpp +++ b/src/features/cli/services/CommandLineHistoryService.cpp @@ -19,11 +19,9 @@ CommandLineHistoryService::CommandLineHistoryService(const std::shared_ptr("m_maxCommandHistoryEntries", static_cast(1000), "Maximal count of history command entries per local save file", - true, - __FILE__, - __LINE__); + true); m_historyFileName = registerProperty( - "m_historyFileName", std::string(".history"), "Filename of local history file", false, __FILE__, __LINE__); + "m_historyFileName", std::string(".history"), "Filename of local history file", false); } std::optional CommandLineHistoryService::nextOf(const std::string &menu) @@ -64,9 +62,8 @@ std::optional CommandLineHistoryService::previousOf(const s return std::nullopt; } -// NOLINTNEXTLINE(bugprone-easily-swappable-parameters) - both are named arguments of the same type. std::optional CommandLineHistoryService::startsWith(const std::string &command, - const std::string &menu) + std::string_view menu) { if (m_history.empty()) { return std::nullopt; } for (std::size_t i = m_history.size(); i > 0; i--) { diff --git a/src/features/cli/services/CommandLineService.cpp b/src/features/cli/services/CommandLineService.cpp index 0644fdc7..986ea71d 100644 --- a/src/features/cli/services/CommandLineService.cpp +++ b/src/features/cli/services/CommandLineService.cpp @@ -137,22 +137,21 @@ void CommandLineService::onPrompt() if (!m_terminalService->getLine().empty()) { std::cout << m_terminalService->getLine(); } } -// NOLINTNEXTLINE(bugprone-easily-swappable-parameters) — named out-params, both mutated by the prompt loop. -void CommandLineService::handleCommandInput(const SymbolEvent &symbolEvent, bool &tabPressed, bool &printPrompt) +CommandLineService::InputOutcome CommandLineService::handleCommandInput(const SymbolEvent &symbolEvent) { - tabPressed = false; - printPrompt = true; + // a command always resets the tab cycle and redraws the prompt + InputOutcome const outcome; std::string temp = symbolEvent.second; bool isLoggedIn = m_userApi->isLoggedIn(); if (!isLoggedIn) { std::cout << "WARNING: timeout of login server wise => shutdown cli\n"; SignalService::raiseSignal(SIGINT); m_running.store(false); - return; + return outcome; } if (std::cin.eof()) { onEndOfFile(); - return; + return outcome; } LOG_TRACE("received input >{}<", temp); std::vector flags = StringUtils::split(temp, ' '); @@ -163,19 +162,17 @@ void CommandLineService::handleCommandInput(const SymbolEvent &symbolEvent, bool } if (m_menu.currentOf() != nullptr) { onComponentCommand(input, flags); - return; + return outcome; } onCommand(input, flags); + return outcome; } -// NOLINTNEXTLINE(bugprone-easily-swappable-parameters) — named out-params, both mutated by the prompt loop. -void CommandLineService::handleTabInput(const SymbolEvent &symbolEvent, bool &tabPressed, bool &printPrompt) +CommandLineService::InputOutcome CommandLineService::handleTabInput(const SymbolEvent &symbolEvent, bool tabPressed) { if (symbolEvent.second.empty() && !tabPressed) { CommandLineUtils::beep(); - tabPressed = true; - printPrompt = false; - return; + return { .m_tabPressed = true, .m_printPrompt = false }; } if (!symbolEvent.second.empty()) { std::vector commands; @@ -195,15 +192,11 @@ void CommandLineService::handleTabInput(const SymbolEvent &symbolEvent, bool &ta if (commands.size() == 1) { std::string const missingPart = commands[0].substr(symbolEvent.second.length()); for (char const c : missingPart) { m_terminalService->onKeyPressed({ KeyType::Ascii, c }, menuNameOf()); } - printPrompt = false; - tabPressed = false; - return; + return { .m_tabPressed = false, .m_printPrompt = false }; } if (commands.empty()) { CommandLineUtils::beep(); - tabPressed = true; - printPrompt = false; - return; + return { .m_tabPressed = true, .m_printPrompt = false }; } std::cout << "\n\t"; int32_t i = 0; @@ -214,17 +207,14 @@ void CommandLineService::handleTabInput(const SymbolEvent &symbolEvent, bool &ta if (i % 10 > 0) { std::cout << "\t"; } } std::cout << "\n"; - printPrompt = true; - tabPressed = false; - return; + return { .m_tabPressed = false, .m_printPrompt = true }; } - printPrompt = true; - tabPressed = false; if (m_menu.currentOf() != nullptr) { m_menu.printCommandList(); - return; + return { .m_tabPressed = false, .m_printPrompt = true }; } m_commandParser.printCommandList(m_menu.allMenuEntriesOf()); + return { .m_tabPressed = false, .m_printPrompt = true }; } void CommandLineService::run() @@ -260,12 +250,18 @@ void CommandLineService::run() tabPressed = false; break; } - case Symbol::Tab: - handleTabInput(symbolEvent, tabPressed, printPrompt); + case Symbol::Tab: { + InputOutcome const outcome = handleTabInput(symbolEvent, tabPressed); + tabPressed = outcome.m_tabPressed; + printPrompt = outcome.m_printPrompt; break; - case Symbol::Command: - handleCommandInput(symbolEvent, tabPressed, printPrompt); + } + case Symbol::Command: { + InputOutcome const outcome = handleCommandInput(symbolEvent); + tabPressed = outcome.m_tabPressed; + printPrompt = outcome.m_printPrompt; break; + } case Symbol::Up: case Symbol::Down: { auto commandHistoryEntry = symbolEvent.first == Symbol::Up ? m_commandLineHistoryService->previousOf(menuNameOf()) diff --git a/src/features/http/controllers/AdminApi.cpp b/src/features/http/controllers/AdminApi.cpp new file mode 100644 index 00000000..5e0ba19e --- /dev/null +++ b/src/features/http/controllers/AdminApi.cpp @@ -0,0 +1,95 @@ +#include + +#include +#include +#include + +#include +#include +#include + +#include "lintel/core/utils/JsonUtils.h" +#include "lintel/features/http/controllers/AdminApi.h" +#include "lintel/features/http/service/HttpClientHelper.h" +#include "lintel/features/http/service/HttpStatusCodes.h" +#include "lintel/features/http/service/HttpUnauthorizedException.h" + +namespace lintel { +namespace { + /** @return {"level":""} */ + std::string logLevelBodyOf(const std::string &levelName) + { + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.Key("level"); + writer.String(levelName.c_str(), static_cast(levelName.size())); + writer.EndObject(); + return std::string(buffer.GetString(), buffer.GetSize()); + } + + /** + * @param result a completed request + * @return the response body, nullopt when no response was received + * @throws HttpUnauthorizedException if the token was missing or rejected + * + * Unlike the read endpoints of the other API clients this does not require + * status 200: the admin routes answer 409 "nothing_to_reload", 400 + * "invalid_level" and 500 "failed" with a body that the caller reports. + */ + std::optional bodyOf(const httplib::Result &result) + { + if (!HttpClientHelper::hasResponse(result)) { return std::nullopt; } + if (HttpStatusCodes(result->status) == HttpStatusCodes::Unauthorized) { throw HttpUnauthorizedException(); } + return result->body; + } +}// namespace + +AdminApi::AdminApi(const std::shared_ptr &clientProvider) : m_client(clientProvider->provide()) {} + +std::optional AdminApi::reloadTlsOf() +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const httplib::Result &result = m_client->post("/admin/reload-tls", headers, "", "application/json"); + const std::optional body = bodyOf(result); + if (!body.has_value()) { return std::nullopt; } + rapidjson::Document document; + document.Parse(body->c_str()); + if (document.HasParseError() || !document.IsObject()) { return std::nullopt; } + AdminTlsReloadResponse response; + if (!json::getString(document, "status", response.status)) { return std::nullopt; } + const rapidjson::Value *results = json::member(document, "results"); + if (results == nullptr || !results->IsObject()) { return std::nullopt; } + for (auto iter = results->MemberBegin(); iter != results->MemberEnd(); ++iter) { + // a malformed entry is skipped instead of failing the whole answer: the + // status still tells the caller whether the reload as a whole succeeded + if (!iter->name.IsString() || !iter->value.IsString()) { continue; } + AdminTlsReloadResult reloadResult; + reloadResult.name = std::string(iter->name.GetString(), iter->name.GetStringLength()); + reloadResult.detail = std::string(iter->value.GetString(), iter->value.GetStringLength()); + reloadResult.reloaded = reloadResult.detail == "ok"; + response.results.push_back(std::move(reloadResult)); + } + return response; +} + +std::optional AdminApi::logLevelOf(const std::string &levelName) +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const httplib::Result &result = + m_client->post("/admin/log-level", headers, logLevelBodyOf(levelName), "application/json"); + const std::optional body = bodyOf(result); + if (!body.has_value()) { return std::nullopt; } + rapidjson::Document document; + document.Parse(body->c_str()); + if (document.HasParseError() || !document.IsObject()) { return std::nullopt; } + AdminLogLevelResponse response; + if (!json::getString(document, "status", response.status)) { return std::nullopt; } + // the controller leaves the level empty for a malformed body, so a missing + // member is not an error here + json::getString(document, "level", response.level); + return response; +} +}// namespace lintel diff --git a/src/features/http/controllers/EventBusApi.cpp b/src/features/http/controllers/EventBusApi.cpp new file mode 100644 index 00000000..ef6ab6fc --- /dev/null +++ b/src/features/http/controllers/EventBusApi.cpp @@ -0,0 +1,184 @@ +#include + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "lintel/core/services/LoggerService.h" +#include "lintel/core/utils/JsonUtils.h" +#include "lintel/features/base/controller/EventBusDtos.h" +#include "lintel/features/http/controllers/EventBusApi.h" +#include "lintel/features/http/service/HttpClientHelper.h" +#include "lintel/features/http/service/HttpStatusCodes.h" +#include "lintel/features/http/service/HttpUnauthorizedException.h" + +namespace lintel { +namespace { + /** @return the JSON body of a publish request */ + std::string publishBodyOf(const std::string &topic, + const std::string &type, + const std::string &payload, + const std::string &priority, + bool toSubscribers) + { + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("PAYLOAD"); + writer.String(payload.c_str()); + writer.String("PRIORITY"); + writer.String(priority.c_str()); + writer.String("TARGET"); + writer.String(toSubscribers ? "SUBSCRIBERS" : "BUS"); + writer.String("TOPIC"); + writer.String(topic.c_str()); + writer.String("TYPE"); + writer.String(type.c_str()); + writer.EndObject(); + return std::string(buffer.GetString()); + } + + /** @return the JSON body of a subscribe/unsubscribe request */ + std::string subscriptionBodyOf(const std::string &topic, const std::string &subscriber) + { + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("SUBSCRIBER"); + writer.String(subscriber.c_str()); + writer.String("TOPIC"); + writer.String(topic.c_str()); + writer.EndObject(); + return std::string(buffer.GetString()); + } + + /** @param result a completed request + * @return true if the response may be parsed, false if the request was rejected + * @throws HttpUnauthorizedException if the token was missing or rejected */ + bool isSuccessful(const httplib::Result &result) + { + if (!HttpClientHelper::hasResponse(result)) { return false; } + HttpStatusCodes const status(result->status); + switch (status) { + case HttpStatusCodes::Unauthorized: + throw HttpUnauthorizedException(); + case HttpStatusCodes::OK: + return true; + default: + LOG_ERROR("event bus request rejected with status {}", status.getCode()); + return false; + } + } + + /** @param body a JSON object + * @param key member name + * @return the boolean member, nullopt when it is missing or mistyped */ + std::optional boolOf(const std::string &body, const char *key) + { + rapidjson::Document document; + document.Parse(body.c_str()); + if (document.HasParseError() || !document.IsObject()) { return std::nullopt; } + bool value = false; + if (!json::getBool(document, key, value)) { return std::nullopt; } + return value; + } +}// namespace + +EventBusApi::EventBusApi(const std::shared_ptr &clientProvider) : m_client(clientProvider->provide()) {} + +std::vector EventBusApi::allOf() +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const httplib::Result &result = m_client->get("/eventBus/buses", headers); + if (!isSuccessful(result)) { return {}; } + EventBusesDto eventBusesDto; + try { + eventBusesDto.deserialize(result->body); + } catch (const std::exception &exception) { + LOG_ERROR("{}", exception.what()); + return {}; + } + return eventBusesDto.getEventBusDtos(); +} + +std::optional EventBusApi::of(const std::string &name) +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const httplib::Result &result = m_client->get("/eventBus/bus/" + name, headers); + if (!isSuccessful(result)) { return std::nullopt; } + EventBusDto eventBusDto; + try { + eventBusDto.deserialize(result->body); + } catch (const std::exception &exception) { + LOG_ERROR("{}", exception.what()); + return std::nullopt; + } + return eventBusDto; +} + +std::optional EventBusApi::publishOf(const EventPublishRequest &request) +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const std::string body = + publishBodyOf(request.topic, request.type, request.payload, request.priority, request.toSubscribers); + const httplib::Result &result = m_client->post("/eventBus/publish/" + request.bus, headers, body, "application/json"); + if (!isSuccessful(result)) { return std::nullopt; } + return boolOf(result->body, "PUBLISHED"); +} + +std::optional EventBusApi::subscribeOf(const EventSubscription &subscription) +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const std::string body = subscriptionBodyOf(subscription.topic, subscription.subscriber); + const httplib::Result &result = + m_client->post("/eventBus/subscribe/" + subscription.bus, headers, body, "application/json"); + if (!isSuccessful(result)) { return std::nullopt; } + return boolOf(result->body, "SUBSCRIBED"); +} + +std::optional EventBusApi::unsubscribeOf(const EventSubscription &subscription) +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const std::string body = subscriptionBodyOf(subscription.topic, subscription.subscriber); + const httplib::Result &result = + m_client->deletes("/eventBus/subscribe/" + subscription.bus, headers, body, "application/json"); + if (!isSuccessful(result)) { return std::nullopt; } + return boolOf(result->body, "UNSUBSCRIBED"); +} + +std::optional EventBusApi::depthOf(const std::string &name, const std::string &subscriber) +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const httplib::Result &result = m_client->get("/eventBus/depth/" + name + "/" + subscriber, headers); + if (!isSuccessful(result)) { return std::nullopt; } + rapidjson::Document document; + document.Parse(result->body.c_str()); + if (document.HasParseError() || !document.IsObject()) { return std::nullopt; } + std::uint64_t depth = 0; + if (!json::getUint64(document, "DEPTH", depth)) { return std::nullopt; } + return static_cast(depth); +} + +std::optional EventBusApi::resetOf(const std::string &name) +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const httplib::Result &result = m_client->post("/eventBus/reset/" + name, headers, "", "application/json"); + if (!isSuccessful(result)) { return std::nullopt; } + return boolOf(result->body, "RESET"); +} +}// namespace lintel diff --git a/src/features/http/controllers/EventBusController.cpp b/src/features/http/controllers/EventBusController.cpp new file mode 100644 index 00000000..8ca6d5a6 --- /dev/null +++ b/src/features/http/controllers/EventBusController.cpp @@ -0,0 +1,421 @@ +#include "lintel/features/http/controllers/EventBusController.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "lintel/core/services/LoggerService.h" +#include "lintel/core/utils/JsonUtils.h" +#include "lintel/features/base/events/Event.h" +#include "lintel/features/http/service/HttpStatusCodes.h" + +namespace lintel { +namespace { + /** Longest topic/type/subscriber name that fits into an Event buffer. */ + constexpr std::size_t kMaxNameLength = Event::NAME_SIZE - 1; + /** Longest payload that fits into an Event buffer. */ + constexpr std::size_t kMaxPayloadLength = Event::CONTENT_SIZE; + + /** @param name a topic, type or subscriber name + * @return true if the name is not empty and fits into an Event */ + bool isValidName(const std::string &name) { return !name.empty() && name.size() <= kMaxNameLength; } + + /** @param payload a payload + * @return true if the payload fits into an Event without truncation */ + bool isValidPayload(const std::string &payload) { return payload.size() <= kMaxPayloadLength; } + + /** @param value a string + * @return the string with all ASCII letters converted to lower case */ + std::string lowerOf(const std::string &value) + { + std::string lower = value; + std::transform(lower.begin(), lower.end(), lower.begin(), [](unsigned char character) { + return static_cast(std::tolower(character)); + }); + return lower; + } + + /** @param value priority name, case-insensitive + * @return the priority, empty for an unknown name */ + std::optional priorityOf(const std::string &value) + { + const std::string lower = lowerOf(value); + if (lower == "critical") { return EventPriority::Critical; } + if (lower == "high") { return EventPriority::High; } + if (lower == "normal") { return EventPriority::Normal; } + if (lower == "low") { return EventPriority::Low; } + return std::nullopt; + } + + /** @param priority a priority + * @return the name used on the wire */ + const char *priorityNameOf(EventPriority priority) + { + switch (priority) { + case EventPriority::Critical: + return "Critical"; + case EventPriority::High: + return "High"; + case EventPriority::Normal: + return "Normal"; + case EventPriority::Low: + return "Low"; + } + return "Normal"; + } + + /** @param value target name, case-insensitive + * @return true to broadcast to the subscribers, false for the point-to-point + * queue, empty for an unknown name */ + std::optional toSubscribersOf(const std::string &value) + { + const std::string lower = lowerOf(value); + if (lower == "subscribers") { return true; } + if (lower == "bus") { return false; } + return std::nullopt; + } + + /** @param match a matched path segment of the route pattern + * @return the segment as string */ + std::string segmentOf(const std::ssub_match &match) { return match.str(); } + + /** @return the current wall clock time as seconds since the epoch */ + std::int64_t timestampNow() + { + return std::chrono::duration_cast(std::chrono::system_clock::now().time_since_epoch()) + .count(); + } + + /** @param body parsed JSON request body + * @param key member name + * @param out receives the string value + * @return true if the member is absent or a string, false on a type mismatch */ + bool optionalStringOf(const rapidjson::Document &body, const char *key, std::string &out) + { + if (json::member(body, key) == nullptr) { return true; } + return json::getString(body, key, out); + } +}// namespace + +EventBusController::EventBusController(const std::shared_ptr &authService, + std::shared_ptr eventBusService) + : Controller(authService), m_eventBusService(std::move(eventBusService)), m_adminGroup("Admin-EventBus", {}, true), + m_userGroup("User-EventBus", {}, true), m_publisherGroup("Publisher-EventBus", {}, true) +{ + add(m_adminGroup); + add(m_userGroup); + add(m_publisherGroup); +} + +bool EventBusController::authorize(httplib::Response &response, + const ContentType &contentType, + const std::optional &user, + Access access) const +{ + if (!user.has_value()) { + // Same answer for "no token" and "wrong group": telling them apart would + // confirm to an anonymous caller that the endpoint exists. + response.status = HttpStatusCodes::Unauthorized; + response.set_content("", contentType.getName()); + return false; + } + const bool isAdmin = user->m_user.has(m_adminGroup); + bool permitted = isAdmin; + if (access == Access::Read) { permitted = permitted || user->m_user.has(m_userGroup); } + if (access == Access::Publish) { permitted = permitted || user->m_user.has(m_publisherGroup); } + if (!permitted) { + // A valid token of a user outside the required group is answered like a + // missing token on purpose. + response.status = HttpStatusCodes::Unauthorized; + response.set_content("", contentType.getName()); + return false; + } + if (contentType != ContentType::ApplicationJson) { + response.status = HttpStatusCodes::Forbidden; + response.set_content("", contentType.getName()); + return false; + } + return true; +} + +std::optional> + EventBusController::busOf(httplib::Response &response, const ContentType &contentType, const std::string &name) +{ + const std::vector names = m_eventBusService->namesOf(); + if (std::find(names.begin(), names.end(), name) == names.end()) { + response.status = HttpStatusCodes::NotFound; + response.set_content("", contentType.getName()); + return std::nullopt; + } + return m_eventBusService->of(name); +} + +EventBusDto EventBusController::describeOf(const std::string &name) const +{ + EventBusDto eventBusDto(name); + try { + eventBusDto = EventBusDto(name, m_eventBusService->configOf(name)); + } catch (const std::exception &exception) { + LOG_WARN("event bus '{}' has no usable configuration: {}", name, exception.what()); + return eventBusDto; + } + try { + const std::shared_ptr bus = m_eventBusService->of(name); + eventBusDto.markAvailable(EventBusCounters{ bus->droppedEventsOf(), + bus->topicCount(), + bus->subscriberCount(), + m_eventBusService->busQueueDepthOf(name), + bus->sequenceOf() }); + } catch (const std::exception &exception) { + // a bus whose segment is missing stays listed, just without live counters + LOG_WARN("event bus '{}' cannot be opened: {}", name, exception.what()); + } + return eventBusDto; +} + +void EventBusController::busesOfGet(const httplib::Request & /*request*/, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!authorize(response, contentType, user, Access::Read)) { return; } + const std::vector names = m_eventBusService->namesOf(); + std::vector eventBusDtos; + eventBusDtos.reserve(names.size()); + for (const auto &name : names) { eventBusDtos.push_back(describeOf(name)); } + const EventBusesDto eventBusesDto(std::move(eventBusDtos)); + response.set_content(eventBusesDto.JsonSerializable::serialize(), contentType.getName()); +} + +void EventBusController::busOfGet(const httplib::Request &request, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!authorize(response, contentType, user, Access::Read)) { return; } + if (!busOf(response, contentType, segmentOf(request.matches[1])).has_value()) { return; } + response.set_content(describeOf(segmentOf(request.matches[1])).JsonSerializable::serialize(), contentType.getName()); +} + +void EventBusController::publishOfPost(const httplib::Request &request, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!authorize(response, contentType, user, Access::Publish)) { return; } + const std::optional> bus = busOf(response, contentType, segmentOf(request.matches[1])); + if (!bus.has_value()) { return; } + const auto badRequest = [&response, &contentType]() { + response.status = HttpStatusCodes::BadRequest; + response.set_content("", contentType.getName()); + }; + rapidjson::Document body; + body.Parse(request.body.c_str()); + std::string topic; + std::string type; + if (body.HasParseError() || !json::getString(body, "TOPIC", topic) || !json::getString(body, "TYPE", type) + || !isValidName(topic) || !isValidName(type)) { + badRequest(); + return; + } + std::string payload; + if (!optionalStringOf(body, "PAYLOAD", payload) || !isValidPayload(payload)) { + badRequest(); + return; + } + std::string priorityName = "Normal"; + if (!optionalStringOf(body, "PRIORITY", priorityName)) { + badRequest(); + return; + } + const std::optional priority = priorityOf(priorityName); + if (!priority.has_value()) { + badRequest(); + return; + } + std::string targetName = "BUS"; + if (!optionalStringOf(body, "TARGET", targetName)) { + badRequest(); + return; + } + const std::optional toSubscribers = toSubscribersOf(targetName); + if (!toSubscribers.has_value()) { + badRequest(); + return; + } + Event event(topic, type); + event.setPriority(priority.value()); + event.setTimestamp(timestampNow()); + if (!payload.empty()) { event.setContent(reinterpret_cast(payload.data()), payload.size()); } + const bool published = toSubscribers.value() ? (*bus)->publishToSubscribers(event) : (*bus)->publish(event); + const std::int64_t sequence = (*bus)->sequenceOf(); + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("BUS"); + writer.String(segmentOf(request.matches[1]).c_str()); + writer.String("PRIORITY"); + writer.String(priorityNameOf(priority.value())); + writer.String("PUBLISHED"); + writer.Bool(published); + writer.String("SEQUENCE"); + writer.Int64(sequence); + writer.String("TARGET"); + writer.String(toSubscribers.value() ? "SUBSCRIBERS" : "BUS"); + writer.String("TOPIC"); + writer.String(topic.c_str()); + writer.String("TYPE"); + writer.String(type.c_str()); + writer.EndObject(); + response.set_content(std::string(buffer.GetString()), contentType.getName()); + LOG_INFO( + "published event '{}' ({}) on bus '{}': published={}", topic, type, segmentOf(request.matches[1]), published); +} + +void EventBusController::subscribeOfPost(const httplib::Request &request, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!authorize(response, contentType, user, Access::Administer)) { return; } + const std::optional> bus = busOf(response, contentType, segmentOf(request.matches[1])); + if (!bus.has_value()) { return; } + rapidjson::Document body; + body.Parse(request.body.c_str()); + std::string topic; + std::string subscriber; + if (body.HasParseError() || !json::getString(body, "TOPIC", topic) || !json::getString(body, "SUBSCRIBER", subscriber) + || !isValidName(topic) || !isValidName(subscriber)) { + response.status = HttpStatusCodes::BadRequest; + response.set_content("", contentType.getName()); + return; + } + // An exact topic has to be registered before a subscriber may join it, while + // a wildcard pattern registers itself inside subscribe(). Registering here + // keeps the endpoint usable without a separate topic route. + if (!(*bus)->isTopicRegistered(topic) && !(*bus)->registerTopic(topic)) { + response.status = HttpStatusCodes::BadRequest; + response.set_content("", contentType.getName()); + return; + } + const bool subscribed = (*bus)->subscribe(topic, subscriber); + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("BUS"); + writer.String(segmentOf(request.matches[1]).c_str()); + writer.String("SUBSCRIBER"); + writer.String(subscriber.c_str()); + writer.String("SUBSCRIBED"); + writer.Bool(subscribed); + writer.String("TOPIC"); + writer.String(topic.c_str()); + writer.EndObject(); + response.set_content(std::string(buffer.GetString()), contentType.getName()); + LOG_INFO("subscriber '{}' of bus '{}': topic='{}' subscribed={}", + subscriber, + segmentOf(request.matches[1]), + topic, + subscribed); +} + +void EventBusController::unsubscribeOfDelete(const httplib::Request &request, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!authorize(response, contentType, user, Access::Administer)) { return; } + const std::optional> bus = busOf(response, contentType, segmentOf(request.matches[1])); + if (!bus.has_value()) { return; } + rapidjson::Document body; + body.Parse(request.body.c_str()); + std::string topic; + std::string subscriber; + if (body.HasParseError() || !json::getString(body, "TOPIC", topic) || !json::getString(body, "SUBSCRIBER", subscriber) + || !isValidName(topic) || !isValidName(subscriber)) { + response.status = HttpStatusCodes::BadRequest; + response.set_content("", contentType.getName()); + return; + } + const bool unsubscribed = (*bus)->unsubscribe(topic, subscriber); + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("BUS"); + writer.String(segmentOf(request.matches[1]).c_str()); + writer.String("SUBSCRIBER"); + writer.String(subscriber.c_str()); + writer.String("TOPIC"); + writer.String(topic.c_str()); + writer.String("UNSUBSCRIBED"); + writer.Bool(unsubscribed); + writer.EndObject(); + response.set_content(std::string(buffer.GetString()), contentType.getName()); + LOG_INFO("subscriber '{}' of bus '{}': topic='{}' unsubscribed={}", + subscriber, + segmentOf(request.matches[1]), + topic, + unsubscribed); +} + +void EventBusController::depthOfGet(const httplib::Request &request, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!authorize(response, contentType, user, Access::Read)) { return; } + if (!busOf(response, contentType, segmentOf(request.matches[1])).has_value()) { return; } + const std::string subscriber = segmentOf(request.matches[2]); + if (!isValidName(subscriber)) { + response.status = HttpStatusCodes::BadRequest; + response.set_content("", contentType.getName()); + return; + } + const std::size_t depth = m_eventBusService->subscriberQueueDepthOf(segmentOf(request.matches[1]), subscriber); + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("BUS"); + writer.String(segmentOf(request.matches[1]).c_str()); + writer.String("DEPTH"); + writer.Uint64(static_cast(depth)); + writer.String("SUBSCRIBER"); + writer.String(subscriber.c_str()); + writer.EndObject(); + response.set_content(std::string(buffer.GetString()), contentType.getName()); +} + +void EventBusController::resetOfPost(const httplib::Request &request, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!authorize(response, contentType, user, Access::Administer)) { return; } + const std::optional> bus = busOf(response, contentType, segmentOf(request.matches[1])); + if (!bus.has_value()) { return; } + (*bus)->reset(); + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("BUS"); + writer.String(segmentOf(request.matches[1]).c_str()); + writer.String("RESET"); + writer.Bool(true); + writer.EndObject(); + response.set_content(std::string(buffer.GetString()), contentType.getName()); + LOG_INFO("reset event bus '{}'", segmentOf(request.matches[1])); +} +}// namespace lintel diff --git a/src/features/http/controllers/HistoryController.cpp b/src/features/http/controllers/HistoryController.cpp index caf6b278..f9f309c3 100644 --- a/src/features/http/controllers/HistoryController.cpp +++ b/src/features/http/controllers/HistoryController.cpp @@ -33,12 +33,7 @@ void HistoryController::historyOfGet(const httplib::Request &request, } } -// groupRepository is not evaluated - the groups are hardcoded in the -// member init list (m_adminGroup/m_userGroup). The signature stays unchanged -// because the constructor is resolved by the DI container; removing the -// parameter would be a wiring and API topic of its own. HistoryController::HistoryController(const std::shared_ptr &authService, - std::shared_ptr groupRepository,// NOLINT(performance-unnecessary-value-param) std::shared_ptr historyRepository) : Controller(authService), m_historyRepository(std::move(historyRepository)), m_adminGroup("Admin-History", {}, true), m_userGroup("User-History", {}, true) diff --git a/src/features/http/controllers/MessageQueueApi.cpp b/src/features/http/controllers/MessageQueueApi.cpp index 4c426594..d8769590 100644 --- a/src/features/http/controllers/MessageQueueApi.cpp +++ b/src/features/http/controllers/MessageQueueApi.cpp @@ -1,8 +1,14 @@ #include +#include +#include + +#include + #include "lintel/features/http/controllers/MessageQueueApi.h" #include "lintel/core/services/LoggerService.h" +#include "lintel/core/utils/JsonUtils.h" #include "lintel/features/base/controller/MessageQueueDtos.h" #include "lintel/features/http/service/HttpClientHelper.h" #include "lintel/features/http/service/HttpStatusCodes.h" @@ -62,4 +68,27 @@ std::optional MesssageQueueApi::purgeOf(const std::string &message } return purgeResult.getPurged(); } + +std::optional MesssageQueueApi::recoverOf() +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const auto &result = m_client->post("/messageQueue/recover", headers, "", "application/json"); + if (!HttpClientHelper::hasResponse(result)) { return std::nullopt; } + HttpStatusCodes const status(result->status); + switch (status) { + case HttpStatusCodes::Unauthorized: + throw HttpUnauthorizedException(); + case HttpStatusCodes::OK: + break; + default: + return std::nullopt; + } + rapidjson::Document document; + document.Parse(result->body.c_str()); + if (document.HasParseError() || !document.IsObject()) { return std::nullopt; } + std::uint64_t recovered = 0; + if (!json::getUint64(document, "RECOVERED", recovered)) { return std::nullopt; } + return static_cast(recovered); +} }// namespace lintel \ No newline at end of file diff --git a/src/features/http/controllers/MessageQueueController.cpp b/src/features/http/controllers/MessageQueueController.cpp index ab69181f..54a42aa2 100644 --- a/src/features/http/controllers/MessageQueueController.cpp +++ b/src/features/http/controllers/MessageQueueController.cpp @@ -3,8 +3,13 @@ #include "lintel/features/base/controller/MessageQueueDtos.h" #include "lintel/features/base/models/MessageQueueStats.h" +#include #include #include +#include + +#include +#include namespace lintel { MessageQueueController::MessageQueueController(const std::shared_ptr &authService, @@ -126,4 +131,36 @@ void MessageQueueController::purgeOfPost(const httplib::Request &request, MessageQueuePurgeDto result(entry->get_message_queue_name(), purged); response.set_content(result.JsonSerializable::serialize(), contentType.getName()); } + +void MessageQueueController::recoverOfPost(const httplib::Request & /*request*/, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!user.has_value()) { + response.status = HttpStatusCodes::Unauthorized; + response.set_content("", contentType.getName()); + return; + } + // reclaiming entries of dead owners mutates queue state, so only administrators may do it + if (!canAdminister(user)) { + response.status = HttpStatusCodes::Unauthorized; + response.set_content("", contentType.getName()); + return; + } + if (contentType != ContentType::ApplicationJson) { + response.status = HttpStatusCodes::Forbidden; + response.set_content("", contentType.getName()); + return; + } + const std::size_t recovered = m_messageQueueService->recoverOrphans(); + LOG_INFO("recoverOf: reclaimed {} orphaned message queue entries", recovered); + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("RECOVERED"); + writer.Uint64(recovered); + writer.EndObject(); + response.set_content(std::string(buffer.GetString()), contentType.getName()); +} }// namespace lintel diff --git a/src/features/http/controllers/SharedMemoryApi.cpp b/src/features/http/controllers/SharedMemoryApi.cpp index c2fca35a..12f6d1fe 100644 --- a/src/features/http/controllers/SharedMemoryApi.cpp +++ b/src/features/http/controllers/SharedMemoryApi.cpp @@ -1,8 +1,15 @@ #include +#include +#include + +#include + #include "lintel/core/services/LoggerService.h" +#include "lintel/core/utils/JsonUtils.h" #include "lintel/features/base/controller/SharedMemoryRepositoriesDto.h" #include "lintel/features/base/controller/SharedMemorySegmentsDto.h" +#include "lintel/features/base/models/SharedMemorySegmentStats.h" #include "lintel/features/http/controllers/SharedMemoryApi.h" #include "lintel/features/http/service/HttpClientHelper.h" #include "lintel/features/http/service/HttpStatusCodes.h" @@ -101,6 +108,100 @@ void SharedMemoryApi::growOf(const std::string &segmentName, const std::string & LOG_TRACE("successfully shrinking segment {}", segmentName); } +std::optional SharedMemoryApi::statsOf(const std::string &segmentName) +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const httplib::Result &result = m_client->get("/shm/stats/" + segmentName, headers); + if (!HttpClientHelper::hasResponse(result)) { return std::nullopt; } + HttpStatusCodes status(result->status); + switch (status) { + case HttpStatusCodes::Unauthorized: + throw HttpUnauthorizedException(); + case HttpStatusCodes::OK: + break; + default: + LOG_ERROR("error {}", status.getCode()); + return std::nullopt; + } + rapidjson::Document document; + document.Parse(result->body.c_str()); + if (document.HasParseError() || !document.IsObject()) { return std::nullopt; } + SharedMemorySegmentStats stats; + if (!json::getString(document, "NAME", stats.name)) { return std::nullopt; } + const auto readCount = [&document](const char *key, std::size_t &out) { + std::uint64_t value = 0; + if (!json::getUint64(document, key, value)) { return false; } + out = static_cast(value); + return true; + }; + if (!readCount("ALLOCATED_BYTES", stats.allocatedBytes) || !readCount("CAPACITY_BYTES", stats.capacityBytes) + || !readCount("FREE_BYTES", stats.freeBytes) || !readCount("GROW_COUNT", stats.growCount) + || !readCount("SHRINK_COUNT", stats.shrinkCount) || !readCount("RECOVERY_COUNT", stats.recoveryCount) + || !readCount("GENERATION", stats.generation)) { + return std::nullopt; + } + return stats; +} + +std::optional SharedMemoryApi::cleanupOf() +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const httplib::Result &result = m_client->post("/shm/cleanup", headers, "", "application/json"); + if (!HttpClientHelper::hasResponse(result)) { return std::nullopt; } + HttpStatusCodes status(result->status); + switch (status) { + case HttpStatusCodes::Unauthorized: + throw HttpUnauthorizedException(); + case HttpStatusCodes::OK: + break; + default: + LOG_ERROR("error {}", status.getCode()); + return std::nullopt; + } + rapidjson::Document document; + document.Parse(result->body.c_str()); + if (document.HasParseError() || !document.IsObject()) { return std::nullopt; } + SharedMemoryCleanupSummary summary; + if (!json::getBool(document, "DRY_RUN", summary.dryRun)) { return std::nullopt; } + const auto readCount = [&document](const char *key, std::size_t &out) { + std::uint64_t value = 0; + if (!json::getUint64(document, key, value)) { return false; } + out = static_cast(value); + return true; + }; + if (!readCount("SCANNED", summary.scanned) || !readCount("ORPHANS", summary.orphans) + || !readCount("REMOVED", summary.removed) || !readCount("FAILED", summary.failed)) { + return std::nullopt; + } + return summary; +} + +std::optional SharedMemoryApi::recoverOf() +{ + httplib::Headers headers{}; + headers.insert({ "Content-Type", "application/json" }); + const httplib::Result &result = m_client->post("/shm/recover", headers, "", "application/json"); + if (!HttpClientHelper::hasResponse(result)) { return std::nullopt; } + HttpStatusCodes status(result->status); + switch (status) { + case HttpStatusCodes::Unauthorized: + throw HttpUnauthorizedException(); + case HttpStatusCodes::OK: + break; + default: + LOG_ERROR("error {}", status.getCode()); + return std::nullopt; + } + rapidjson::Document document; + document.Parse(result->body.c_str()); + if (document.HasParseError() || !document.IsObject()) { return std::nullopt; } + std::uint64_t recovered = 0; + if (!json::getUint64(document, "RECOVERED", recovered)) { return std::nullopt; } + return static_cast(recovered); +} + std::string SharedMemoryApi::repositoryOf(const std::string &uuid) { httplib::Headers headers{}; diff --git a/src/features/http/controllers/SharedMemoryController.cpp b/src/features/http/controllers/SharedMemoryController.cpp index 45575a0f..b287c1bf 100644 --- a/src/features/http/controllers/SharedMemoryController.cpp +++ b/src/features/http/controllers/SharedMemoryController.cpp @@ -2,19 +2,78 @@ #include +#include +#include + +#include "lintel/core/exceptions/ShmSegmentNotFound.h" #include "lintel/core/utils/MemorySize.h" #include "lintel/core/utils/RegexUtils.h" #include "lintel/features/base/controller/SharedMemoryRepositoriesDto.h" #include "lintel/features/base/controller/SharedMemorySegmentDto.h" #include "lintel/features/base/controller/SharedMemorySegmentsDto.h" +#include "lintel/features/base/models/SharedMemorySegmentStats.h" namespace lintel { +namespace { + /** Serialize the statistics of one segment. + * @param stats statistics to write + * @return JSON object with the uppercase field names used by the other DTOs */ + std::string shmStatsJsonOf(const SharedMemorySegmentStats &stats) + { + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("NAME"); + writer.String(stats.name.c_str()); + writer.String("ALLOCATED_BYTES"); + writer.Uint64(stats.allocatedBytes); + writer.String("CAPACITY_BYTES"); + writer.Uint64(stats.capacityBytes); + writer.String("FREE_BYTES"); + writer.Uint64(stats.freeBytes); + writer.String("GROW_COUNT"); + writer.Uint64(stats.growCount); + writer.String("SHRINK_COUNT"); + writer.Uint64(stats.shrinkCount); + writer.String("RECOVERY_COUNT"); + writer.Uint64(stats.recoveryCount); + writer.String("GENERATION"); + writer.Uint64(stats.generation); + writer.EndObject(); + return std::string(buffer.GetString()); + } + + /** Serialize the counts of one cleanup run. + * @param report report of scan() or apply() + * @return JSON object with the uppercase field names used by the other DTOs */ + std::string shmCleanupJsonOf(const SharedMemoryCleanupReport &report) + { + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("DRY_RUN"); + writer.Bool(report.dryRun); + writer.String("SCANNED"); + writer.Uint64(report.scanned); + writer.String("ORPHANS"); + writer.Uint64(report.orphanCount()); + writer.String("REMOVED"); + writer.Uint64(report.removed.size()); + writer.String("FAILED"); + writer.Uint64(report.failed); + writer.EndObject(); + return std::string(buffer.GetString()); + } +}// namespace + SharedMemoryController::SharedMemoryController(const std::shared_ptr &authServicie, std::shared_ptr sharedMemoryService, std::shared_ptr sharedMemorySegmentManager, - std::vector> sharedMemoryRepositories) + std::vector> sharedMemoryRepositories, + std::shared_ptr sharedMemoryCleanupService) : Controller(authServicie), m_sharedMemoryService(std::move(sharedMemoryService)), m_sharedMemorySegmentManager(std::move(sharedMemorySegmentManager)), + m_sharedMemoryCleanupService(std::move(sharedMemoryCleanupService)), m_sharedMemoryRepositories(std::move(sharedMemoryRepositories)), m_uuidSharedMemoryRepositories(build(m_sharedMemoryRepositories)), m_adminGroup("Admin-Shm", {}, true), m_userGroup("User-Shm", {}, true) @@ -23,6 +82,16 @@ SharedMemoryController::SharedMemoryController(const std::shared_ptr &user) const +{ + return user.has_value() && (user->m_user.has(m_userGroup) || user->m_user.has(m_adminGroup)); +} + +bool SharedMemoryController::canAdminister(const std::optional &user) const +{ + return user.has_value() && user->m_user.has(m_adminGroup); +} + std::map> SharedMemoryController::build( const std::vector> &repositories) { @@ -223,4 +292,110 @@ void SharedMemoryController::growSegmentOfPut(const httplib::Request &request, response.set_content("", contentType.getName()); } } + +void SharedMemoryController::statsOfGet(const httplib::Request &request, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!canRead(user)) { + response.status = HttpStatusCodes::Unauthorized; + response.set_content("", contentType.getName()); + return; + } + if (contentType != ContentType::ApplicationJson) { + response.status = HttpStatusCodes::Forbidden; + response.set_content("", contentType.getName()); + return; + } + const std::string segmentName = request.matches[1]; + const std::shared_ptr segment = m_sharedMemorySegmentManager->of(segmentName); + if (segment == nullptr) { + LOG_WARN("statsOf: segment {} not found", segmentName); + response.status = HttpStatusCodes::NotFound; + response.set_content("", contentType.getName()); + return; + } + SharedMemorySegmentStats stats; + try { + stats = m_sharedMemoryService->statsOf(segment); + } catch (const ShmSegmentNotFound &error) { + // the segment manager knows the segment, but this process does not map it + LOG_WARN("statsOf: {}", error.what()); + response.status = HttpStatusCodes::NotFound; + response.set_content("", contentType.getName()); + return; + } + response.set_content(shmStatsJsonOf(stats), contentType.getName()); +} + +void SharedMemoryController::cleanupOfPost(const httplib::Request & /*request*/, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!user.has_value()) { + response.status = HttpStatusCodes::Unauthorized; + response.set_content("", contentType.getName()); + return; + } + // cleanup removes objects, so only administrators may run it + if (!canAdminister(user)) { + response.status = HttpStatusCodes::Unauthorized; + response.set_content("", contentType.getName()); + return; + } + if (contentType != ContentType::ApplicationJson) { + response.status = HttpStatusCodes::Forbidden; + response.set_content("", contentType.getName()); + return; + } + if (m_sharedMemoryCleanupService == nullptr) { + LOG_ERROR("cleanupOf: no cleanup service is registered"); + response.status = HttpStatusCodes::InternalServerError; + response.set_content("", contentType.getName()); + return; + } + const SharedMemoryCleanupReport report = m_sharedMemoryCleanupService->apply(); + LOG_INFO("cleanupOf: scanned {} entries, removed {} orphans, {} failures", + report.scanned, + report.removed.size(), + report.failed); + response.set_content(shmCleanupJsonOf(report), contentType.getName()); +} + +void SharedMemoryController::recoverOfPost(const httplib::Request & /*request*/, + httplib::Response &response, + const ContentType &contentType, + const std::optional &user) +{ + if (!user.has_value()) { + response.status = HttpStatusCodes::Unauthorized; + response.set_content("", contentType.getName()); + return; + } + // reclaiming a stuck remap lock changes the lock state of a segment, so only + // administrators may run it + if (!canAdminister(user)) { + response.status = HttpStatusCodes::Unauthorized; + response.set_content("", contentType.getName()); + return; + } + if (contentType != ContentType::ApplicationJson) { + response.status = HttpStatusCodes::Forbidden; + response.set_content("", contentType.getName()); + return; + } + // One pass over the segments this process maps: locks whose holder is gone + // are reclaimed, locks of a living holder stay untouched. + const std::size_t recovered = m_sharedMemoryService->recover(); + LOG_INFO("recoverOf: reclaimed {} orphaned remap locks", recovered); + rapidjson::StringBuffer buffer; + rapidjson::Writer writer(buffer); + writer.StartObject(); + writer.String("RECOVERED"); + writer.Uint64(recovered); + writer.EndObject(); + response.set_content(std::string(buffer.GetString()), contentType.getName()); +} }// namespace lintel diff --git a/src/features/property/repositories/DatabasePropertyRepository.cpp b/src/features/property/repositories/DatabasePropertyRepository.cpp index 04fed697..c9641c61 100644 --- a/src/features/property/repositories/DatabasePropertyRepository.cpp +++ b/src/features/property/repositories/DatabasePropertyRepository.cpp @@ -181,11 +181,13 @@ void DatabasePropertyRepository::deleteOf(const std::vector> DatabasePropertyRepository::allOf(const std::string &processName, - const std::string &className, - const std::string &instanceName, - const std::string &name) +std::vector> DatabasePropertyRepository::allOf(const PropertyFilter &filter) { + // SQL only understands LIKE, so the regex default ".*" becomes "%" + const std::string &processName = filter.m_processName; + const std::string &className = filter.m_className; + const std::string &instanceName = filter.m_instanceName; + const std::string &name = filter.m_name; std::string processNameLike = StringUtils::replaceAll(processName, ".*", "%"); std::string classNameLike = StringUtils::replaceAll(className, ".*", "%"); std::string instanceNameLike = StringUtils::replaceAll(instanceName, ".*", "%"); diff --git a/src/features/property/repositories/SharedMemoryPropertyRepository.cpp b/src/features/property/repositories/SharedMemoryPropertyRepository.cpp index 4352c37d..a033031b 100644 --- a/src/features/property/repositories/SharedMemoryPropertyRepository.cpp +++ b/src/features/property/repositories/SharedMemoryPropertyRepository.cpp @@ -83,18 +83,14 @@ std::vector> SharedMemoryPropertyRepository::awake return properties; } -std::vector> - // four identically typed parameters are part of the pure - // PropertyRepository interface; the order follows the - // process/property hierarchy and is shared by all implementations and - // callers. A change would be a breaking change of the interface. - // clang-format does not expect a line break here, hence NOLINT instead of a break. - // NOLINTNEXTLINE(bugprone-easily-swappable-parameters) - SharedMemoryPropertyRepository::allOf(const std::string &processName, - const std::string &className, - const std::string &instanceName, - const std::string &name) +std::vector> SharedMemoryPropertyRepository::allOf(const PropertyFilter &filter) { + // the filter is passed as one named aggregate; bind its fields so that the + // matching code below reads like the individual regexes + const std::string &processName = filter.m_processName; + const std::string &className = filter.m_className; + const std::string &instanceName = filter.m_instanceName; + const std::string &name = filter.m_name; boost::interprocess::sharable_lock lock(m_upgradableMutex); LOG_TRACE("start awake {}", getMap().size()); std::vector> properties; diff --git a/src/features/property/services/PropertyService.cpp b/src/features/property/services/PropertyService.cpp index b26d885d..b1a8c0ad 100644 --- a/src/features/property/services/PropertyService.cpp +++ b/src/features/property/services/PropertyService.cpp @@ -200,7 +200,7 @@ std::vector> PropertyService::allOf(const std::str }); return propertiesVector; } - return m_shadowPropertyRepositories[0]->allOf(processName); + return m_shadowPropertyRepositories[0]->allOf(PropertyFilter{ .m_processName = processName }); } std::vector> PropertyService::allOf(const std::string &processName, @@ -217,7 +217,8 @@ std::vector> PropertyService::allOf(const std::str } return propertiesVector; } - return m_shadowPropertyRepositories[0]->allOf(processName, className); + return m_shadowPropertyRepositories[0]->allOf( + PropertyFilter{ .m_processName = processName, .m_className = className }); } std::vector> @@ -238,7 +239,8 @@ std::vector> } return propertiesVector; } - return m_shadowPropertyRepositories[0]->allOf(processName, className, instanceName); + return m_shadowPropertyRepositories[0]->allOf( + PropertyFilter{ .m_processName = processName, .m_className = className, .m_instanceName = instanceName }); } std::string PropertyService::createIdentifier(const std::string &name, @@ -376,7 +378,8 @@ std::optional PropertyService::resolveChangeValue(const PropertyCha if (repository->getType() != PropertyRepositoryType::SHM_REPOSITORY) { continue; } std::vector> candidates; try { - candidates = repository->allOf(change.processName, change.className, change.instanceName, change.name); + candidates = + repository->allOf(PropertyFilter{ change.processName, change.className, change.instanceName, change.name }); } catch (const std::exception &exception) { LOG_ERROR("cannot resolve value of {}: {}", change.identifier(), exception.what()); return std::nullopt; diff --git a/src/include/lintel/app/plugins/VirtualGroupBootstrapPlugin.h b/src/include/lintel/app/plugins/VirtualGroupBootstrapPlugin.h index faf90b71..ffe691f7 100644 --- a/src/include/lintel/app/plugins/VirtualGroupBootstrapPlugin.h +++ b/src/include/lintel/app/plugins/VirtualGroupBootstrapPlugin.h @@ -13,7 +13,11 @@ namespace lintel { class VirtualGroupBootstrapPlugin : public BootstrapPlugin { private: - std::shared_ptr m_container; + /** Weak on purpose: the plugin is a singleton *inside* the container, so a + * shared_ptr here would close a reference cycle (container -> plugin -> + * container) and the container would never be destroyed - which in turn kept + * every service alive, so no service destructor ever ran. */ + std::weak_ptr m_container; std::shared_ptr m_connectionEntry; public: @@ -22,7 +26,7 @@ class VirtualGroupBootstrapPlugin : public BootstrapPlugin * @param container the dependency injection container */ explicit VirtualGroupBootstrapPlugin( const std::shared_ptr &connectionConfigurations, - std::shared_ptr container); + const std::shared_ptr &container); /** Load virtual group configuration from the database. */ void onStart() override; diff --git a/src/include/lintel/core/Feature.h b/src/include/lintel/core/Feature.h index 6f61ea51..7edf7509 100644 --- a/src/include/lintel/core/Feature.h +++ b/src/include/lintel/core/Feature.h @@ -84,15 +84,21 @@ template> * = nullptr> class Feat std::shared_ptr m_features; protected: - /** The name of this feature derived from the enum value */ - std::string_view m_name; + /** The name of this feature derived from the enum value. + * + * Stored by value on purpose: magic_enum::enum_name() returns a view into + * static storage, but the initialiser below used to build a temporary + * std::string and store a std::string_view of it. That view dangled as soon + * as the constructor returned, so getName() handed out freed stack bytes - + * which StartupBuilder::verifyFeatureDependencies() then compared against. */ + std::string m_name; public: /** Construct a feature and register it with the central Features manager * @param type The enum value identifying this feature * @param features The central Features manager instance */ Feature(T type, std::shared_ptr features) - : m_name(std::string(magic_enum::enum_name(type))), m_features(std::move(features)) + : m_name(magic_enum::enum_name(type)), m_features(std::move(features)) { m_features->registerFeature( type, diff --git a/src/include/lintel/core/persistence/sqlite3/Connection.h b/src/include/lintel/core/persistence/sqlite3/Connection.h index 6d76bac4..0f6ddea9 100644 --- a/src/include/lintel/core/persistence/sqlite3/Connection.h +++ b/src/include/lintel/core/persistence/sqlite3/Connection.h @@ -6,6 +6,7 @@ #include #include #include +#include #include "lintel/core/configuration/DatabaseConnectionEntry.h" #include "lintel/core/persistence/Parameter.h" @@ -73,7 +74,7 @@ namespace sqlite { * @param query the SQL query * @return shared pointer to the result */ - std::shared_ptr prepareStatement(const std::string &queryName, const std::string &query); + std::shared_ptr prepareStatement(std::string_view queryName, const std::string &query); /** * Executes a previously prepared statement. @@ -90,16 +91,6 @@ namespace sqlite { */ void finalizePreparedStatement(const std::string &queryName); - /** - * Static callback for sqlite3_exec results. - * @param funcPtr user data pointer - * @param argc number of columns - * @param argv column values - * @param column column names - * @return 0 on success - */ - static int callBack(void *funcPtr, int argc, char **argv, char **column); - /** @return the last error message from SQLite */ [[nodiscard]] std::string getErrorMessage() const; diff --git a/src/include/lintel/core/platform/Platform.h b/src/include/lintel/core/platform/Platform.h index c37de29f..1a3094ed 100644 --- a/src/include/lintel/core/platform/Platform.h +++ b/src/include/lintel/core/platform/Platform.h @@ -5,6 +5,7 @@ #include #include #include +#include #if defined(_WIN32) // Keep the transitive Windows headers from defining the min/max macros: the @@ -77,6 +78,25 @@ inline std::string homeDirectory() #endif } +/** + * Expand a leading "~" to the user's home directory. + * + * Only a leading separator-less tilde is a home directory; a tilde elsewhere is + * a literal character. Windows short names such as "C:\Users\RUNNER~1\..." + * contain one, and replacing it unconditionally turned that path into garbage. + * @param path a path that may start with "~" + * @return the path with a leading "~" replaced by the home directory, unchanged + * otherwise (or when the home directory cannot be resolved) + */ +inline std::string expandHomeDirectory(std::string_view path) +{ + if (path.empty() || path.front() != '~') { return std::string(path); } + if (path.size() > 1 && path[1] != '/' && path[1] != '\\') { return std::string(path); } + const std::string home = homeDirectory(); + if (home.empty()) { return std::string(path); } + return home + std::string(path.substr(1)); +} + /** @return the character that separates entries in the PATH environment variable */ inline constexpr char pathListSeparator() noexcept { diff --git a/src/include/lintel/core/services/ISharedMemoryService.h b/src/include/lintel/core/services/ISharedMemoryService.h index bb21dd95..92401d03 100644 --- a/src/include/lintel/core/services/ISharedMemoryService.h +++ b/src/include/lintel/core/services/ISharedMemoryService.h @@ -7,6 +7,7 @@ namespace lintel { class SharedMemorySegment; class SharedMemorySegmentInfo; +struct SharedMemorySegmentStats; class ISharedMemoryService { @@ -15,6 +16,15 @@ class ISharedMemoryService virtual SharedMemorySegmentInfo showStateOf(const std::shared_ptr &segment) const = 0; + /** + * Collect the runtime statistics of a segment. + * + * @param segment the segment to inspect + * @return size, capacity, free bytes, lifecycle counters and generation + * @throws ShmSegmentNotFound if the service does not map the segment + */ + [[nodiscard]] virtual SharedMemorySegmentStats statsOf(const std::shared_ptr &segment) const = 0; + virtual void growOf(const std::shared_ptr &segment, std::size_t grow) = 0; virtual void shrinkOf(const std::shared_ptr &segment) = 0; @@ -24,9 +34,9 @@ class ISharedMemoryService * * A process that crashes while holding a segment's remap semaphore leaves it * stuck, which would make every later acquirer wait for the bounded timeout - * and then fail. Called automatically when a segment is opened, so a segment - * that gets stuck later on is recovered at its next open. It never touches a - * lock whose holder is still alive. + * and then fail. Opening a segment runs the same check once; this method + * repeats it for a segment that got stuck later on. It never touches a lock + * whose holder is still alive. * * @return the number of locks that were reclaimed */ diff --git a/src/include/lintel/core/services/LoggerService.h b/src/include/lintel/core/services/LoggerService.h index ddc73d66..6b6984fb 100644 --- a/src/include/lintel/core/services/LoggerService.h +++ b/src/include/lintel/core/services/LoggerService.h @@ -17,6 +17,7 @@ #include #include #include +#include #include #include @@ -37,17 +38,6 @@ namespace lintel { */ enum class LogLevel : std::uint8_t { trace = 0, debug = 1, info = 2, warn = 3, error = 4, critical = 5, off = 6 }; -/** Source location function name for the LOG_* macros, equivalent to spdlog's SPDLOG_FUNCTION. */ -#ifndef LINTEL_FUNCTION -#if defined(_MSC_VER) -#define LINTEL_FUNCTION __FUNCSIG__ -#elif defined(__GNUC__) || defined(__clang__) -#define LINTEL_FUNCTION __PRETTY_FUNCTION__ -#else -#define LINTEL_FUNCTION __func__ -#endif -#endif - /** Singleton service providing spdlog-based logging for the application. * * JSON output (`json="true"` on `` in the bootstrap XML) formats every @@ -75,12 +65,7 @@ class LoggerService * Non-template formatting backend: filters the level and performs all fmt formatting. * spdlog is touched here only. Defined in LoggerService.cpp. */ - void logImpl(const char *file, - int line, - const char *function, - LogLevel lvl, - std::string_view message, - std::format_args args); + void logImpl(const std::source_location &location, LogLevel lvl, std::string_view message, std::format_args args); static std::unique_ptr m_instance; static std::mutex m_instanceMutex; @@ -121,16 +106,14 @@ class LoggerService * Template-eliminated: boxes the arguments (std::make_format_args) and defers all formatting * to the non-template logImpl() in LoggerService.cpp. * @tparam Args the argument types - * @param file the source file name - * @param line the source line number - * @param function the source function name + * @param location the source location of the log statement * @param lvl the log level * @param message the format string * @param args the format arguments */ template - void log(const char *file, int line, const char *function, LogLevel lvl, std::string_view message, Args &&...args) + void log(const std::source_location &location, LogLevel lvl, std::string_view message, Args &&...args) { - logImpl(file, line, function, lvl, message, std::make_format_args(args...)); + logImpl(location, lvl, message, std::make_format_args(args...)); } /** Change the log level at runtime. @@ -199,17 +182,16 @@ class RequestIdGuard #define DECLARE_LOGGER(processName, configuration) lintel::LoggerService::getOrCreate(processName, configuration) #define LOG_INFO(message, ...) \ - lintel::LoggerService::get().log(__FILE__, __LINE__, LINTEL_FUNCTION, lintel::LogLevel::info, message, ##__VA_ARGS__) + lintel::LoggerService::get().log(std::source_location::current(), lintel::LogLevel::info, message, ##__VA_ARGS__) #define LOG_DEBUG(message, ...) \ - lintel::LoggerService::get().log(__FILE__, __LINE__, LINTEL_FUNCTION, lintel::LogLevel::debug, message, ##__VA_ARGS__) + lintel::LoggerService::get().log(std::source_location::current(), lintel::LogLevel::debug, message, ##__VA_ARGS__) #define LOG_TRACE(message, ...) \ - lintel::LoggerService::get().log(__FILE__, __LINE__, LINTEL_FUNCTION, lintel::LogLevel::trace, message, ##__VA_ARGS__) + lintel::LoggerService::get().log(std::source_location::current(), lintel::LogLevel::trace, message, ##__VA_ARGS__) #define LOG_ERROR(message, ...) \ - lintel::LoggerService::get().log(__FILE__, __LINE__, LINTEL_FUNCTION, lintel::LogLevel::error, message, ##__VA_ARGS__) -#define LOG_FATAL(message, ...) \ - lintel::LoggerService::get().log( \ - __FILE__, __LINE__, LINTEL_FUNCTION, lintel::LogLevel::critical, message, ##__VA_ARGS__) + lintel::LoggerService::get().log(std::source_location::current(), lintel::LogLevel::error, message, ##__VA_ARGS__) +#define LOG_FATAL(message, ...) \ + lintel::LoggerService::get().log(std::source_location::current(), lintel::LogLevel::critical, message, ##__VA_ARGS__) #define LOG_WARN(message, ...) \ - lintel::LoggerService::get().log(__FILE__, __LINE__, LINTEL_FUNCTION, lintel::LogLevel::warn, message, ##__VA_ARGS__) + lintel::LoggerService::get().log(std::source_location::current(), lintel::LogLevel::warn, message, ##__VA_ARGS__) }// namespace lintel #endif// LOGGING_LOGGER_H diff --git a/src/include/lintel/core/services/PropertyRegistration.h b/src/include/lintel/core/services/PropertyRegistration.h index 570452c6..0319c468 100644 --- a/src/include/lintel/core/services/PropertyRegistration.h +++ b/src/include/lintel/core/services/PropertyRegistration.h @@ -1,8 +1,8 @@ #ifndef PROPERTYREGISTRATION_H #define PROPERTYREGISTRATION_H -#include #include +#include #include "lintel/core/models/Property.h" #include "lintel/core/services/AbstractService.h" @@ -19,16 +19,14 @@ template class PropertyRegistration : public AbstractService * @param defaultValue the default value * @param description a human-readable description * @param runtime whether the property can be changed at runtime - * @param fileName the source file name for data storage - * @param position the position within the file for data storage + * @param location the source location of the registration (file and line) for data storage * @return a shared pointer to the registered property */ template std::shared_ptr> registerProperty(std::string name, type defaultValue, std::string description, bool runtime, - const std::string &fileName, - int32_t position) + const std::source_location &location = std::source_location::current()) { std::shared_ptr> property = std::make_shared>(name, this->getInstanceName(), @@ -38,10 +36,10 @@ template class PropertyRegistration : public AbstractService description, runtime); LOG_INFO("Property<{}> {} = {}", type_name(), name, property->toString()); - const std::filesystem::path &path(fileName); + const std::filesystem::path path(location.file_name()); - property->setDataStorage( - DataStorage(PropertyRepositoryType::DEFAULT, path.filename().generic_string() + ":" + std::to_string(position))); + property->setDataStorage(DataStorage( + PropertyRepositoryType::DEFAULT, path.filename().generic_string() + ":" + std::to_string(location.line()))); this->m_properties.push_back(property); return property; } diff --git a/src/include/lintel/core/shm/BoostSegmentAllocator.h b/src/include/lintel/core/shm/BoostSegmentAllocator.h index 785e8e4b..b8247f85 100644 --- a/src/include/lintel/core/shm/BoostSegmentAllocator.h +++ b/src/include/lintel/core/shm/BoostSegmentAllocator.h @@ -3,6 +3,8 @@ #include +#include + #include "lintel/core/shm/ShmSegmentAccessor.h" namespace lintel { @@ -27,6 +29,25 @@ class BoostSegmentAllocator final : public ShmSegmentAccessor::Allocator void deallocate(void *pointer) override { m_segment->deallocate(pointer); } + /** + * Find or construct a named byte array of the requested size. + * + * `find_or_construct(name)[count]()` is the array form of the boost + * named construct: it allocates `count` elements of `char` and value + * initialises them, so a freshly created array is zeroed. The lookup and the + * construction run under the segment manager's cross-process mutex, so a + * caller in another process either finds the finished array or creates it + * itself - never both. + * + * @param name name of the array inside the segment + * @param bytes size of the array in bytes + * @return pointer to the array + */ + void *findOrConstructArray(const char *name, std::size_t bytes) override + { + return m_segment->find_or_construct(name)[bytes](); + } + private: boost::interprocess::managed_mapped_file::segment_manager *m_segment; }; diff --git a/src/include/lintel/core/shm/SegmentLayoutPolicy.h b/src/include/lintel/core/shm/SegmentLayoutPolicy.h new file mode 100644 index 00000000..b67f7986 --- /dev/null +++ b/src/include/lintel/core/shm/SegmentLayoutPolicy.h @@ -0,0 +1,121 @@ +#ifndef LINTEL_SEGMENTLAYOUTPOLICY_H +#define LINTEL_SEGMENTLAYOUTPOLICY_H + +#include +#include +#include +#include +#include + +namespace lintel { +namespace shm { + + /** What a process does when it opens a segment written with another object layout. */ + enum class LayoutMismatchPolicy : std::uint8_t { + /** Refuse to start and name the file - the default, because only the operator + * knows whether the segment still holds something that is needed. */ + Fail, + /** Discard the contents and start with an empty segment. */ + Recreate, + /** Ask the registered migrator first; it decides whether the contents may be + * discarded. */ + Migrate, + }; + + /** + * Migration hook for a stale shared memory segment. + * + * Called with the path of the segment **as it currently is on disk**, before + * anything is removed, plus the layout version that was found and the version + * this build writes. The migrator carries whatever it needs into a durable + * place (a file, the database, a new named object) and returns true once the + * old contents are expendable; returning false aborts the start. + * + * Typical use: export the payloads of the typed repositories + * (`shm::SharedMemoryRepository`), which are the only objects in a segment + * that hold data a program may not be able to recompute. + */ + using SegmentMigrator = std::function< + bool(const std::filesystem::path &staleSegment, std::uint64_t foundVersion, std::uint64_t currentVersion)>; + + /** + * Process wide layout policy for shared memory segments. + * + * Set it once during startup, before the `SharedMemoryService` is constructed: + * + * @code + * lintel::shm::SegmentLayoutPolicy::setMigrator([](const auto &path, auto from, auto to) { + * // read the old segment, write what matters somewhere durable + * return true; + * }); + * @endcode + * + * The default is LayoutMismatchPolicy::Fail, so a process never discards data + * on its own. `LINTEL_SHM_RECREATE=1` in the environment selects Recreate + * without recompiling, which is what a developer machine usually wants. + * + * Not synchronized: like MessageQueueBackendConfig this is startup wiring, not + * a runtime switch. Call it before the container starts. + */ + class SegmentLayoutPolicy + { + public: + /** Select the policy explicitly. + * + * @param policy what to do on a layout mismatch. Migrate without a registered + * migrator behaves like Fail. */ + static void set(LayoutMismatchPolicy policy) noexcept { state().policy = policy; } + + /** Register a migrator and select LayoutMismatchPolicy::Migrate. + * + * @param migrator the hook; an empty function selects Fail instead */ + static void setMigrator(SegmentMigrator migrator) + { + state().migrator = std::move(migrator); + state().policy = state().migrator != nullptr ? LayoutMismatchPolicy::Migrate : LayoutMismatchPolicy::Fail; + } + + /** @return the configured policy */ + [[nodiscard]] static LayoutMismatchPolicy policy() noexcept { return state().policy; } + + /** @return true when a migrator is registered */ + [[nodiscard]] static bool hasMigrator() { return static_cast(state().migrator); } + + /** + * Run the registered migrator. + * + * @param staleSegment the segment file as it is on disk, still untouched + * @param foundVersion layout version found in the segment + * @param currentVersion layout version this build writes + * @return true when the old contents may be discarded; false to abort the start + */ + static bool + migrate(const std::filesystem::path &staleSegment, std::uint64_t foundVersion, std::uint64_t currentVersion) + { + if (!state().migrator) { return false; } + return state().migrator(staleSegment, foundVersion, currentVersion); + } + + /** Restore the defaults. Intended for tests. */ + static void reset() noexcept { state() = State{}; } + + private: + /** Process wide policy state. */ + struct State + { + LayoutMismatchPolicy policy = LayoutMismatchPolicy::Fail; + SegmentMigrator migrator; + }; + + /** @return the policy of this process; a function local static is shared by + * every translation unit */ + static State &state() + { + static State instance; + return instance; + } + }; + +}// namespace shm +}// namespace lintel +#endif// LINTEL_SEGMENTLAYOUTPOLICY_H diff --git a/src/include/lintel/core/shm/SharedMemoryRepository.h b/src/include/lintel/core/shm/SharedMemoryRepository.h index 6686ae40..d47e6a71 100644 --- a/src/include/lintel/core/shm/SharedMemoryRepository.h +++ b/src/include/lintel/core/shm/SharedMemoryRepository.h @@ -4,14 +4,17 @@ #include #include #include +#include #include #include #include #include +#include "lintel/core/models/SharedMemorySegment.h" +#include "lintel/core/shm/SharedMemoryRepositoryRegistry.h" + namespace lintel { -class SharedMemorySegment; class SharedMemoryService; /** @@ -195,6 +198,44 @@ namespace shm { SharedMemoryRepository(const SharedMemoryRepository &) = delete; SharedMemoryRepository &operator=(const SharedMemoryRepository &) = delete; + /** Bind to the named cell, reusing the instance this process already holds. + * + * The registry key is `:`, so a second call with + * the same segment and cell name returns the instance created first: the + * segment keeps one cell and one payload instead of a second binding - and + * a second ShmCell - to the same name. The registry keeps the returned + * repository alive; remove() it from SharedMemoryRepositoryRegistry to + * release it again. + * + * @param service shared memory service of the segment + * @param segment segment the object lives in + * @param name name of the object inside the segment + * @param args constructor arguments, used only when the cell is created + * @return the repository bound to the cell + * @throws std::invalid_argument on invalid arguments, or when the key is + * already registered with another payload type + * @throws ShmSegmentNotFound when the service does not know the segment */ + template + static std::shared_ptr> of(std::shared_ptr service, + std::shared_ptr segment, + std::string name, + Args &&...args) + { + if (service == nullptr || segment == nullptr) { + throw std::invalid_argument("shared memory repository factory needs a service and a segment"); + } + const std::string key = segment->getName() + ":" + name; + SharedMemoryRepositoryRegistry ®istry = SharedMemoryRepositoryRegistry::instance(); + // Throws std::invalid_argument when the key holds another payload type: + // handing that instance out would fail somewhere inside get()/set(). + std::shared_ptr> existing = registry.getAs>(key); + if (existing != nullptr) { return existing; } + std::shared_ptr> repository = std::make_shared>( + std::move(service), std::move(segment), std::move(name), std::forward(args)...); + registry.registerRepository(key, repository); + return repository; + } + /** @return const reference to the object in shared memory; the reference * is re-resolved when the segment was remapped */ [[nodiscard]] const T &get() const { return *static_cast(m_cell.resolve()); } diff --git a/src/include/lintel/features/base/services/SharedMemoryRepositoryRegistry.h b/src/include/lintel/core/shm/SharedMemoryRepositoryRegistry.h similarity index 100% rename from src/include/lintel/features/base/services/SharedMemoryRepositoryRegistry.h rename to src/include/lintel/core/shm/SharedMemoryRepositoryRegistry.h diff --git a/src/include/lintel/core/shm/ShmSegmentAccessor.h b/src/include/lintel/core/shm/ShmSegmentAccessor.h index 2c8c92a7..de5509e8 100644 --- a/src/include/lintel/core/shm/ShmSegmentAccessor.h +++ b/src/include/lintel/core/shm/ShmSegmentAccessor.h @@ -2,24 +2,47 @@ #define LINTEL_SHMSEGMENTACCESSOR_H #include +#include +#include +#include +#include +#include +#include +#include namespace lintel { /** * Typed access to the allocator of a shared memory segment. * - * The accessor is a small value type (one pointer) that keeps the boost - * interprocess headers out of EventBus.h: objects stored in shared memory hold - * a copy of the accessor, while the process local allocator it points to is - * owned by SharedMemoryService. Only the raw block allocation itself is - * untyped, everything above it stays typed. + * The accessor is a small value type that keeps the boost interprocess headers + * out of EventBus.h: objects stored in shared memory hold a copy of the + * accessor, while the process local allocator it points to is owned by + * SharedMemoryService. Only the raw block allocation itself is untyped, + * everything above it stays typed. * * Copying is intentional: the copy lives inside the shared memory object and - * therefore keeps a stable address for the whole lifetime of that object, - * independent of the process that created it. + * therefore keeps a stable address for the whole lifetime of that object. + * + * A pointer alone cannot survive the process boundary: the copy inside the + * segment would point at the heap of the process that created the object, and + * that address means nothing in any other process (it may not even be mapped + * there). The accessor therefore carries the *name* of its segment, which is + * stable across processes, and resolves that name to a process local allocator + * on every allocation. `bind()` is what fills the process local side; it is + * called by SharedMemoryService whenever it hands out an accessor. + * + * An accessor constructed without a name (the single argument constructor) + * keeps working on the allocator it was built with. That is the right thing + * for objects that never leave the process - a segment used as a plain buffer, + * tests and benchmarks - and it is what the named form falls back to when the + * name was never bound in this process. */ class ShmSegmentAccessor { public: + /** Capacity of the stored segment name in bytes, including the terminating NUL. */ + static constexpr std::size_t NAME_SIZE = 32; + /** Process local allocator an accessor forwards to. */ class Allocator { @@ -36,24 +59,147 @@ class ShmSegmentAccessor * @param pointer block previously returned by allocate */ virtual void deallocate(void *pointer) = 0; + + /** + * Find or construct a named byte array inside the segment. + * + * The name is the cross-process identity of the object: the first caller + * creates the array, every later caller - in this or in any other process + * - gets the same bytes back. A fresh array is value initialised, so it + * starts zeroed. + * + * The default implementation does not support named objects and reports + * that with nullptr, which keeps the allocator interface usable for + * segment stand-ins that only need allocate/deallocate. + * + * @param name name of the array inside the segment + * @param bytes size of the array in bytes (must be positive) + * @return pointer to the array, or nullptr when named objects are not supported + */ + virtual void *findOrConstructArray(const char * /*name*/, std::size_t /*bytes*/) { return nullptr; } }; - /** @param allocator process local allocator of the segment */ + /** + * Construct an accessor that is bound to its allocator only, without a + * segment name. The resulting accessor is process local: a copy stored in + * the segment is not usable from another process. + * + * @param allocator process local allocator of the segment + */ explicit ShmSegmentAccessor(Allocator &allocator) noexcept : m_allocator(&allocator) {} + /** + * Construct an accessor that identifies its segment by name. + * + * The name is stored in the accessor (and therefore in every copy that ends + * up inside the segment) so that another process can resolve it to its own + * allocator. The name is not bound here; call bind() for that, which + * SharedMemoryService does before it hands the accessor out. + * + * @param segmentName name of the shared memory segment + * @param allocator process local allocator of that segment + * @throws std::invalid_argument if the name does not fit into NAME_SIZE bytes + */ + ShmSegmentAccessor(std::string_view segmentName, Allocator &allocator) : m_allocator(&allocator) + { + validateName(segmentName); + std::memcpy(m_segmentName, segmentName.data(), segmentName.size()); + m_segmentName[segmentName.size()] = '\0'; + } + + /** + * Bind a segment name to the allocator of this process. + * + * Every accessor that carries this name resolves it to @p allocator, no + * matter which process created the accessor or how old the copy is. Binding + * again replaces the previous allocator, which is what a remapped segment + * needs: the old allocator belongs to the closed mapping. + * + * The registry lives for the whole process; @p allocator has to outlive + * every accessor that is used in this process, exactly like the allocator of + * an unnamed accessor. + * + * @param segmentName name of the shared memory segment + * @param allocator this process' allocator of that segment + * @throws std::invalid_argument if the name does not fit into NAME_SIZE bytes + */ + static void bind(std::string_view segmentName, Allocator &allocator) + { + validateName(segmentName); + std::lock_guard lock(registry().mutex); + registry().allocators.insert_or_assign(std::string(segmentName), &allocator); + } + /** * @param bytes to allocate from the segment * @return pointer to the allocated block */ - [[nodiscard]] void *allocate(std::size_t bytes) const { return m_allocator->allocate(bytes); } + [[nodiscard]] void *allocate(std::size_t bytes) const { return allocator()->allocate(bytes); } /** * @param pointer block previously returned by allocate */ - void deallocate(void *pointer) const { m_allocator->deallocate(pointer); } + void deallocate(void *pointer) const { allocator()->deallocate(pointer); } + + /** + * Find or construct a named byte array in the segment of this accessor. + * + * @param name name of the array inside the segment + * @param bytes size of the array in bytes + * @return pointer to the array, or nullptr when the allocator does not support named objects + */ + [[nodiscard]] void *findOrConstructArray(const char *name, std::size_t bytes) const + { + return allocator()->findOrConstructArray(name, bytes); + } + + /** @return the segment name this accessor identifies its segment by; empty for an unnamed accessor */ + [[nodiscard]] std::string_view segmentName() const noexcept { return std::string_view(m_segmentName); } private: + /** Process wide name -> allocator map, see bind(). */ + struct Registry + { + std::mutex mutex; + /** std::less<> makes the lookup accept a string_view without building a temporary string. */ + std::map> allocators; + }; + + /** @return the registry of this process; a function local static is shared by every translation unit */ + static Registry ®istry() + { + static Registry instance; + return instance; + } + + /** @param segmentName name to check against NAME_SIZE + * @throws std::invalid_argument if the name does not fit, including its NUL */ + static void validateName(std::string_view segmentName) + { + if (segmentName.size() < NAME_SIZE) { return; } + throw std::invalid_argument("shared memory segment name '" + std::string(segmentName) + + "' is too long: a ShmSegmentAccessor stores at most " + std::to_string(NAME_SIZE - 1) + + " characters"); + } + + /** + * Allocator to forward to: the one bound to this accessor's segment name in + * this process, otherwise the one captured at construction time. + * + * @return allocator of this process + */ + [[nodiscard]] Allocator *allocator() const + { + if (m_segmentName[0] == '\0') { return m_allocator; } + std::lock_guard lock(registry().mutex); + const auto found = registry().allocators.find(std::string_view(m_segmentName)); + if (found == registry().allocators.end()) { return m_allocator; } + return found->second; + } + Allocator *m_allocator; + /** Name of the segment this accessor belongs to; empty for an unnamed accessor. */ + char m_segmentName[NAME_SIZE]{}; }; }// namespace lintel diff --git a/src/include/lintel/core/shm/containers/QueueShm.h b/src/include/lintel/core/shm/containers/QueueShm.h index e0f3c1fb..9cba2314 100644 --- a/src/include/lintel/core/shm/containers/QueueShm.h +++ b/src/include/lintel/core/shm/containers/QueueShm.h @@ -22,7 +22,8 @@ namespace lintel { * the segment, later processes find it again and use it through their own * accessor-backed view of the same bytes. * - * Not copyable (RingBufferShm is not, see there for the reason). + * Not movable and not copyable (RingBufferShm is neither, see there for the + * reason: its slot array is addressed relative to the ring object). * * Requirements on T are RingBufferShm's: default constructible, copy * assignable, copy constructible, identical in every sharing process. diff --git a/src/include/lintel/core/shm/containers/RingBufferShm.h b/src/include/lintel/core/shm/containers/RingBufferShm.h index 85cdc5e1..0205981e 100644 --- a/src/include/lintel/core/shm/containers/RingBufferShm.h +++ b/src/include/lintel/core/shm/containers/RingBufferShm.h @@ -4,6 +4,7 @@ #include "lintel/core/shm/ShmSegmentAccessor.h" #include +#include #include #include #include @@ -25,18 +26,21 @@ namespace lintel { * Storage layout: the ring object itself (head, tail, bookkeeping) lives * wherever the ring is constructed - typically as the payload of a * SharedMemoryRepository inside a segment - while the slot array is allocated - * separately from the segment through ShmSegmentAccessor and addressed via a - * byte offset relative to the ring object (same pattern as - * EventBus::ShmRing). The relative layout is identical in every mapping of - * the segment, so the queue works from every process regardless of the - * virtual address it mapped the segment at. + * from the same segment through ShmSegmentAccessor. The slot buffer is + * addressed relative to `this`: m_bufferOffset stores the distance from the + * ring object to the aligned slot array (same pattern as + * EventBus::ShmRing). Only that offset is part of the shared state, no + * pointer, so the relative layout is identical in every mapping of the + * segment and the ring works from every process regardless of the virtual + * address it mapped the segment at. * * The ring is usable both as a shared memory object (constructed once by the * first process, found by the others) and as a process local object on the * stack in tests - the accessor decides where the slots come from. * * Not movable and not copyable: the slot buffer is addressed relative to - * `this`, so moving the ring would invalidate m_bufferOffset. + * `this`, so moving or copying the ring would leave m_bufferOffset pointing + * at the slots of the original object. * * Requirements on T: default constructible (slot array initialization), * copy assignable (push) and copy constructible (pop) - in other words @@ -87,7 +91,7 @@ template class RingBufferShm } aligned = static_cast(alignedRaw); } - m_buffer = aligned; + m_bufferOffset = reinterpret_cast(aligned) - reinterpret_cast(this); Slot *slots = buffer(); for (std::size_t i = 0; i < capacity; ++i) { new (&slots[i]) Slot(); @@ -207,16 +211,33 @@ template class RingBufferShm std::size_t m_capacity = 0; ShmSegmentAccessor m_segment; /** First byte of the slot array allocation; nullptr when construction of - * the array failed (the ring then never gets used). */ + * the array failed (the ring then never gets used). A process local + * pointer, only used to release the block again - never dereferenced + * through the shared ring. */ std::byte *m_allocation = nullptr; - /** Aligned base of the slot array in this process' mapping. A plain byte - * pointer, so addressing needs neither an integer-to-pointer cast nor a - * byte count scaled by a wider element type. */ - std::byte *m_buffer = nullptr; + /** Distance from this object to the aligned slot array. The relative + * offset is what makes the ring usable from another mapping of the same + * segment: every process relocates the ring and the slots by the same + * amount, so this value stays valid while a pointer would not. */ + std::intptr_t m_bufferOffset = 0; - [[nodiscard]] Slot *buffer() { return reinterpret_cast(m_buffer); } + [[nodiscard]] Slot *buffer() + { + // The slot array lives in the shared segment, not inside this object (see the class + // documentation), and is addressed by the offset m_bufferOffset. The aligned buffer is not + // necessarily a multiple of sizeof(Slot), so element arithmetic is not an option; the address + // is therefore materialised in the integer domain. std::bit_cast restores it: a plain + // reinterpret_cast from the integer would be reported as performance-no-int-to-ptr, byte + // pointer arithmetic as cpp/suspicious-pointer-scaling. + const std::uintptr_t address = reinterpret_cast(this) + static_cast(m_bufferOffset); + return std::bit_cast(address); + } - [[nodiscard]] const Slot *buffer() const { return reinterpret_cast(m_buffer); } + [[nodiscard]] const Slot *buffer() const + { + const std::uintptr_t address = reinterpret_cast(this) + static_cast(m_bufferOffset); + return std::bit_cast(address); + } }; }// namespace lintel diff --git a/src/include/lintel/features/base/controller/EventBusDtos.h b/src/include/lintel/features/base/controller/EventBusDtos.h new file mode 100644 index 00000000..95f22e49 --- /dev/null +++ b/src/include/lintel/features/base/controller/EventBusDtos.h @@ -0,0 +1,138 @@ +#ifndef LINTEL_EVENTBUSDTOS_H +#define LINTEL_EVENTBUSDTOS_H + +#include +#include +#include +#include +#include + +#include "lintel/core/models/JsonSerializable.h" +#include "lintel/features/base/events/EventBus.h" + +namespace lintel { +/** Runtime counters of an event bus, read while the bus is open. */ +struct EventBusCounters +{ + /** Number of events dropped because a queue was full */ + std::uint64_t dropped = 0; + /** Number of registered topics */ + std::size_t topics = 0; + /** Number of reserved subscriber slots */ + std::size_t subscribers = 0; + /** Best-effort depth of the point-to-point queue */ + std::size_t busDepth = 0; + /** Last assigned sequence number */ + std::int64_t sequence = 0; +}; + +/** + * DTO representing one configured event bus. + * + * The configuration is optional: a bus whose bootstrap configuration is + * invalid is still listed, but without any configuration fields. The runtime + * counters are only present while the bus could be opened in its shared memory + * segment (AVAILABLE), because they have to be read from the bus itself. + */ +class EventBusDto : public JsonSerializable +{ +private: + /** The bus name */ + std::string m_name; + /** The validated configuration, empty if the entry is invalid */ + std::optional m_config; + /** True if the bus could be opened in its shared memory segment */ + bool m_available = false; + /** Number of events dropped because a queue was full */ + std::uint64_t m_dropped = 0; + /** Number of registered topics */ + std::size_t m_topics = 0; + /** Number of reserved subscriber slots */ + std::size_t m_subscribers = 0; + /** Best-effort depth of the point-to-point queue */ + std::size_t m_busDepth = 0; + /** Last assigned sequence number */ + std::int64_t m_sequence = 0; + +public: + /** Default constructor */ + EventBusDto() = default; + + /** Construct the DTO of a configured bus + * @param name the bus name + * @param config the validated bus configuration */ + EventBusDto(std::string name, const EventBusConfig &config); + + /** Construct the DTO of a bus without a usable configuration + * @param name the bus name */ + explicit EventBusDto(std::string name); + + /** Store the runtime counters read from the bus. + * @param counters the counters read while the bus was open */ + void markAvailable(const EventBusCounters &counters); + + /** @return the bus name */ + [[nodiscard]] const std::string &getName() const; + + /** @return the configuration, empty if the entry is invalid */ + [[nodiscard]] const std::optional &getConfig() const; + + /** @return true if the bus could be opened */ + [[nodiscard]] bool isAvailable() const; + + /** @return the number of dropped events */ + [[nodiscard]] std::uint64_t getDropped() const; + + /** @return the number of registered topics */ + [[nodiscard]] std::size_t getTopics() const; + + /** @return the number of reserved subscriber slots */ + [[nodiscard]] std::size_t getSubscribers() const; + + /** @return the depth of the point-to-point queue */ + [[nodiscard]] std::size_t getBusDepth() const; + + /** @return the last assigned sequence number */ + [[nodiscard]] std::int64_t getSequence() const; + + using JsonSerializable::deserialize; + + /** Serialize to JSON + * @param writer the rapidjson writer */ + void serialize(rapidjson::Writer *writer) const override; + + /** Deserialize from JSON + * @param obj the JSON value + * @return true on success */ + bool deserialize(const rapidjson::Value &obj) override; +}; + +/** DTO representing the collection of all configured event buses */ +class EventBusesDto : public JsonSerializable +{ +private: + /** The bus DTOs */ + std::vector m_eventBusDtos; + +public: + /** Default constructor */ + EventBusesDto() = default; + + /** Construct from existing bus DTOs + * @param eventBusDtos the bus DTOs */ + explicit EventBusesDto(std::vector eventBusDtos); + + /** @return the bus DTOs */ + [[nodiscard]] const std::vector &getEventBusDtos() const; + + /** Serialize to JSON + * @param writer the rapidjson writer */ + void serialize(rapidjson::Writer *writer) const override; + + /** Deserialize from JSON string + * @param json the JSON array to parse */ + void deserialize(const std::string &json) override; +}; + +}// namespace lintel +#endif// LINTEL_EVENTBUSDTOS_H diff --git a/src/include/lintel/features/base/msg/MessageQueueShmCore.h b/src/include/lintel/features/base/msg/MessageQueueShmCore.h index 6ba09c1e..377ea39f 100644 --- a/src/include/lintel/features/base/msg/MessageQueueShmCore.h +++ b/src/include/lintel/features/base/msg/MessageQueueShmCore.h @@ -25,12 +25,21 @@ namespace lintel { * allocation is involved per send/receive; the bytes stay in the shared * segment (hence "zero-copy" in the sense of the SHM storage model). * - * Instances of one process that use the same queue name share the same ring: + * The queue is shared across processes: a header that describes the ring + * layout is stored in the segment under a name derived from the queue name, so + * the second process finds the ring the first one created instead of building + * one of its own. The segment accessor therefore has to be a *named* one (see + * SharedMemoryService::getSegmentAccessor) - an unnamed accessor is bound to a + * single process and is rejected. + * + * Instances of one process that use the same queue name share the same state: * a queue created by one core is found by every later core until the last one * is destroyed (or removeOf() is called). Queue names are process wide unique, - * matching the naming of the Boost backend. The implementation keeps the - * ring-offset bookkeeping in the segment and only the process local registry - * and wait coordination outside of it. + * matching the naming of the Boost backend. + * + * Waiters poll in slices of about a millisecond: the condition variable only + * reaches the threads of one process, and a producer in another process cannot + * signal it (see QueueState::waitForProgress). */ class MessageQueueShmCore { @@ -42,12 +51,19 @@ class MessageQueueShmCore /** * open or create the shared memory message queue * - * @param name of the message queue. No filepath just name. + * The queue is looked up in the segment by name: a core created by another + * process is found and adopted, including its layout. The ring is destroyed + * by the process that created it. + * + * @param name of the message queue. No filepath just name; at most 21 + * characters, because the segment object name adds the "lintel_mq_" + * prefix and is limited to 32 characters including its NUL. * @param msgCount capacity of the queue in messages (must be positive) * @param msgSize maximum size of a single message in bytes (must be positive) - * @param segment accessor of the shared memory segment holding the queue - * @throws std::invalid_argument on invalid size/capacity or when an existing - * queue with the same name uses a different layout + * @param segment named accessor of the shared memory segment holding the queue + * @throws std::invalid_argument on invalid size/capacity, on an unnamed + * segment accessor, on a queue name that is too long, or when an + * existing queue with the same name uses a different layout */ MessageQueueShmCore(const std::string &name, int32_t msgCount, std::size_t msgSize, ShmSegmentAccessor segment); @@ -61,6 +77,10 @@ class MessageQueueShmCore * remove of message queue (drops the process local association; the segment * itself owns the memory) * + * The ring stays in the segment as long as a core of the creating process is + * alive; that core marks the header uninitialised when it goes away, so a + * later open re-creates the ring instead of adopting a destroyed one. + * * @param messageQueueName name of the queue to remove */ static void removeOf(const std::string &messageQueueName); @@ -68,6 +88,11 @@ class MessageQueueShmCore /** * Look up an existing queue without creating it and read its message count. * + * The lookup is process local: it answers for the cores this process has + * open. A queue that another process created and this one never opened is + * reported as 0, because the count lives behind an accessor this call does + * not have. + * * @param messageQueueName name of the queue to inspect * @return number of messages, 0 if the queue does not exist */ diff --git a/src/include/lintel/features/base/services/EventBusService.h b/src/include/lintel/features/base/services/EventBusService.h index 93e1dda6..55844edf 100644 --- a/src/include/lintel/features/base/services/EventBusService.h +++ b/src/include/lintel/features/base/services/EventBusService.h @@ -1,9 +1,11 @@ #ifndef LINTEL_EVENTBUSSERVICE_H #define LINTEL_EVENTBUSSERVICE_H +#include #include #include #include +#include #include "lintel/core/configuration/Configuration.h" #include "lintel/core/models/ProcessName.h" @@ -39,7 +41,13 @@ class EventBusService : public AbstractService * @param entry the configuration entry * @return the validated bus configuration * @throws std::runtime_error if the configuration is invalid */ - EventBusConfig configOf(const std::shared_ptr &entry); + static EventBusConfig configOfEntry(const std::shared_ptr &entry); + + /** Look up a configured bus entry. + * @param name the bus name + * @return the configuration entry + * @throws std::runtime_error if the bus is not configured */ + [[nodiscard]] const std::shared_ptr &entryOf(const std::string &name) const; /** Resolve the EventBus of an entry; re-resolves after a segment remap. * @param entry the configuration entry @@ -63,6 +71,37 @@ class EventBusService : public AbstractService * @throws std::runtime_error if the bus is not configured */ std::shared_ptr of(const std::string &name); + /** Names of all configured event buses. + * @return the bus names in ascending order */ + [[nodiscard]] std::vector namesOf() const; + + /** Runtime configuration of a configured event bus. + * + * The configuration is read from the bootstrap configuration and validated; + * the bus itself is not resolved, so a bus whose segment is missing still + * reports its configuration. + * @param name the bus name + * @return the validated bus configuration + * @throws std::runtime_error if the bus is not configured or its configuration is invalid */ + [[nodiscard]] EventBusConfig configOf(const std::string &name) const; + + /** Best-effort depth of the point-to-point queue of a bus. + * + * IEventBus does not expose the queue depth, so the view is bypassed here. + * @param name the bus name + * @return the number of queued events + * @throws std::runtime_error if the bus is not configured or cannot be resolved */ + [[nodiscard]] std::size_t busQueueDepthOf(const std::string &name); + + /** Best-effort depth of one subscriber queue of a bus. + * + * IEventBus does not expose the queue depth, so the view is bypassed here. + * @param name the bus name + * @param subscriber the subscriber name + * @return the number of queued events, 0 for an unknown subscriber + * @throws std::runtime_error if the bus is not configured or cannot be resolved */ + [[nodiscard]] std::size_t subscriberQueueDepthOf(const std::string &name, const std::string &subscriber); + /** Construct all configured event buses in their segments. */ void onInitialize() override; diff --git a/src/include/lintel/features/base/services/ProcessService.h b/src/include/lintel/features/base/services/ProcessService.h index 39c618ed..5a144559 100644 --- a/src/include/lintel/features/base/services/ProcessService.h +++ b/src/include/lintel/features/base/services/ProcessService.h @@ -39,6 +39,10 @@ namespace lintel { class ProcessService : public PropertyRegistration { private: + /** Stop and, if a child ignores SIGINT, terminate every process this service + * started. Idempotent, so onShutdown() and the destructor can both call it. */ + void stopAllProcesses(); + /** Internal wrapper holding a running process, its child handle, and an exit-code promise. */ class ProcessExecutes { @@ -287,6 +291,12 @@ class ProcessService : public PropertyRegistration ShutdownPriority shutdownPriorityOf() const override { return ShutdownPriority::PROCESS_SERVICE; } void onInitialize() override; + + /** Stop and, if a child ignores SIGINT, terminate every process this service + * started. Idempotent: the destructor calls it again, where it finds nothing + * left to do. Runs during shutdown so the children never outlive their + * parent, even when the container outlives the builder. */ + void onShutdown() override; }; }// namespace lintel diff --git a/src/include/lintel/features/base/services/SharedMemoryService.h b/src/include/lintel/features/base/services/SharedMemoryService.h index 876e38d2..a11239e4 100644 --- a/src/include/lintel/features/base/services/SharedMemoryService.h +++ b/src/include/lintel/features/base/services/SharedMemoryService.h @@ -98,10 +98,15 @@ class SharedMemoryService /** Reclaim the remap locks of segments whose holder process is gone. * - * Opened segments are checked once when they are created (see - * SharedMemorySegmentComponent/`create()`); this is the repeat an operator - * needs for a segment that got stuck later on, and it never touches a lock - * whose holder is still alive. + * Iterates over the segments this process has mapped and hands each one to + * shm::recoverOrphanedSegmentLock(), which reclaims the remap semaphore only + * when its recorded holder process no longer exists; a lock whose holder is + * still alive is never touched. A reclaimed semaphore is reopened so this + * process keeps using the object every other process sees, and the segment's + * recovery counter (reported by statsOf() as RECOVERY_COUNT) is bumped. + * + * Opening a segment already runs the same check once (see create()), so this + * is the repeat an operator needs for a segment that got stuck later on. * @return the number of locks that were reclaimed */ std::size_t recover() override; @@ -124,8 +129,11 @@ class SharedMemoryService /** Register a lifecycle hook. Hook must be short and non-blocking. */ void registerLifecycleHook(SegmentLifecycleHook hook); - /** Get statistics for a segment. */ - [[nodiscard]] SharedMemorySegmentStats statsOf(const std::shared_ptr &segment) const; + /** Get statistics for a segment. + * @param segment the segment to inspect + * @return size, capacity, free bytes, lifecycle counters and generation + * @throws ShmSegmentNotFound if the segment is not mapped by this service */ + [[nodiscard]] SharedMemorySegmentStats statsOf(const std::shared_ptr &segment) const override; /** Get the allocated state of the named shared memory segment. * Internal: the caller has to hold segmentsMutexOf(). @@ -137,9 +145,16 @@ class SharedMemoryService /** Get an accessor to allocate from the given shared memory segment. The * returned accessor is a value type and stays valid as long as the segment * is mapped by this service. + * + * The accessor carries the segment name and this process' allocator is bound + * to that name, so a copy stored inside the segment allocates from the + * process that uses it, not from the process that created it. + * * @param segment the shared memory segment * @return accessor of the segment - * @throws ShmSegmentNotFound if the segment does not exist */ + * @throws ShmSegmentNotFound if the segment does not exist + * @throws std::invalid_argument if the segment name is longer than + * ShmSegmentAccessor::NAME_SIZE - 1 characters */ ShmSegmentAccessor getSegmentAccessor(const std::shared_ptr &segment); }; diff --git a/src/include/lintel/features/base/services/ShmConstructors.h b/src/include/lintel/features/base/services/ShmConstructors.h index 957f6198..6f1eb701 100644 --- a/src/include/lintel/features/base/services/ShmConstructors.h +++ b/src/include/lintel/features/base/services/ShmConstructors.h @@ -185,7 +185,10 @@ class SharedMemorySegmentHandle std::shared_ptr m_lockOwner; // Typed allocator adapter of this segment. Created on demand and reset // whenever the mapping is replaced, so accessors always refer to the - // segment manager of the current mapping. + // segment manager of the current mapping. Accessors handed out by + // SharedMemoryService::getSegmentAccessor() carry the segment name and bind + // this allocator to it (ShmSegmentAccessor::bind), which is what makes a + // copy of an accessor inside the segment usable from another process. std::shared_ptr m_segmentAllocator; }; diff --git a/src/include/lintel/features/cli/components/AdminCliComponent.h b/src/include/lintel/features/cli/components/AdminCliComponent.h new file mode 100644 index 00000000..af1d2a0e --- /dev/null +++ b/src/include/lintel/features/cli/components/AdminCliComponent.h @@ -0,0 +1,65 @@ +#ifndef LINTEL_ADMINCLICOMPONENT_H +#define LINTEL_ADMINCLICOMPONENT_H + +#include +#include +#include +#include +#include +#include +#include + +#include "lintel/features/cli/models/CommandLineComponent.h" +#include "lintel/features/cli/models/CommandParser.h" +#include "lintel/features/http/controllers/AdminApi.h" + +namespace lintel { +/** CLI component that drives the administrative endpoints of the running server */ +class AdminCliComponent : public CommandLineComponent +{ +private: + static constexpr std::string_view m_name = "Admin"; + static constexpr std::string_view m_alias = "Adm"; + static constexpr std::string_view m_description = + "The component can be used to reload the TLS certificate and to change the log level"; + + std::shared_ptr m_adminApi; + + enum Commands : std::uint8_t { Undefined, ReloadTls, LogLevel }; + + // Flags + std::optional m_level; + + CommandParser m_commandParser; + + /** @return the value of the required flag or throw when it is missing + * @param value the flag value + * @param flag the flag name used in the error message */ + static const std::string &requiredOf(const std::optional &value, const char *flag); + + /** Print the per-resource result of a reload and the overall outcome + * @param response the answer of the controller */ + static void printReloadResults(const AdminTlsReloadResponse &response); + +public: + /** @param adminApi API for the administrative endpoints */ + explicit AdminCliComponent(std::shared_ptr adminApi); + + void onHelp() override; + + void + onCommand(const UserDto &userDto, const std::string &command, const std::vector ¶meters) override; + + void onShowMenu() override; + + bool onMenu(const std::string &component) override; + + bool onExit() override; + + void printCommandList(std::set menuAlias) override; + + std::vector allCommandsOf() override; +}; + +}// namespace lintel +#endif// LINTEL_ADMINCLICOMPONENT_H diff --git a/src/include/lintel/features/cli/components/EventBusCliComponent.h b/src/include/lintel/features/cli/components/EventBusCliComponent.h new file mode 100644 index 00000000..a65c2552 --- /dev/null +++ b/src/include/lintel/features/cli/components/EventBusCliComponent.h @@ -0,0 +1,81 @@ +#ifndef LINTEL_EVENTBUSCLICOMPONENT_H +#define LINTEL_EVENTBUSCLICOMPONENT_H + +#include +#include +#include +#include +#include +#include +#include + +#include "lintel/features/base/controller/EventBusDtos.h" +#include "lintel/features/cli/models/CommandLineComponent.h" +#include "lintel/features/cli/models/CommandParser.h" +#include "lintel/features/http/controllers/EventBusApi.h" + +namespace lintel { +/** CLI component that inspects and drives the configured event buses */ +class EventBusCliComponent : public CommandLineComponent +{ +private: + static constexpr std::string_view m_name = "EventBus"; + static constexpr std::string_view m_alias = "Eb"; + static constexpr std::string_view m_description = + "The component can be used to inspect the event buses and to publish events"; + + std::shared_ptr m_eventBusApi; + + enum Commands : std::uint8_t { + Undefined, + ShowEventBuses, + ShowEventBus, + PublishEvent, + SubscribeEvent, + UnsubscribeEvent, + EventQueueDepth, + ResetEventBus + }; + + // Flags + std::optional m_bus; + std::optional m_topic; + std::optional m_type; + std::optional m_payload; + std::optional m_priority; + std::optional m_subscriber; + bool m_toSubscribers = false; + + CommandParser m_commandParser; + + /** Print a table of event bus DTOs to the console + * @param eventBusDtos the buses to print */ + static void printEventBuses(const std::vector &eventBusDtos); + + /** @return the value of the required flag or throw when it is missing + * @param value the flag value + * @param flag the flag name used in the error message */ + static const std::string &requiredOf(const std::optional &value, const char *flag); + +public: + /** @param eventBusApi API for the event bus endpoints */ + explicit EventBusCliComponent(std::shared_ptr eventBusApi); + + void onHelp() override; + + void + onCommand(const UserDto &userDto, const std::string &command, const std::vector ¶meters) override; + + void onShowMenu() override; + + bool onMenu(const std::string &component) override; + + bool onExit() override; + + void printCommandList(std::set menuAlias) override; + + std::vector allCommandsOf() override; +}; + +}// namespace lintel +#endif// LINTEL_EVENTBUSCLICOMPONENT_H diff --git a/src/include/lintel/features/cli/components/HistoryCliComponent.h b/src/include/lintel/features/cli/components/HistoryCliComponent.h index f801efc3..35c5f1e2 100644 --- a/src/include/lintel/features/cli/components/HistoryCliComponent.h +++ b/src/include/lintel/features/cli/components/HistoryCliComponent.h @@ -9,7 +9,7 @@ namespace lintel { /** CLI component that displays command history */ class HistoryCliComponent : public CommandLineComponent { - static constexpr std::string_view n_name = "History"; + static constexpr std::string_view m_name = "History"; static constexpr std::string_view m_alias = "Hist"; static constexpr std::string_view m_description = "The Component can be used to show the history of components"; std::shared_ptr m_historyApi; diff --git a/src/include/lintel/features/cli/components/MessageQueueCliComponent.h b/src/include/lintel/features/cli/components/MessageQueueCliComponent.h index 63ee6d6e..232b90ad 100644 --- a/src/include/lintel/features/cli/components/MessageQueueCliComponent.h +++ b/src/include/lintel/features/cli/components/MessageQueueCliComponent.h @@ -16,7 +16,7 @@ class MessageQueueCliComponent : public CommandLineComponent static constexpr std::string_view m_description = "The component can be used to show the message queue statistics"; std::shared_ptr m_messageQueueApi; - enum Commands : std::uint8_t { Undefined, ShowMessageQueues, PurgeMessageQueue }; + enum Commands : std::uint8_t { Undefined, ShowMessageQueues, PurgeMessageQueue, RecoverMessageQueues }; // Flags std::optional m_processName; diff --git a/src/include/lintel/features/cli/components/SharedMemoryCliComponent.h b/src/include/lintel/features/cli/components/SharedMemoryCliComponent.h index b8bea7ae..0f5740ab 100644 --- a/src/include/lintel/features/cli/components/SharedMemoryCliComponent.h +++ b/src/include/lintel/features/cli/components/SharedMemoryCliComponent.h @@ -18,13 +18,22 @@ class SharedMemoryCliComponent : public CommandLineComponent std::shared_ptr m_sharedMemoryApi; // Commands - enum Commands { Undefined, ShowSegments, ShowRepositories, ShrinkSegment, GrowSegment, ExportRepository }; + enum Commands { + Undefined, + ShowSegments, + ShowRepositories, + ShowStats, + Cleanup, + RecoverSegments, + ShrinkSegment, + GrowSegment, + ExportRepository + }; // Flags std::optional m_segmentName; std::optional m_size; std::optional m_repositoryName; - std::optional m_type; std::optional m_file; std::optional m_uuid; @@ -36,6 +45,9 @@ class SharedMemoryCliComponent : public CommandLineComponent /** Print formatted repository DTOs to console */ static void printRepositories(const std::vector &repositories); + /** Print the statistics of one segment to console */ + static void printStats(const SharedMemorySegmentStats &stats); + /** Handle the export repository command */ void handleExportRepository(); diff --git a/src/include/lintel/features/cli/components/UserManagementCliComponent.h b/src/include/lintel/features/cli/components/UserManagementCliComponent.h index abddf81d..b5feb162 100644 --- a/src/include/lintel/features/cli/components/UserManagementCliComponent.h +++ b/src/include/lintel/features/cli/components/UserManagementCliComponent.h @@ -24,9 +24,6 @@ class UserManagementCliComponent : public CommandLineComponent Undefined, // group operations AllGroups, - AddGroup, - RemoveGroup, - UpdateGroup, // user operations AllUsers, AddUser, diff --git a/src/include/lintel/features/cli/services/CommandLineHistoryService.h b/src/include/lintel/features/cli/services/CommandLineHistoryService.h index 2179aba1..c49bad92 100644 --- a/src/include/lintel/features/cli/services/CommandLineHistoryService.h +++ b/src/include/lintel/features/cli/services/CommandLineHistoryService.h @@ -4,6 +4,7 @@ #include "lintel/core/services/PropertyRegistration.h" #include "lintel/features/cli/models/CommandHistoryEntry.h" #include +#include namespace lintel { /** @@ -44,7 +45,7 @@ class CommandLineHistoryService : public PropertyRegistration startsWith(const std::string &command, const std::string &menu); + std::optional startsWith(const std::string &command, std::string_view menu); /** * historizes the last entered command in the local file diff --git a/src/include/lintel/features/cli/services/CommandLineService.h b/src/include/lintel/features/cli/services/CommandLineService.h index ae445fc4..86a5f347 100644 --- a/src/include/lintel/features/cli/services/CommandLineService.h +++ b/src/include/lintel/features/cli/services/CommandLineService.h @@ -5,6 +5,7 @@ #include +#include #include #include #include @@ -28,7 +29,21 @@ class CommandLineComponent; class CommandLineService : public AbstractService { private: - enum Commands { CommandHelp, CommandClear, CommandMenu, CommandHistory, CommandExit, CommandUndefined }; + enum Commands : std::uint8_t { + CommandHelp, + CommandClear, + CommandMenu, + CommandHistory, + CommandExit, + CommandUndefined + }; + + /** Flags the prompt loop derives from one handled input symbol. */ + struct InputOutcome + { + bool m_tabPressed = false; + bool m_printPrompt = true; + }; AbstractCommandLineMenu m_menu; std::thread m_thread; @@ -64,11 +79,19 @@ class CommandLineService : public AbstractService /** Display the input prompt */ void onPrompt(); - /** Handle a regular command input symbol */ - void handleCommandInput(const SymbolEvent &symbolEvent, bool &tabPressed, bool &printPrompt); - - /** Handle tab key input for autocomplete */ - void handleTabInput(const SymbolEvent &symbolEvent, bool &tabPressed, bool &printPrompt); + /** + * Handle a regular command input symbol. + * @return the prompt flags to continue with + */ + InputOutcome handleCommandInput(const SymbolEvent &symbolEvent); + + /** + * Handle tab key input for autocomplete. + * @param symbolEvent the tab symbol, carrying the prefix typed so far + * @param tabPressed whether the previous tab press already completed a cycle + * @return the prompt flags to continue with + */ + InputOutcome handleTabInput(const SymbolEvent &symbolEvent, bool tabPressed); /** Main CLI loop running in a background thread */ void run(); diff --git a/src/include/lintel/features/http/controllers/AdminApi.h b/src/include/lintel/features/http/controllers/AdminApi.h new file mode 100644 index 00000000..2cb4f115 --- /dev/null +++ b/src/include/lintel/features/http/controllers/AdminApi.h @@ -0,0 +1,68 @@ +#ifndef LINTEL_ADMINAPI_H +#define LINTEL_ADMINAPI_H + +#include +#include +#include +#include + +#include "lintel/features/http/provider/ClientProvider.h" + +namespace lintel { +/** Result of one reloadable TLS resource, as reported by POST /admin/reload-tls */ +struct AdminTlsReloadResult +{ + /** Name of the reloadable resource, e.g. "tls" */ + std::string name; + /** "ok" or "failed: " */ + std::string detail; + /** True when the resource serves the new certificate */ + bool reloaded = false; +}; + +/** Answer of POST /admin/reload-tls */ +struct AdminTlsReloadResponse +{ + /** "reloaded", "failed" or "nothing_to_reload" */ + std::string status; + /** One entry per reloadable resource, empty when nothing is registered */ + std::vector results; +}; + +/** Answer of POST /admin/log-level */ +struct AdminLogLevelResponse +{ + /** "changed" when the level was applied, "invalid_level" when it was rejected */ + std::string status; + /** The level that was requested, empty for a malformed body */ + std::string level; +}; + +/** API client for the administrative endpoints of the HTTP service */ +class AdminApi +{ +private: + std::shared_ptr m_client; + +public: + /** @param clientProvider provider of the configured HTTP client */ + explicit AdminApi(const std::shared_ptr &clientProvider); + + /** + * Re-read the certificate and private key of the running TLS server. + * @return the per-resource result, nullopt when no response was received or + * the body could not be parsed + */ + std::optional reloadTlsOf(); + + /** + * Change the log level of the running process. + * @param levelName trace, debug, info, warn, err (or error), critical or off + * @return the applied or rejected level, nullopt when no response was + * received or the body could not be parsed + */ + std::optional logLevelOf(const std::string &levelName); +}; + +}// namespace lintel +#endif// LINTEL_ADMINAPI_H diff --git a/src/include/lintel/features/http/controllers/EventBusApi.h b/src/include/lintel/features/http/controllers/EventBusApi.h new file mode 100644 index 00000000..2ac8b40b --- /dev/null +++ b/src/include/lintel/features/http/controllers/EventBusApi.h @@ -0,0 +1,103 @@ +#ifndef LINTEL_EVENTBUSAPI_H +#define LINTEL_EVENTBUSAPI_H + +#include +#include +#include +#include +#include + +#include "lintel/features/base/controller/EventBusDtos.h" +#include "lintel/features/http/provider/ClientProvider.h" + +namespace lintel { +/** Description of an event that should be published */ +struct EventPublishRequest +{ + /** Name of the bus */ + std::string bus; + /** Topic of the event (at most 31 bytes) */ + std::string topic; + /** Type of the event (at most 31 bytes) */ + std::string type; + /** Payload of the event (at most 256 bytes) */ + std::string payload; + /** Priority: Critical, High, Normal or Low */ + std::string priority = "Normal"; + /** True to broadcast to the subscribers, false for the point-to-point queue */ + bool toSubscribers = false; +}; + +/** A subscription of a named subscriber to a topic of one bus */ +struct EventSubscription +{ + /** Name of the bus */ + std::string bus; + /** Topic name or wildcard pattern */ + std::string topic; + /** Name of the subscriber */ + std::string subscriber; +}; + +/** API client for the event bus endpoints of the HTTP service */ +class EventBusApi +{ +private: + std::shared_ptr m_client; + +public: + /** @param clientProvider provider of the configured HTTP client */ + explicit EventBusApi(const std::shared_ptr &clientProvider); + + /** @return all configured event buses with their configuration and runtime counters */ + std::vector allOf(); + + /** + * Read one configured event bus. + * @param name exact bus name + * @return the bus, nullopt when it is unknown or the request was rejected + */ + std::optional of(const std::string &name); + + /** + * Publish an event on a bus. + * @param request the event to publish + * @return true if the bus accepted the event, nullopt when the request was + * rejected; false means every matching queue was full + */ + std::optional publishOf(const EventPublishRequest &request); + + /** + * Subscribe a named subscriber to a topic. + * @param subscription the subscription to create + * @return true if the subscription exists afterwards, nullopt when the + * request was rejected + */ + std::optional subscribeOf(const EventSubscription &subscription); + + /** + * Remove a subscription. + * @param subscription the subscription to remove + * @return true if a subscription was removed, nullopt when the request was + * rejected + */ + std::optional unsubscribeOf(const EventSubscription &subscription); + + /** + * Read the queue depth of one subscriber. + * @param name exact bus name + * @param subscriber subscriber name + * @return the number of queued events, nullopt when the request was rejected + */ + std::optional depthOf(const std::string &name, const std::string &subscriber); + + /** + * Drop every queued event of a bus and of its subscriber queues. + * @param name exact bus name + * @return true if the bus was reset, nullopt when the request was rejected + */ + std::optional resetOf(const std::string &name); +}; + +}// namespace lintel +#endif// LINTEL_EVENTBUSAPI_H diff --git a/src/include/lintel/features/http/controllers/EventBusController.h b/src/include/lintel/features/http/controllers/EventBusController.h new file mode 100644 index 00000000..13e8f428 --- /dev/null +++ b/src/include/lintel/features/http/controllers/EventBusController.h @@ -0,0 +1,89 @@ +#ifndef LINTEL_EVENTBUSCONTROLLER_H +#define LINTEL_EVENTBUSCONTROLLER_H + +#include +#include +#include +#include +#include + +#include "lintel/features/base/controller/EventBusDtos.h" +#include "lintel/features/base/services/EventBusService.h" +#include "lintel/features/base/services/IAuthService.h" +#include "lintel/features/http/service/ContentType.h" +#include "lintel/features/http/service/Controller.h" + +namespace lintel { +/** + * HTTP controller that exposes the configured event buses. + * + * Reading (list, details, subscriber depth) is open to the "User-EventBus" and + * "Admin-EventBus" groups. Publishing requires the "Publisher-EventBus" or + * "Admin-EventBus" group, so a plain reader cannot inject events. Subscribing, + * unsubscribing and resetting mutate the bus registry or drop queued events, so + * they require the "Admin-EventBus" group. A missing token and a token of a user + * outside the required group both answer 401, which keeps the endpoints from + * confirming their existence to an anonymous caller. + */ +class EventBusController : public Controller +{ +private: + std::shared_ptr m_eventBusService; + Group m_adminGroup; + Group m_userGroup; + Group m_publisherGroup; + + /** Required permission of a route. */ + enum class Access : std::uint8_t { + /** read bus information - user or admin */ + Read, + /** publish an event - publisher or admin */ + Publish, + /** mutate subscriptions or drop queued events - admin only */ + Administer + }; + + /** Check the token, the group membership and the content type of a request. + * @param response receives 401 (missing token or wrong group) or 403 + * (unsupported content type) when the request is rejected + * @param contentType negotiated content type + * @param user authenticated user, if any + * @param access required permission of the route + * @return true if the request may proceed */ + [[nodiscard]] bool authorize(httplib::Response &response, + const ContentType &contentType, + const std::optional &user, + Access access) const; + + /** Resolve a configured bus. + * @param response receives 404 when the name is not configured + * @param contentType negotiated content type + * @param name bus name + * @return the bus view, empty when the response was set to 404 */ + [[nodiscard]] std::optional> + busOf(httplib::Response &response, const ContentType &contentType, const std::string &name); + + /** Build the DTO of one configured bus, including the best-effort runtime + * counters while the bus can be opened. + * @param name bus name + * @return the bus DTO */ + [[nodiscard]] EventBusDto describeOf(const std::string &name) const; + +public: + ADD_HANDLER_METHOD(R"(/eventBus/buses)", Get, busesOf); + ADD_HANDLER_METHOD(R"(/eventBus/bus/([^\/]+))", Get, busOf); + ADD_HANDLER_METHOD(R"(/eventBus/publish/([^\/]+))", Post, publishOf); + ADD_HANDLER_METHOD(R"(/eventBus/subscribe/([^\/]+))", Post, subscribeOf); + ADD_HANDLER_METHOD(R"(/eventBus/subscribe/([^\/]+))", Delete, unsubscribeOf); + ADD_HANDLER_METHOD(R"(/eventBus/depth/([^\/]+)/([^\/]+))", Get, depthOf); + ADD_HANDLER_METHOD(R"(/eventBus/reset/([^\/]+))", Post, resetOf); + + /** Construct an EventBusController. + * @param authService authentication service for access control + * @param eventBusService service that owns the configured buses */ + EventBusController(const std::shared_ptr &authService, + std::shared_ptr eventBusService); +}; + +}// namespace lintel +#endif// LINTEL_EVENTBUSCONTROLLER_H diff --git a/src/include/lintel/features/http/controllers/HistoryController.h b/src/include/lintel/features/http/controllers/HistoryController.h index ba05bb19..5c857eb8 100644 --- a/src/include/lintel/features/http/controllers/HistoryController.h +++ b/src/include/lintel/features/http/controllers/HistoryController.h @@ -20,7 +20,6 @@ class HistoryController : public Controller public: explicit HistoryController(const std::shared_ptr &authService, - std::shared_ptr groupRepository, std::shared_ptr historyRepository); }; diff --git a/src/include/lintel/features/http/controllers/MessageQueueApi.h b/src/include/lintel/features/http/controllers/MessageQueueApi.h index 690d96a7..54fc628c 100644 --- a/src/include/lintel/features/http/controllers/MessageQueueApi.h +++ b/src/include/lintel/features/http/controllers/MessageQueueApi.h @@ -4,6 +4,7 @@ #include "lintel/features/base/repositories/MessageQueueQuery.h" #include "lintel/features/http/provider/ClientProvider.h" +#include #include namespace lintel { @@ -27,6 +28,12 @@ class MesssageQueueApi * @return number of removed messages, nullopt if the purge was rejected */ std::optional purgeOf(const std::string &messageQueueName); + + /** + * Reclaim the queue entries whose owner process is gone. + * @return number of reclaimed entries, nullopt if the request was rejected + */ + std::optional recoverOf(); }; }// namespace lintel diff --git a/src/include/lintel/features/http/controllers/MessageQueueController.h b/src/include/lintel/features/http/controllers/MessageQueueController.h index de0bd081..7b7dadcb 100644 --- a/src/include/lintel/features/http/controllers/MessageQueueController.h +++ b/src/include/lintel/features/http/controllers/MessageQueueController.h @@ -38,6 +38,7 @@ class MessageQueueController : public Controller ADD_HANDLER_METHOD(R"(/messageQueue/([^\/]+)/([^\/]+))", Get, messageQueueOf); ADD_HANDLER_METHOD(R"(/messageQueue/stats/([^\/]+)/([^\/]+))", Get, statisticsOf); ADD_HANDLER_METHOD(R"(/messageQueue/purge/([^\/]+))", Post, purgeOf); + ADD_HANDLER_METHOD(R"(/messageQueue/recover)", Post, recoverOf); public: explicit MessageQueueController(const std::shared_ptr &authService, diff --git a/src/include/lintel/features/http/controllers/SharedMemoryApi.h b/src/include/lintel/features/http/controllers/SharedMemoryApi.h index bb951f33..0764b9f7 100644 --- a/src/include/lintel/features/http/controllers/SharedMemoryApi.h +++ b/src/include/lintel/features/http/controllers/SharedMemoryApi.h @@ -1,13 +1,32 @@ #ifndef LINTEL_SHAREDMEMORYAPI_H #define LINTEL_SHAREDMEMORYAPI_H +#include #include +#include #include "lintel/features/base/controller/SharedMemoryRepositoryDto.h" #include "lintel/features/base/controller/SharedMemorySegmentDto.h" +#include "lintel/features/base/models/SharedMemorySegmentStats.h" #include "lintel/features/http/provider/ClientProvider.h" namespace lintel { +/** Counts returned by the shared memory cleanup endpoint. + * The endpoint only reports numbers, not the individual orphaned objects. */ +struct SharedMemoryCleanupSummary +{ + /** true when the run only reported and did not remove anything */ + bool dryRun = false; + /** number of registry entries inspected */ + std::size_t scanned = 0; + /** orphaned objects found by this run */ + std::size_t orphans = 0; + /** orphaned objects actually removed by this run */ + std::size_t removed = 0; + /** removals that failed and are retried by the next run */ + std::size_t failed = 0; +}; + /** API client for shared memory management via HTTP */ class SharedMemoryApi { @@ -27,6 +46,25 @@ class SharedMemoryApi /** Grow a shared memory segment by name and size */ void growOf(const std::string &segmentName, const std::string &sizeStr); + /** + * Read the runtime statistics of one segment. + * @param segmentName exact segment name + * @return the statistics, or nullopt when the segment is unknown or the request was rejected + */ + std::optional statsOf(const std::string &segmentName); + + /** + * Remove the shared memory objects whose owner process is gone. + * @return the counts of the run, or nullopt when the request was rejected + */ + std::optional cleanupOf(); + + /** + * Reclaim the remap locks of segments whose holder process is gone. + * @return the number of reclaimed locks, or nullopt when the request was rejected + */ + std::optional recoverOf(); + // repositories /** @return all repositories matching the given filters */ std::vector allRepositoriesOf(const std::string &repositoryName = ".*", diff --git a/src/include/lintel/features/http/controllers/SharedMemoryController.h b/src/include/lintel/features/http/controllers/SharedMemoryController.h index 8435794c..0f85fdcd 100644 --- a/src/include/lintel/features/http/controllers/SharedMemoryController.h +++ b/src/include/lintel/features/http/controllers/SharedMemoryController.h @@ -5,6 +5,7 @@ #include "lintel/features/base/repositories/SharedMemoryRepository.h" #include "lintel/features/base/services/IAuthService.h" #include "lintel/features/base/services/ISharedMemorySegmentManager.h" +#include "lintel/features/base/services/SharedMemoryCleanupService.h" #include "lintel/features/http/service/ContentType.h" #include "lintel/features/http/service/Controller.h" @@ -15,6 +16,7 @@ class SharedMemoryController : public Controller private: std::shared_ptr m_sharedMemoryService; std::shared_ptr m_sharedMemorySegmentManager; + std::shared_ptr m_sharedMemoryCleanupService; std::vector> m_sharedMemoryRepositories; std::map> m_uuidSharedMemoryRepositories; Group m_adminGroup; @@ -24,6 +26,12 @@ class SharedMemoryController : public Controller static std::map> build( const std::vector> &repositories); + /** @return true if the user may read segment information */ + [[nodiscard]] bool canRead(const std::optional &user) const; + + /** @return true if the user may mutate segment or cleanup state */ + [[nodiscard]] bool canAdminister(const std::optional &user) const; + public: // Shared Memory Segment handler methods ADD_HANDLER_METHOD(R"(/shm/segments/([^\/]+))", Get, allSegmentsOf); @@ -34,11 +42,17 @@ class SharedMemoryController : public Controller ADD_HANDLER_METHOD(R"(/shm/repositories/([^\/]+)/([^\/]+))", Get, allRepositoriesOf); ADD_HANDLER_METHOD(R"(/shm/repository/([^\/]+))", Get, exportRepositoryOf); + // Shared Memory Operations handler methods + ADD_HANDLER_METHOD(R"(/shm/stats/([^\/]+))", Get, statsOf); + ADD_HANDLER_METHOD(R"(/shm/cleanup)", Post, cleanupOf); + ADD_HANDLER_METHOD(R"(/shm/recover)", Post, recoverOf); + SharedMemoryController(const std::shared_ptr &authServicie, std::shared_ptr sharedMemoryService, std::shared_ptr sharedMemorySegmentManager, - std::vector> sharedMemoryRepositories); + std::vector> sharedMemoryRepositories, + std::shared_ptr sharedMemoryCleanupService); }; }// namespace lintel diff --git a/src/include/lintel/features/property/repositories/DatabasePropertyRepository.h b/src/include/lintel/features/property/repositories/DatabasePropertyRepository.h index 9dc6b64b..fde8c2fa 100644 --- a/src/include/lintel/features/property/repositories/DatabasePropertyRepository.h +++ b/src/include/lintel/features/property/repositories/DatabasePropertyRepository.h @@ -42,10 +42,7 @@ class DatabasePropertyRepository : public PropertyRepository */ std::vector> awake() override; - std::vector> allOf(const std::string &processName = ".*", - const std::string &className = ".*", - const std::string &instanceName = ".*", - const std::string &name = ".*") override; + std::vector> allOf(const PropertyFilter &filter) override; }; }// namespace lintel diff --git a/src/include/lintel/features/property/repositories/PropertyRepository.h b/src/include/lintel/features/property/repositories/PropertyRepository.h index fe2711d9..0ef76905 100644 --- a/src/include/lintel/features/property/repositories/PropertyRepository.h +++ b/src/include/lintel/features/property/repositories/PropertyRepository.h @@ -2,6 +2,7 @@ #define LOGGING_PROPERTYREPOSITORY_H #include +#include #include #include "lintel/core/configuration/Configuration.h" @@ -13,6 +14,31 @@ namespace lintel { class PropertyBase; +/** + * Filter for PropertyRepository::allOf; every field is a regular expression. + * The four coordinates are same-typed and follow the process/property + * hierarchy, so a named aggregate keeps them from being swapped at the call + * site. + */ +struct PropertyFilter +{ + /** Process name pattern; pass ".*" to match every process */ + std::string m_processName{ ".*" }; + /** Class name pattern; pass ".*" to match every class */ + std::string m_className{ ".*" }; + /** Instance name pattern; pass ".*" to match every instance */ + std::string m_instanceName{ ".*" }; + /** Property name pattern; pass ".*" to match every property */ + std::string m_name{ ".*" }; + + /** Equality so the filter can be matched by value (trompeloeil, tests). */ + friend bool operator==(const PropertyFilter &lhs, const PropertyFilter &rhs) + { + return lhs.m_processName == rhs.m_processName && lhs.m_className == rhs.m_className + && lhs.m_instanceName == rhs.m_instanceName && lhs.m_name == rhs.m_name; + } +}; + /** Abstract base class for property repositories */ class PropertyRepository { @@ -103,18 +129,9 @@ class PropertyRepository * "keep the configured default". The default is only reached for * `FilePropertyRepository`, whose values come from the XML configuration * and are never queried at runtime. - * @param processName regex for process name - * @param className regex for class name - * @param instanceName regex for instance name - * @param name regex for property name + * @param filter regexes for process, class, instance and property name */ - virtual std::vector> allOf(const std::string &processName = ".*", - const std::string &className = ".*", - const std::string &instanceName = ".*", - const std::string &name = ".*") - { - return {}; - } + virtual std::vector> allOf(const PropertyFilter &filter = {}) { return {}; } }; }// namespace lintel diff --git a/src/include/lintel/features/property/repositories/SharedMemoryPropertyRepository.h b/src/include/lintel/features/property/repositories/SharedMemoryPropertyRepository.h index 6ad0b750..d64ca5fe 100644 --- a/src/include/lintel/features/property/repositories/SharedMemoryPropertyRepository.h +++ b/src/include/lintel/features/property/repositories/SharedMemoryPropertyRepository.h @@ -135,10 +135,7 @@ class SharedMemoryPropertyRepository std::vector> awake() override; - std::vector> allOf(const std::string &processName, - const std::string &className, - const std::string &instanceName, - const std::string &name) override; + std::vector> allOf(const PropertyFilter &filter) override; void onMigrate(int32_t currentActiveVersion) override; diff --git a/src/include/lintel/features/property/services/PropertyService.h b/src/include/lintel/features/property/services/PropertyService.h index 8b5c5152..043e60ad 100644 --- a/src/include/lintel/features/property/services/PropertyService.h +++ b/src/include/lintel/features/property/services/PropertyService.h @@ -21,7 +21,7 @@ #define DEFINE_PROPERTY(name, type, defaultValue, description, runtime) \ std::shared_ptr> name = \ - registerProperty(std::string(#name), defaultValue, description, runtime, __FILE__, __LINE__) + registerProperty(std::string(#name), defaultValue, description, runtime) #define LOAD_PROPERTIES() \ if (propertyService != nullptr) { propertyService->getOrCreate(m_properties); } diff --git a/swagger.md b/swagger.md index c518413a..b39a06d6 100644 --- a/swagger.md +++ b/swagger.md @@ -205,6 +205,333 @@ Returns message queues matching the given name patterns as a JSON array. With th | --------------- | ------ | | bearerAuth | | +### /messageQueue/stats/{processName}/{messageQueueName} + +#### GET +##### Summary + +Get message queue statistics + +##### Description + +Returns the message queues matching the given name patterns with their operational state as a JSON array. The payload is built by the same code path as /messageQueue/{processName}/{messageQueueName}: every entry carries NAME, PROCESS, MAX_MESSAGES, MESSAGES and CONSUMER_LAG plus ALIVE, and - only while an owner is registered - OWNER_PID, OWNER_HOST, OWNER_STATE, STARTED_AT, LAST_HEARTBEAT_AT and HEARTBEAT_AGE_MS, as well as DEAD_LETTER_QUEUE and DEAD_LETTER_SIZE when a dead-letter queue is configured. Absent optional fields are omitted from the JSON object. CONSUMER_LAG is the number of messages still waiting for a consumer; the queue has no consumer cursor of its own, so it equals MESSAGES. + +With the optional query parameters 'after' and 'limit' the result is keyset-paginated (ordered by queue name ascending) and returned as an envelope object {items, has_more, next_after} whose entries are built without a queue reading, i.e. with MESSAGES and CONSUMER_LAG set to 0. Follow next_after to retrieve the next page. Reading requires a token of a user in the "Admin-MessageQueue" or "User-MessageQueue" group. + +##### Parameters + +| Name | Located in | Description | Required | Schema | +| ---- | ---------- | ----------- | -------- | ------ | +| processName | path | | Yes | string | +| messageQueueName | path | | Yes | string | +| after | query | Exclusive lower bound on queue name; use next_after of the previous page. | No | string | +| limit | query | Maximum number of entries per page (default 100). | No | integer | + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | A successful response (array of message queues with their statistics, or paging envelope when after/limit is given) | +| 400 | Bad request (invalid limit) | +| 401 | Unauthorized | +| 403 | Forbidden (wrong content type) | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +### /messageQueue/purge/{processName} + +#### POST +##### Summary + +Purge a message queue + +##### Description + +Removes every message that is currently in the named queue and answers with the queue name and the number of removed messages ({"NAME":"orders","PURGED":42}). The queue name is looked up through a repository API that treats it as a pattern, so only an exact name match is purged: a name that merely matches a queue as a pattern (for example "mq-.*") answers 404, one call can therefore never drain several queues. If opening the queue or deleting its messages fails, the handler answers 500. Purging is destructive and requires a token of a user in the "Admin-MessageQueue" group; a member of "User-MessageQueue" alone gets 401. + +##### Parameters + +| Name | Located in | Description | Required | Schema | +| ---- | ---------- | ----------- | -------- | ------ | +| processName | path | Exact message queue name to drain; patterns are rejected with 404. | Yes | string | + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | Queue purged (MessageQueuePurgeDto JSON) | +| 401 | Unauthorized (missing token or not in the Admin-MessageQueue group) | +| 403 | Forbidden (wrong content type) | +| 404 | Unknown queue name (the name matches no queue exactly) | +| 500 | Internal server error (queue could not be opened or drained) | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +### /messageQueue/recover + +#### POST +##### Summary + +Recover orphaned message queues + +##### Description + +Reclaims the message queue entries whose recorded owner process is gone and answers with the number of reclaimed entries ({"RECOVERED":2}). An entry whose owner is still alive is never touched, no matter how old its heartbeat is. Reclaiming mutates queue state and requires a token of a user in the "Admin-MessageQueue" group; a member of "User-MessageQueue" alone gets 401. + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | Entries reclaimed ({"RECOVERED":n}) | +| 401 | Unauthorized (missing token or not in the Admin-MessageQueue group) | +| 403 | Forbidden (wrong content type) | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +### /eventBus/buses + +#### GET +##### Summary + +List event buses + +##### Description + +Returns every configured event bus with its validated bootstrap configuration and, while the bus can be opened in its shared memory segment (AVAILABLE), its runtime counters. A bus whose bootstrap configuration is invalid is still listed, with AVAILABLE false and without configuration fields; a bus whose segment is missing reports AVAILABLE false but keeps its configuration. Reading requires a token of a user in the "Admin-EventBus" or "User-EventBus" group. + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | All configured event buses (array of EventBusDto) | +| 401 | Unauthorized (missing token or not in an event bus group) | +| 403 | Forbidden (wrong content type) | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +### /eventBus/bus/{name} + +#### GET +##### Summary + +Get one event bus + +##### Description + +Returns the same object as one entry of /eventBus/buses for exactly one bus. An unknown name answers 404. Reading requires a token of a user in the "Admin-EventBus" or "User-EventBus" group. + +##### Parameters + +| Name | Located in | Description | Required | Schema | +| ---- | ---------- | ----------- | -------- | ------ | +| name | path | Exact bus name | Yes | string | + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | The event bus (EventBusDto JSON) | +| 401 | Unauthorized (missing token or not in an event bus group) | +| 403 | Forbidden (wrong content type) | +| 404 | Unknown bus name | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +### /eventBus/publish/{name} + +#### POST +##### Summary + +Publish an event on a bus + +##### Description + +Builds an Event from the JSON body and hands it to the bus. TOPIC and TYPE are required, PAYLOAD is an optional string of at most 256 bytes, PRIORITY is one of Critical, High, Normal or Low (default Normal) and TARGET is BUS (default, the point-to-point queue) or SUBSCRIBERS (the queues of every matching subscriber). Names longer than 31 bytes and payloads longer than 256 bytes answer 400 instead of being truncated. Publishing requires the "Publisher-EventBus" group (or "Admin-EventBus"); "User-EventBus" is read-only and gets 401. A full queue is not an error: the answer carries PUBLISHED false and the bus counts the event as dropped. With TARGET SUBSCRIBERS, PUBLISHED is also false when no subscription matched the topic. + +##### Parameters + +| Name | Located in | Description | Required | Schema | +| ---- | ---------- | ----------- | -------- | ------ | +| name | path | Exact bus name | Yes | string | +| body | body | {"TOPIC":"orders.created","TYPE":"OrderCreated","PAYLOAD":"42","PRIORITY":"High","TARGET":"BUS"} | Yes | object | + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | The event was handed to the bus ({"BUS":...,"PRIORITY":...,"PUBLISHED":...,"SEQUENCE":...,"TARGET":...,"TOPIC":...,"TYPE":...}) | +| 400 | Body is not json, a required member is missing, a name or the payload is too long, or PRIORITY/TARGET is unknown | +| 401 | Unauthorized (missing token or not in an event bus group) | +| 403 | Forbidden (wrong content type) | +| 404 | Unknown bus name | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +### /eventBus/subscribe/{name} + +#### POST +##### Summary + +Subscribe a subscriber to a topic + +##### Description + +Adds the named subscriber to the topic. An exact topic is registered implicitly, so a subscriber can join a topic that was never published before; wildcard patterns (`*` matches any sequence, `?` matches one character) register themselves. The answer reports SUBSCRIBED false when the topic or subscriber table of the bus is full. Subscribing requires a token of a user in the "Admin-EventBus" group; a member of "User-EventBus" alone gets 401. + +##### Parameters + +| Name | Located in | Description | Required | Schema | +| ---- | ---------- | ----------- | -------- | ------ | +| name | path | Exact bus name | Yes | string | +| body | body | {"TOPIC":"orders.*","SUBSCRIBER":"worker"} | Yes | object | + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | The subscription was attempted ({"BUS":...,"SUBSCRIBER":...,"SUBSCRIBED":...,"TOPIC":...}) | +| 400 | Body is not json, a required member is missing or a name is too long | +| 401 | Unauthorized (missing token or not in the Admin-EventBus group) | +| 403 | Forbidden (wrong content type) | +| 404 | Unknown bus name | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +#### DELETE +##### Summary + +Unsubscribe a subscriber from a topic + +##### Description + +Removes the subscription of the named subscriber from the topic and answers with UNSUBSCRIBED false when there was no such subscription. Unsubscribing requires a token of a user in the "Admin-EventBus" group. + +##### Parameters + +| Name | Located in | Description | Required | Schema | +| ---- | ---------- | ----------- | -------- | ------ | +| name | path | Exact bus name | Yes | string | +| body | body | {"TOPIC":"orders.*","SUBSCRIBER":"worker"} | Yes | object | + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | The subscription was removed or did not exist ({"BUS":...,"SUBSCRIBER":...,"TOPIC":...,"UNSUBSCRIBED":...}) | +| 400 | Body is not json, a required member is missing or a name is too long | +| 401 | Unauthorized (missing token or not in the Admin-EventBus group) | +| 403 | Forbidden (wrong content type) | +| 404 | Unknown bus name | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +### /eventBus/depth/{name}/{subscriber} + +#### GET +##### Summary + +Get the queue depth of one subscriber + +##### Description + +Returns the best-effort number of events waiting in the queues of the named subscriber. The value is approximate while other processes push or pop events concurrently; an unknown subscriber reports 0. Reading requires a token of a user in the "Admin-EventBus" or "User-EventBus" group. + +##### Parameters + +| Name | Located in | Description | Required | Schema | +| ---- | ---------- | ----------- | -------- | ------ | +| name | path | Exact bus name | Yes | string | +| subscriber | path | Subscriber name (at most 31 bytes) | Yes | string | + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | Queue depth of the subscriber ({"BUS":...,"DEPTH":...,"SUBSCRIBER":...}) | +| 400 | Subscriber name is longer than 31 bytes | +| 401 | Unauthorized (missing token or not in an event bus group) | +| 403 | Forbidden (wrong content type) | +| 404 | Unknown bus name | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +### /eventBus/reset/{name} + +#### POST +##### Summary + +Reset an event bus + +##### Description + +Drops every queued event of the bus and of all subscriber queues and answers with RESET true. Resetting is destructive and requires a token of a user in the "Admin-EventBus" group; a member of "User-EventBus" alone gets 401. + +##### Parameters + +| Name | Located in | Description | Required | Schema | +| ---- | ---------- | ----------- | -------- | ------ | +| name | path | Exact bus name | Yes | string | + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | The bus was reset ({"BUS":...,"RESET":true}) | +| 401 | Unauthorized (missing token or not in the Admin-EventBus group) | +| 403 | Forbidden (wrong content type) | +| 404 | Unknown bus name | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + ### /process/processes/{groupName} #### GET @@ -445,6 +772,91 @@ Terminate a process | --------------- | ------ | | bearerAuth | | +### /shm/stats/{segmentName} + +#### GET +##### Summary + +Get the runtime statistics of one segment + +##### Description + +Returns the statistics of exactly one shared memory segment as a JSON object with the fields NAME, ALLOCATED_BYTES, CAPACITY_BYTES, FREE_BYTES, GROW_COUNT, SHRINK_COUNT, RECOVERY_COUNT and GENERATION. CAPACITY_BYTES is the configured maximum size, or ALLOCATED_BYTES when the segment has no maximum. Reading requires a token of a user in the "Admin-Shm" or "User-Shm" group; an unknown segment answers 404. + +##### Parameters + +| Name | Located in | Description | Required | Schema | +| ---- | ---------- | ----------- | -------- | ------ | +| segmentName | path | Exact segment name | Yes | string | + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | Segment statistics (JSON object) | +| 401 | Unauthorized (missing token or neither in the Admin-Shm nor the User-Shm group) | +| 403 | Forbidden (wrong content type) | +| 404 | Unknown segment | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +### /shm/cleanup + +#### POST +##### Summary + +Remove orphaned shared memory objects + +##### Description + +Runs the cleanup service in apply mode: every registered shared memory object whose recorded owner process is gone is removed from its segment and answers with the counts of the run as a JSON object with the fields DRY_RUN, SCANNED, ORPHANS, REMOVED and FAILED. DRY_RUN is always false because this endpoint applies; the scan-only mode of the service is not exposed. An object whose owner is still alive is never touched, no matter how old its heartbeat is. A failed removal keeps its registration and is retried by the next run, so FAILED may be non-zero while the request still answers 200. Cleanup is destructive and requires a token of a user in the "Admin-Shm" group; a member of "User-Shm" alone gets 401. + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | Cleanup applied (JSON object with the run counts) | +| 401 | Unauthorized (missing token or not in the Admin-Shm group) | +| 403 | Forbidden (wrong content type) | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + +### /shm/recover + +#### POST +##### Summary + +Reclaim the remap locks of segments whose holder is gone + +##### Description + +Reclaims the remap semaphore of every shared memory segment this process maps whose recorded holder process no longer exists and answers with the number of reclaimed locks ({"RECOVERED":1}). A lock whose holder is still alive is never touched, no matter how long it has been held. Opening a segment already runs the same check once, so this endpoint is the repeat an operator needs for a segment that got stuck later on. Reclaiming changes the lock state of a segment and requires a token of a user in the "Admin-Shm" group; a member of "User-Shm" alone gets 401. + +##### Responses + +| Code | Description | +| ---- | ----------- | +| 200 | Locks reclaimed ({"RECOVERED":n}) | +| 401 | Unauthorized (missing token or not in the Admin-Shm group) | +| 403 | Forbidden (wrong content type) | +| 500 | Internal server error | + +##### Security + +| Security Schema | Scopes | +| --------------- | ------ | +| bearerAuth | | + ### /shm/segments/{segmentName} #### GET @@ -1079,6 +1491,26 @@ Update a property ## Models +### EventBusDto + +| Name | Type | Description | Required | +| ---- | ---- | ----------- | -------- | +| NAME | string | Name of the bus | Yes | +| AVAILABLE | boolean | True while the bus can be opened in its shared memory segment | Yes | +| BUS_CAPACITY | integer | Capacity of the point-to-point bus queue | No | +| SUBSCRIBER_CAPACITY | integer | Capacity of every per-subscriber queue | No | +| MAX_TOPICS | integer | Maximum number of registerable topics | No | +| MAX_SUBSCRIBERS | integer | Maximum number of registerable subscribers | No | +| MAX_PRIORITIES | integer | Number of priority queues per subscriber | No | +| PUB_POLICY | string | Drop policy when a queue is full | No | +| STICKY | boolean | Whether sticky/last-value events are enabled | No | +| STICKY_TTL_MS | integer | TTL of sticky events in milliseconds | No | +| DROPPED | integer | Number of events dropped because a queue was full | No | +| TOPIC_COUNT | integer | Number of registered topics | No | +| SUBSCRIBER_COUNT | integer | Number of reserved subscriber slots | No | +| BUS_DEPTH | integer | Best-effort depth of the point-to-point queue | No | +| SEQUENCE | integer | Last sequence number assigned by the bus | No | + ### ProcessInfoDto | Name | Type | Description | Required | diff --git a/swagger.yaml b/swagger.yaml index b98f164d..0b5cc439 100644 --- a/swagger.yaml +++ b/swagger.yaml @@ -227,6 +227,505 @@ paths: description: Forbidden (wrong content type) "500": description: Internal server error + /messageQueue/stats/{processName}/{messageQueueName}: + get: + summary: Get message queue statistics + description: > + Returns the message queues matching the given name patterns with their + operational state as a JSON array. The payload is built by the same code + path as /messageQueue/{processName}/{messageQueueName}: every entry + carries NAME, PROCESS, MAX_MESSAGES, MESSAGES and CONSUMER_LAG plus + ALIVE, and - only while an owner is registered - OWNER_PID, OWNER_HOST, + OWNER_STATE, STARTED_AT, LAST_HEARTBEAT_AT and HEARTBEAT_AGE_MS, as well + as DEAD_LETTER_QUEUE and DEAD_LETTER_SIZE when a dead-letter queue is + configured. Absent optional fields are omitted from the JSON object. + CONSUMER_LAG is the number of messages still waiting for a consumer; the + queue has no consumer cursor of its own, so it equals MESSAGES. + With the optional query parameters 'after' and 'limit' the result is + keyset-paginated (ordered by queue name ascending) and returned as an + envelope object {items, has_more, next_after} whose entries are built + without a queue reading, i.e. with MESSAGES and CONSUMER_LAG set to 0. + Follow next_after to retrieve the next page. Reading requires a token of + a user in the "Admin-MessageQueue" or "User-MessageQueue" group. + security: + - bearerAuth: [] + parameters: + - name: processName + in: path + required: true + type: string + - name: messageQueueName + in: path + required: true + type: string + - name: after + in: query + required: false + type: string + description: Exclusive lower bound on queue name; use next_after of the previous page. + - name: limit + in: query + required: false + type: integer + minimum: 1 + maximum: 1000 + description: Maximum number of entries per page (default 100). + responses: + "200": + description: A successful response (array of message queues with their statistics, or paging envelope when after/limit is given) + "400": + description: Bad request (invalid limit) + "401": + description: Unauthorized + "403": + description: Forbidden (wrong content type) + "500": + description: Internal server error + /messageQueue/purge/{processName}: + post: + summary: Purge a message queue + description: > + Removes every message that is currently in the named queue and answers + with the queue name and the number of removed messages + ({"NAME":"orders","PURGED":42}). The queue name is + looked up through a repository API that treats it as a pattern, so only an + exact name match is purged: a name that merely matches a queue as a + pattern (for example "mq-.*") answers 404, one call can therefore never + drain several queues. If opening the queue or deleting its messages fails, + the handler answers 500. Purging is destructive and requires a token of a + user in the "Admin-MessageQueue" group; a member of "User-MessageQueue" + alone gets 401. + security: + - bearerAuth: [] + parameters: + - name: processName + in: path + required: true + type: string + description: Exact message queue name to drain; patterns are rejected with 404. + responses: + "200": + description: Queue purged (MessageQueuePurgeDto JSON) + schema: + type: object + required: + - NAME + - PURGED + properties: + NAME: + type: string + description: Name of the purged queue + example: orders + PURGED: + type: integer + format: int32 + description: Number of messages that were removed + example: 42 + "401": + description: Unauthorized (missing token or not in the Admin-MessageQueue group) + "403": + description: Forbidden (wrong content type) + "404": + description: Unknown queue name (the name matches no queue exactly) + "500": + description: Internal server error (queue could not be opened or drained) + /messageQueue/recover: + post: + summary: Recover orphaned message queues + description: > + Reclaims the message queue entries whose recorded owner process is gone + and answers with the number of reclaimed entries ({"RECOVERED":2}). An + entry whose owner is still alive is never touched, no matter how old its + heartbeat is. Reclaiming mutates queue state and requires a token of a + user in the "Admin-MessageQueue" group; a member of "User-MessageQueue" + alone gets 401. + security: + - bearerAuth: [] + responses: + "200": + description: Entries reclaimed ({"RECOVERED":n}) + schema: + type: object + required: + - RECOVERED + properties: + RECOVERED: + type: integer + format: int64 + description: Number of reclaimed queue entries + example: 2 + "401": + description: Unauthorized (missing token or not in the Admin-MessageQueue group) + "403": + description: Forbidden (wrong content type) + "500": + description: Internal server error + /eventBus/buses: + get: + summary: List event buses + description: > + Returns every configured event bus with its validated bootstrap + configuration and, while the bus can be opened in its shared memory + segment (AVAILABLE), its runtime counters. A bus whose bootstrap + configuration is invalid is still listed, with AVAILABLE false and + without configuration fields; a bus whose segment is missing reports + AVAILABLE false but keeps its configuration. Reading requires a token + of a user in the "Admin-EventBus" or "User-EventBus" group. + security: + - bearerAuth: [] + responses: + "200": + description: All configured event buses + schema: + type: array + items: + $ref: '#/definitions/EventBusDto' + "401": + description: Unauthorized (missing token or not in an event bus group) + "403": + description: Forbidden (wrong content type) + "500": + description: Internal server error + /eventBus/bus/{name}: + get: + summary: Get one event bus + description: > + Returns the same object as one entry of /eventBus/buses for exactly one + bus. An unknown name answers 404. Reading requires a token of a user in + the "Admin-EventBus" or "User-EventBus" group. + security: + - bearerAuth: [] + parameters: + - name: name + in: path + required: true + type: string + description: Exact bus name + responses: + "200": + description: The event bus (EventBusDto JSON) + schema: + $ref: '#/definitions/EventBusDto' + "401": + description: Unauthorized (missing token or not in an event bus group) + "403": + description: Forbidden (wrong content type) + "404": + description: Unknown bus name + "500": + description: Internal server error + /eventBus/publish/{name}: + post: + summary: Publish an event on a bus + description: > + Builds an Event from the JSON body and hands it to the bus. TOPIC and + TYPE are required, PAYLOAD is an optional string of at most 256 bytes, + PRIORITY is one of Critical, High, Normal or Low (default Normal) and + TARGET is BUS (default, the point-to-point queue) or SUBSCRIBERS (the + queues of every matching subscriber). Names longer than 31 bytes and + payloads longer than 256 bytes answer 400 instead of being truncated. + Publishing requires the "Publisher-EventBus" group (or "Admin-EventBus"); + "User-EventBus" is read-only and gets 401. + A full queue is not an error: the answer carries PUBLISHED false and + the bus counts the event as dropped. With TARGET SUBSCRIBERS, PUBLISHED + is also false when no subscription matched the topic. + security: + - bearerAuth: [] + parameters: + - name: name + in: path + required: true + type: string + description: Exact bus name + - name: body + in: body + required: true + schema: + type: object + required: + - TOPIC + - TYPE + properties: + TOPIC: + type: string + maxLength: 31 + description: Topic of the event + example: orders.created + TYPE: + type: string + maxLength: 31 + description: Type of the event + example: OrderCreated + PAYLOAD: + type: string + maxLength: 256 + description: Payload of the event + example: "42" + PRIORITY: + type: string + enum: [Critical, High, Normal, Low] + description: Delivery priority inside the subscriber queues (default Normal) + TARGET: + type: string + enum: [BUS, SUBSCRIBERS] + description: BUS for the point-to-point queue (default), SUBSCRIBERS for a broadcast + responses: + "200": + description: The event was handed to the bus + schema: + type: object + required: + - BUS + - PRIORITY + - PUBLISHED + - SEQUENCE + - TARGET + - TOPIC + - TYPE + properties: + BUS: + type: string + example: orders + PRIORITY: + type: string + enum: [Critical, High, Normal, Low] + PUBLISHED: + type: boolean + description: False when every matching queue was full or no subscription matched + SEQUENCE: + type: integer + format: int64 + description: Last sequence number assigned by the bus + TARGET: + type: string + enum: [BUS, SUBSCRIBERS] + TOPIC: + type: string + TYPE: + type: string + "400": + description: Body is not json, a required member is missing, a name or the payload is too long, or PRIORITY/TARGET is unknown + "401": + description: Unauthorized (missing token or not in an event bus group) + "403": + description: Forbidden (wrong content type) + "404": + description: Unknown bus name + "500": + description: Internal server error + /eventBus/subscribe/{name}: + post: + summary: Subscribe a subscriber to a topic + description: > + Adds the named subscriber to the topic. An exact topic is registered + implicitly, so a subscriber can join a topic that was never published + before; wildcard patterns (`*` matches any sequence, `?` matches one + character) register themselves. The answer reports SUBSCRIBED false when + the topic or subscriber table of the bus is full. Subscribing requires a + token of a user in the "Admin-EventBus" group; a member of + "User-EventBus" alone gets 401. + security: + - bearerAuth: [] + parameters: + - name: name + in: path + required: true + type: string + description: Exact bus name + - name: body + in: body + required: true + schema: + type: object + required: + - TOPIC + - SUBSCRIBER + properties: + TOPIC: + type: string + maxLength: 31 + description: Topic name or wildcard pattern + example: orders.* + SUBSCRIBER: + type: string + maxLength: 31 + description: Name of the subscriber + example: worker + responses: + "200": + description: The subscription was attempted + schema: + type: object + required: + - BUS + - SUBSCRIBER + - SUBSCRIBED + - TOPIC + properties: + BUS: + type: string + SUBSCRIBER: + type: string + SUBSCRIBED: + type: boolean + description: False when the topic or subscriber table is full + TOPIC: + type: string + "400": + description: Body is not json, a required member is missing or a name is too long + "401": + description: Unauthorized (missing token or not in the Admin-EventBus group) + "403": + description: Forbidden (wrong content type) + "404": + description: Unknown bus name + "500": + description: Internal server error + delete: + summary: Unsubscribe a subscriber from a topic + description: > + Removes the subscription of the named subscriber from the topic and + answers with UNSUBSCRIBED false when there was no such subscription. + Unsubscribing requires a token of a user in the "Admin-EventBus" group. + security: + - bearerAuth: [] + parameters: + - name: name + in: path + required: true + type: string + description: Exact bus name + - name: body + in: body + required: true + schema: + type: object + required: + - TOPIC + - SUBSCRIBER + properties: + TOPIC: + type: string + maxLength: 31 + SUBSCRIBER: + type: string + maxLength: 31 + responses: + "200": + description: The subscription was removed or did not exist + schema: + type: object + required: + - BUS + - SUBSCRIBER + - TOPIC + - UNSUBSCRIBED + properties: + BUS: + type: string + SUBSCRIBER: + type: string + TOPIC: + type: string + UNSUBSCRIBED: + type: boolean + description: True when a subscription was removed + "400": + description: Body is not json, a required member is missing or a name is too long + "401": + description: Unauthorized (missing token or not in the Admin-EventBus group) + "403": + description: Forbidden (wrong content type) + "404": + description: Unknown bus name + "500": + description: Internal server error + /eventBus/depth/{name}/{subscriber}: + get: + summary: Get the queue depth of one subscriber + description: > + Returns the best-effort number of events waiting in the queues of the + named subscriber. The value is approximate while other processes push + or pop events concurrently; an unknown subscriber reports 0. Reading + requires a token of a user in the "Admin-EventBus" or "User-EventBus" + group. + security: + - bearerAuth: [] + parameters: + - name: name + in: path + required: true + type: string + description: Exact bus name + - name: subscriber + in: path + required: true + type: string + description: Subscriber name (at most 31 bytes) + responses: + "200": + description: Queue depth of the subscriber + schema: + type: object + required: + - BUS + - DEPTH + - SUBSCRIBER + properties: + BUS: + type: string + DEPTH: + type: integer + format: int64 + SUBSCRIBER: + type: string + "400": + description: Subscriber name is longer than 31 bytes + "401": + description: Unauthorized (missing token or not in an event bus group) + "403": + description: Forbidden (wrong content type) + "404": + description: Unknown bus name + "500": + description: Internal server error + /eventBus/reset/{name}: + post: + summary: Reset an event bus + description: > + Drops every queued event of the bus and of all subscriber queues and + answers with RESET true. Resetting is destructive and requires a token + of a user in the "Admin-EventBus" group; a member of "User-EventBus" + alone gets 401. + security: + - bearerAuth: [] + parameters: + - name: name + in: path + required: true + type: string + description: Exact bus name + responses: + "200": + description: The bus was reset + schema: + type: object + required: + - BUS + - RESET + properties: + BUS: + type: string + RESET: + type: boolean + "401": + description: Unauthorized (missing token or not in the Admin-EventBus group) + "403": + description: Forbidden (wrong content type) + "404": + description: Unknown bus name + "500": + description: Internal server error /process/processes/{groupName}: get: summary: Get all processes @@ -408,6 +907,179 @@ paths: description: Unknown process id "500": description: Internal server error + /shm/stats/{segmentName}: + get: + summary: Get the runtime statistics of one segment + description: > + Returns the statistics of exactly one shared memory segment as a JSON + object with the fields NAME, ALLOCATED_BYTES, CAPACITY_BYTES, + FREE_BYTES, GROW_COUNT, SHRINK_COUNT, RECOVERY_COUNT and GENERATION. + CAPACITY_BYTES is the configured maximum size, or ALLOCATED_BYTES when + the segment has no maximum. Reading requires a token of a user in the + "Admin-Shm" or "User-Shm" group; an unknown segment answers 404. + security: + - bearerAuth: [] + parameters: + - name: segmentName + in: path + required: true + type: string + description: Exact segment name + responses: + "200": + description: Segment statistics (JSON object) + schema: + type: object + required: + - NAME + - ALLOCATED_BYTES + - CAPACITY_BYTES + - FREE_BYTES + - GROW_COUNT + - SHRINK_COUNT + - RECOVERY_COUNT + - GENERATION + properties: + NAME: + type: string + description: Name of the segment + example: user + ALLOCATED_BYTES: + type: integer + format: int64 + description: Currently allocated size of the segment + example: 67108864 + CAPACITY_BYTES: + type: integer + format: int64 + description: Configured maximum size, or the allocated size when there is no maximum + example: 268435456 + FREE_BYTES: + type: integer + format: int64 + description: Free bytes inside the segment + example: 12345678 + GROW_COUNT: + type: integer + format: int64 + description: Number of grow operations on this segment + example: 2 + SHRINK_COUNT: + type: integer + format: int64 + description: Number of shrink operations on this segment + example: 0 + RECOVERY_COUNT: + type: integer + format: int64 + description: Number of reclaimed remap locks of this segment + example: 1 + GENERATION: + type: integer + format: int64 + description: Generation counter of the segment mapping + example: 3 + "401": + description: Unauthorized (missing token or neither in the Admin-Shm nor the User-Shm group) + "403": + description: Forbidden (wrong content type) + "404": + description: Unknown segment + "500": + description: Internal server error + /shm/cleanup: + post: + summary: Remove orphaned shared memory objects + description: > + Runs the cleanup service in apply mode: every registered shared memory + object whose recorded owner process is gone is removed from its segment + and answers with the counts of the run as a JSON object with the fields + DRY_RUN, SCANNED, ORPHANS, REMOVED and FAILED. DRY_RUN is always false + because this endpoint applies; the scan-only mode of the service is not + exposed. An object whose owner is still alive is never touched, no + matter how old its heartbeat is. A failed removal keeps its registration + and is retried by the next run, so FAILED may be non-zero while the + request still answers 200. Cleanup is destructive and requires a token + of a user in the "Admin-Shm" group; a member of "User-Shm" alone gets + 401. + security: + - bearerAuth: [] + responses: + "200": + description: Cleanup applied (JSON object with the run counts) + schema: + type: object + required: + - DRY_RUN + - SCANNED + - ORPHANS + - REMOVED + - FAILED + properties: + DRY_RUN: + type: boolean + description: Always false - this endpoint applies the cleanup + example: false + SCANNED: + type: integer + format: int64 + description: Number of registered objects inspected + example: 3 + ORPHANS: + type: integer + format: int64 + description: Number of orphaned objects found + example: 1 + REMOVED: + type: integer + format: int64 + description: Number of orphaned objects actually removed + example: 1 + FAILED: + type: integer + format: int64 + description: Number of removals that failed and are retried by the next run + example: 0 + "401": + description: Unauthorized (missing token or not in the Admin-Shm group) + "403": + description: Forbidden (wrong content type) + "500": + description: Internal server error + /shm/recover: + post: + summary: Reclaim the remap locks of segments whose holder is gone + description: > + Reclaims the remap semaphore of every shared memory segment this + process maps whose recorded holder process no longer exists and answers + with the number of reclaimed locks ({"RECOVERED":1}). A lock whose + holder is still alive is never touched, no matter how long it has been + held. Opening a segment already runs the same check once, so this + endpoint is the repeat an operator needs for a segment that got stuck + later on. Reclaiming changes the lock state of a segment and requires a + token of a user in the "Admin-Shm" group; a member of "User-Shm" alone + gets 401. + security: + - bearerAuth: [] + responses: + "200": + description: Locks reclaimed ({"RECOVERED":n}) + schema: + type: object + required: + - RECOVERED + properties: + RECOVERED: + type: integer + format: int64 + description: Number of reclaimed remap locks + example: 1 + "401": + description: Unauthorized (missing token or not in the Admin-Shm group) + "403": + description: Forbidden (wrong content type) + "500": + description: Internal server error /shm/segments/{segmentName}: get: summary: Get all segments @@ -916,6 +1588,85 @@ paths: "500": description: Internal server error definitions: + EventBusDto: + type: object + description: > + One configured event bus. The configuration fields are omitted while the + bootstrap entry is invalid; the runtime counters are omitted while the bus + cannot be opened in its shared memory segment. + properties: + NAME: + type: string + description: Name of the bus + example: orders + AVAILABLE: + type: boolean + description: True while the bus can be opened in its shared memory segment + BUS_CAPACITY: + type: integer + format: int64 + description: Capacity of the point-to-point bus queue + example: 128 + SUBSCRIBER_CAPACITY: + type: integer + format: int64 + description: Capacity of every per-subscriber queue + example: 128 + MAX_TOPICS: + type: integer + format: int64 + description: Maximum number of registerable topics + example: 16 + MAX_SUBSCRIBERS: + type: integer + format: int64 + description: Maximum number of registerable subscribers + example: 32 + MAX_PRIORITIES: + type: integer + format: int64 + description: Number of priority queues per subscriber + example: 4 + PUB_POLICY: + type: string + enum: [Fail, DropOldest, DropNewest, Block] + description: Drop policy when a queue is full + STICKY: + type: boolean + description: Whether sticky/last-value events are enabled + STICKY_TTL_MS: + type: integer + format: int32 + description: TTL of sticky events in milliseconds + example: 60000 + DROPPED: + type: integer + format: int64 + description: Number of events dropped because a queue was full + example: 0 + TOPIC_COUNT: + type: integer + format: int64 + description: Number of registered topics + example: 2 + SUBSCRIBER_COUNT: + type: integer + format: int64 + description: Number of reserved subscriber slots + example: 1 + BUS_DEPTH: + type: integer + format: int64 + description: Best-effort depth of the point-to-point queue + example: 0 + SEQUENCE: + type: integer + format: int64 + description: Last sequence number assigned by the bus + example: 7 + required: + - NAME + - AVAILABLE ProcessInfoDto: type: object properties: diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 39d0049a..044f1406 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -79,9 +79,11 @@ ADD_EXECUTABLE(base_tests plugins/VirtualGroupBootstrapPluginTest.cpp plugins/SharedMemoryBootstrapPluginTest.cpp plugins/SingleInstanceBootstrapPluginTest.cpp + cli/AdminCliComponentTest.cpp cli/CliParserTest.cpp cli/HistoryCliComponentTest.cpp cli/MessageQueueCliComponentTest.cpp + cli/EventBusCliComponentTest.cpp cli/ProcessCliComponentTest.cpp cli/SharedMemoryCliComponentTest.cpp cli/PropertyCliComponentTest.cpp @@ -104,6 +106,7 @@ ADD_EXECUTABLE(base_tests http/SharedMemoryControllerMockTest.cpp http/HistoryControllerTest.cpp http/MessageQueueControllerTest.cpp + http/EventBusControllerTest.cpp http/UserControllerTest.cpp http/ProcessControllerTest.cpp http/AdminControllerTest.cpp diff --git a/tests/cli/AdminCliComponentTest.cpp b/tests/cli/AdminCliComponentTest.cpp new file mode 100644 index 00000000..eaa72ecb --- /dev/null +++ b/tests/cli/AdminCliComponentTest.cpp @@ -0,0 +1,384 @@ +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "lintel/core/configuration/Configuration.h" +#include "lintel/core/configuration/EnvironmentConfiguration.h" +#include "lintel/core/utils/TypeName.h" +#include "lintel/features/base/controller/UserDto.h" +#include "lintel/features/cli/components/AdminCliComponent.h" +#include "lintel/features/http/configuration/ClientConfiguration.h" +#include "lintel/features/http/configuration/HttpComponent.h" +#include "lintel/features/http/configuration/HttpEntry.h" +#include "lintel/features/http/provider/ClientProvider.h" + +namespace lintel { +namespace { + + // `admCli` prefix: unity builds merge several test files into one anonymous + // namespace, so a generic capture or backend name would collide. + class admCliCoutCapture + { + public: + admCliCoutCapture() : m_oldBuffer(std::cout.rdbuf(m_stream.rdbuf())) {} + + ~admCliCoutCapture() { std::cout.rdbuf(m_oldBuffer); } + + admCliCoutCapture(const admCliCoutCapture &) = delete; + admCliCoutCapture &operator=(const admCliCoutCapture &) = delete; + + [[nodiscard]] std::string str() const { return m_stream.str(); } + + private: + std::ostringstream m_stream; + std::streambuf *m_oldBuffer; + }; + + class admCliCerrCapture + { + public: + admCliCerrCapture() : m_oldBuffer(std::cerr.rdbuf(m_stream.rdbuf())) {} + + ~admCliCerrCapture() { std::cerr.rdbuf(m_oldBuffer); } + + admCliCerrCapture(const admCliCerrCapture &) = delete; + admCliCerrCapture &operator=(const admCliCerrCapture &) = delete; + + [[nodiscard]] std::string str() const { return m_stream.str(); } + + private: + std::ostringstream m_stream; + std::streambuf *m_oldBuffer; + }; + + /** + * `AdminApi` has no virtual seam, so the component is driven against a + * loopback HTTP server that records every request and answers with a canned + * body. + */ + class admCliBackend + { + public: + admCliBackend() + { + // Record in the route handlers, not in a pre-routing handler: the + // pre-routing hook runs before httplib has read the request body, so it + // only ever sees an empty request.body. + const auto record = [this](const httplib::Request &request, httplib::Response &response) { + paths.push_back(request.path); + methods.emplace_back(request.method); + bodies.push_back(request.body); + response.status = m_status; + response.set_content(m_responseBody, "application/json"); + }; + m_server.Get(".*", record); + m_server.Post(".*", record); + m_server.Put(".*", record); + m_server.Delete(".*", record); + m_port = m_server.bind_to_any_port("127.0.0.1"); + m_thread = std::thread([this]() { m_server.listen_after_bind(); }); + // Server::stop() is a no-op until the accept loop is up. + const auto admCliDeadline = std::chrono::steady_clock::now() + std::chrono::seconds(5); + while (!m_server.is_running() && std::chrono::steady_clock::now() < admCliDeadline) { + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + } + + ~admCliBackend() + { + m_server.stop(); + if (m_thread.joinable()) { m_thread.join(); } + } + + admCliBackend(const admCliBackend &) = delete; + admCliBackend &operator=(const admCliBackend &) = delete; + + void answerWith(const std::string &body) { m_responseBody = body; } + + void setStatus(int status) { m_status = status; } + + [[nodiscard]] int port() const { return m_port; } + + [[nodiscard]] bool running() const { return m_server.is_running(); } + + std::vector paths; + std::vector methods; + std::vector bodies; + + private: + httplib::Server m_server; + std::thread m_thread; + int m_port{ -1 }; + std::string m_responseBody{ "{}" }; + int m_status{ 200 }; + }; + + /** Client provider whose only client entry points at the loopback backend. */ + std::shared_ptr admCliProviderOf(int port) + { + static const std::string kAdmCliAbsentConfigDir = "/nonexistent/lintel-admin-cli-test-config"; + auto environment = std::make_shared(); + environment->overrides(EnvironmentConfiguration::ConfigDirectory, kAdmCliAbsentConfigDir); + auto clientConfiguration = std::make_shared("127.0.0.1", + port, + std::chrono::milliseconds(2000), + std::chrono::milliseconds(2000), + std::chrono::milliseconds(2000)); + std::vector> entries{ std::make_shared( + type_name(), clientConfiguration) }; + auto configuration = std::make_shared(std::vector>{}, environment); + configuration->setEntries(entries); + return std::make_shared(configuration); + } + + /** @return the JSON body AdminController would send for a successful reload */ + std::string admCliReloadedBody() { return R"({"status":"reloaded","results":{"tls":"ok"}})"; } + +}// namespace + +TEST_CASE("AdminCliComponent: onShowMenu prints the no-submenu notice") +{ + auto admCliApi = std::make_shared(admCliProviderOf(1)); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.onShowMenu(); + + REQUIRE(admCliOut.str().find("No submenu available!") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: onMenu returns false for an unknown component") +{ + auto admCliApi = std::make_shared(admCliProviderOf(1)); + AdminCliComponent admCliComponent(admCliApi); + + // CommandLineService only prints "Invalid command" when the component reports false. + REQUIRE(admCliComponent.onMenu("History") == false); +} + +TEST_CASE("AdminCliComponent: onHelp prints the header and every command with its arguments") +{ + auto admCliApi = std::make_shared(admCliProviderOf(1)); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.onHelp(); + const std::string admCliHelp = admCliOut.str(); + + REQUIRE(admCliHelp.find("Help Menu of Admin [Adm]:") != std::string::npos); + REQUIRE(admCliHelp.find("reload_tls") != std::string::npos); + REQUIRE(admCliHelp.find("log_level") != std::string::npos); + REQUIRE(admCliHelp.find("--level, -l") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: printCommandList lists the registered commands") +{ + auto admCliApi = std::make_shared(admCliProviderOf(1)); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.printCommandList({}); + const std::string admCliList = admCliOut.str(); + + REQUIRE(admCliList.find("reload_tls") != std::string::npos); + REQUIRE(admCliList.find("log_level") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: printCommandList appends the sub-menu aliases") +{ + auto admCliApi = std::make_shared(admCliProviderOf(1)); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.printCommandList({ "exit", "help" }); + const std::string admCliList = admCliOut.str(); + + REQUIRE(admCliList.find("reload_tls") != std::string::npos); + REQUIRE(admCliList.find("exit") != std::string::npos); + REQUIRE(admCliList.find("help") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: allCommandsOf returns the registered commands") +{ + auto admCliApi = std::make_shared(admCliProviderOf(1)); + AdminCliComponent admCliComponent(admCliApi); + + const std::vector admCliCommands = admCliComponent.allCommandsOf(); + const auto admCliHas = [&admCliCommands](const std::string &command) { + return std::find(admCliCommands.begin(), admCliCommands.end(), command) != admCliCommands.end(); + }; + + REQUIRE(admCliCommands.size() == 2); + REQUIRE(admCliHas("reload_tls")); + REQUIRE(admCliHas("log_level")); +} + +TEST_CASE("AdminCliComponent: the component registers itself under name and alias") +{ + auto admCliApi = std::make_shared(admCliProviderOf(1)); + AdminCliComponent admCliComponent(admCliApi); + + REQUIRE(admCliComponent.getName() == "Admin"); + REQUIRE(admCliComponent.getAlias() == "Adm"); + REQUIRE(admCliComponent.onExit() == true); +} + +TEST_CASE("AdminCliComponent: reload_tls posts to the admin route and prints every resource") +{ + admCliBackend admCliServer; + REQUIRE(admCliServer.port() > 0); + REQUIRE(admCliServer.running()); + admCliServer.answerWith(admCliReloadedBody()); + auto admCliApi = std::make_shared(admCliProviderOf(admCliServer.port())); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.onCommand(UserDto{}, "reload_tls", {}); + const std::string admCliPrinted = admCliOut.str(); + + REQUIRE(admCliServer.paths.size() == 1); + REQUIRE(admCliServer.paths[0] == "/admin/reload-tls"); + REQUIRE(admCliServer.methods[0] == "POST"); + REQUIRE(admCliServer.bodies[0].empty()); + REQUIRE(admCliPrinted.find("tls: ok") != std::string::npos); + REQUIRE(admCliPrinted.find("Reloaded the TLS certificate") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: reload_tls reports that there is nothing to reload") +{ + admCliBackend admCliServer; + admCliServer.setStatus(409); + admCliServer.answerWith(R"({"status":"nothing_to_reload","results":{}})"); + auto admCliApi = std::make_shared(admCliProviderOf(admCliServer.port())); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.onCommand(UserDto{}, "reload_tls", {}); + const std::string admCliPrinted = admCliOut.str(); + + REQUIRE(admCliServer.paths.size() == 1); + REQUIRE(admCliPrinted.find("ERROR") != std::string::npos); + REQUIRE(admCliPrinted.find("nothing to reload") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: reload_tls prints the failure reason and the failed count") +{ + admCliBackend admCliServer; + admCliServer.setStatus(500); + admCliServer.answerWith(R"({"status":"failed","results":{"tls":"failed: key does not match","audit":"ok"}})"); + auto admCliApi = std::make_shared(admCliProviderOf(admCliServer.port())); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.onCommand(UserDto{}, "reload_tls", {}); + const std::string admCliPrinted = admCliOut.str(); + + REQUIRE(admCliPrinted.find("tls: failed: key does not match") != std::string::npos); + REQUIRE(admCliPrinted.find("audit: ok") != std::string::npos); + REQUIRE(admCliPrinted.find("1 of 2") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: reload_tls reports an unauthorized token") +{ + admCliBackend admCliServer; + admCliServer.setStatus(401); + admCliServer.answerWith(""); + auto admCliApi = std::make_shared(admCliProviderOf(admCliServer.port())); + AdminCliComponent admCliComponent(admCliApi); + + admCliCerrCapture admCliErr; + admCliComponent.onCommand(UserDto{}, "reload_tls", {}); + const std::string admCliReported = admCliErr.str(); + + REQUIRE(admCliServer.paths.size() == 1); + REQUIRE(admCliReported.find("ERROR") != std::string::npos); + REQUIRE(admCliReported.find("Unauthorized") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: log_level posts the level and prints what the controller applied") +{ + admCliBackend admCliServer; + admCliServer.answerWith(R"({"status":"changed","level":"debug"})"); + auto admCliApi = std::make_shared(admCliProviderOf(admCliServer.port())); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.onCommand(UserDto{}, "log_level", { "--level", "debug" }); + const std::string admCliPrinted = admCliOut.str(); + + REQUIRE(admCliServer.paths.size() == 1); + REQUIRE(admCliServer.paths[0] == "/admin/log-level"); + REQUIRE(admCliServer.methods[0] == "POST"); + REQUIRE(admCliServer.bodies[0] == R"({"level":"debug"})"); + REQUIRE(admCliPrinted.find("Log level changed to debug") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: log_level accepts the short flag") +{ + admCliBackend admCliServer; + admCliServer.answerWith(R"({"status":"changed","level":"warn"})"); + auto admCliApi = std::make_shared(admCliProviderOf(admCliServer.port())); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.onCommand(UserDto{}, "log_level", { "-l", "warn" }); + + REQUIRE(admCliServer.bodies[0] == R"({"level":"warn"})"); + REQUIRE(admCliOut.str().find("Log level changed to warn") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: log_level without a level prints an error and never reaches the api") +{ + admCliBackend admCliServer; + auto admCliApi = std::make_shared(admCliProviderOf(admCliServer.port())); + AdminCliComponent admCliComponent(admCliApi); + + admCliCerrCapture admCliErr; + admCliComponent.onCommand(UserDto{}, "log_level", {}); + const std::string admCliReported = admCliErr.str(); + + REQUIRE(admCliServer.paths.empty()); + REQUIRE(admCliReported.find("ERROR") != std::string::npos); + REQUIRE(admCliReported.find("--level") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: log_level reports a rejected level") +{ + admCliBackend admCliServer; + admCliServer.setStatus(400); + admCliServer.answerWith(R"({"status":"invalid_level","level":"verbose"})"); + auto admCliApi = std::make_shared(admCliProviderOf(admCliServer.port())); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.onCommand(UserDto{}, "log_level", { "-l", "verbose" }); + const std::string admCliPrinted = admCliOut.str(); + + REQUIRE(admCliPrinted.find("ERROR") != std::string::npos); + REQUIRE(admCliPrinted.find("verbose") != std::string::npos); + REQUIRE(admCliPrinted.find("invalid_level") != std::string::npos); +} + +TEST_CASE("AdminCliComponent: an unknown command never reaches the api") +{ + admCliBackend admCliServer; + auto admCliApi = std::make_shared(admCliProviderOf(admCliServer.port())); + AdminCliComponent admCliComponent(admCliApi); + + admCliCoutCapture admCliOut; + admCliComponent.onCommand(UserDto{}, "reload_certificate", { "-l", "ignored" }); + + REQUIRE(admCliServer.paths.empty()); + REQUIRE(admCliOut.str().empty()); +} +}// namespace lintel diff --git a/tests/cli/EventBusCliComponentTest.cpp b/tests/cli/EventBusCliComponentTest.cpp new file mode 100644 index 00000000..ddc6ef94 --- /dev/null +++ b/tests/cli/EventBusCliComponentTest.cpp @@ -0,0 +1,430 @@ +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "lintel/core/configuration/Configuration.h" +#include "lintel/core/configuration/EnvironmentConfiguration.h" +#include "lintel/core/utils/TypeName.h" +#include "lintel/features/base/controller/UserDto.h" +#include "lintel/features/cli/components/EventBusCliComponent.h" +#include "lintel/features/http/configuration/ClientConfiguration.h" +#include "lintel/features/http/configuration/HttpComponent.h" +#include "lintel/features/http/configuration/HttpEntry.h" +#include "lintel/features/http/provider/ClientProvider.h" + +namespace lintel { +namespace { + + // `ebCli` prefix: unity builds merge several test files into one anonymous + // namespace, so a generic capture or backend name would collide. + class ebCliCoutCapture + { + public: + ebCliCoutCapture() : m_oldBuffer(std::cout.rdbuf(m_stream.rdbuf())) {} + + ~ebCliCoutCapture() { std::cout.rdbuf(m_oldBuffer); } + + ebCliCoutCapture(const ebCliCoutCapture &) = delete; + ebCliCoutCapture &operator=(const ebCliCoutCapture &) = delete; + + [[nodiscard]] std::string str() const { return m_stream.str(); } + + private: + std::ostringstream m_stream; + std::streambuf *m_oldBuffer; + }; + + class ebCliCerrCapture + { + public: + ebCliCerrCapture() : m_oldBuffer(std::cerr.rdbuf(m_stream.rdbuf())) {} + + ~ebCliCerrCapture() { std::cerr.rdbuf(m_oldBuffer); } + + ebCliCerrCapture(const ebCliCerrCapture &) = delete; + ebCliCerrCapture &operator=(const ebCliCerrCapture &) = delete; + + [[nodiscard]] std::string str() const { return m_stream.str(); } + + private: + std::ostringstream m_stream; + std::streambuf *m_oldBuffer; + }; + + /** + * `EventBusApi` has no virtual seam, so the component is driven against a + * loopback HTTP server that records every request and answers with a canned + * body. + */ + class ebCliBackend + { + public: + ebCliBackend() + { + // Record in the route handlers, not in a pre-routing handler: the + // pre-routing hook runs before httplib has read the request body, so it + // only ever sees an empty request.body. + const auto record = [this](const httplib::Request &request, httplib::Response &response) { + paths.push_back(request.path); + methods.emplace_back(request.method); + bodies.push_back(request.body); + response.status = m_status; + response.set_content(m_responseBody, "application/json"); + }; + m_server.Get(".*", record); + m_server.Post(".*", record); + m_server.Put(".*", record); + m_server.Delete(".*", record); + m_port = m_server.bind_to_any_port("127.0.0.1"); + m_thread = std::thread([this]() { m_server.listen_after_bind(); }); + // Server::stop() is a no-op until the accept loop is up. + const auto ebCliDeadline = std::chrono::steady_clock::now() + std::chrono::seconds(5); + while (!m_server.is_running() && std::chrono::steady_clock::now() < ebCliDeadline) { + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + } + + ~ebCliBackend() + { + m_server.stop(); + if (m_thread.joinable()) { m_thread.join(); } + } + + ebCliBackend(const ebCliBackend &) = delete; + ebCliBackend &operator=(const ebCliBackend &) = delete; + + void answerWith(const std::string &body) { m_responseBody = body; } + + void setStatus(int status) { m_status = status; } + + [[nodiscard]] int port() const { return m_port; } + + [[nodiscard]] bool running() const { return m_server.is_running(); } + + std::vector paths; + std::vector methods; + std::vector bodies; + + private: + httplib::Server m_server; + std::thread m_thread; + int m_port{ -1 }; + std::string m_responseBody{ "[]" }; + int m_status{ 200 }; + }; + + /** Client provider whose only client entry points at the loopback backend. */ + std::shared_ptr ebCliProviderOf(int port) + { + static const std::string kEbCliAbsentConfigDir = "/nonexistent/lintel-eb-cli-test-config"; + auto environment = std::make_shared(); + environment->overrides(EnvironmentConfiguration::ConfigDirectory, kEbCliAbsentConfigDir); + auto clientConfiguration = std::make_shared("127.0.0.1", + port, + std::chrono::milliseconds(2000), + std::chrono::milliseconds(2000), + std::chrono::milliseconds(2000)); + std::vector> entries{ std::make_shared( + type_name(), clientConfiguration) }; + auto configuration = std::make_shared(std::vector>{}, environment); + configuration->setEntries(entries); + return std::make_shared(configuration); + } + + constexpr char ebCliOneBusBody[] = R"([{)" + R"("NAME":"orders",)" + R"("AVAILABLE":true,)" + R"("BUS_CAPACITY":8,)" + R"("SUBSCRIBER_CAPACITY":8,)" + R"("MAX_TOPICS":8,)" + R"("MAX_SUBSCRIBERS":8,)" + R"("MAX_PRIORITIES":4,)" + R"("PUB_POLICY":"Fail",)" + R"("STICKY":false,)" + R"("STICKY_TTL_MS":60000,)" + R"("DROPPED":3,)" + R"("TOPIC_COUNT":2,)" + R"("SUBSCRIBER_COUNT":1,)" + R"("BUS_DEPTH":0,)" + R"("SEQUENCE":7)" + R"(}])"; + +}// namespace + +TEST_CASE("EventBusCliComponent: onShowMenu prints the no-submenu notice") +{ + auto ebCliApi = std::make_shared(ebCliProviderOf(1)); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onShowMenu(); + + REQUIRE(ebCliOut.str().find("No submenu available!") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: onMenu returns false for an unknown component") +{ + auto ebCliApi = std::make_shared(ebCliProviderOf(1)); + EventBusCliComponent ebCliComponent(ebCliApi); + + REQUIRE(ebCliComponent.onMenu("History") == false); +} + +TEST_CASE("EventBusCliComponent: onHelp prints the header and every command") +{ + auto ebCliApi = std::make_shared(ebCliProviderOf(1)); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onHelp(); + const std::string ebCliHelp = ebCliOut.str(); + + REQUIRE(ebCliHelp.find("Help Menu of EventBus [Eb]:") != std::string::npos); + REQUIRE(ebCliHelp.find("show_event_buses") != std::string::npos); + REQUIRE(ebCliHelp.find("publish_event") != std::string::npos); + REQUIRE(ebCliHelp.find("event_queue_depth") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: allCommandsOf returns the registered commands") +{ + auto ebCliApi = std::make_shared(ebCliProviderOf(1)); + EventBusCliComponent ebCliComponent(ebCliApi); + + const std::vector ebCliCommands = ebCliComponent.allCommandsOf(); + const auto ebCliHas = [&ebCliCommands](const std::string &command) { + return std::find(ebCliCommands.begin(), ebCliCommands.end(), command) != ebCliCommands.end(); + }; + + REQUIRE(ebCliCommands.size() == 7); + REQUIRE(ebCliHas("show_event_buses")); + REQUIRE(ebCliHas("show_event_bus")); + REQUIRE(ebCliHas("publish_event")); + REQUIRE(ebCliHas("subscribe_event")); + REQUIRE(ebCliHas("unsubscribe_event")); + REQUIRE(ebCliHas("event_queue_depth")); + REQUIRE(ebCliHas("reset_event_bus")); +} + +TEST_CASE("EventBusCliComponent: the component registers itself under name and alias") +{ + auto ebCliApi = std::make_shared(ebCliProviderOf(1)); + EventBusCliComponent ebCliComponent(ebCliApi); + + REQUIRE(ebCliComponent.getName() == "EventBus"); + REQUIRE(ebCliComponent.getAlias() == "Eb"); + REQUIRE(ebCliComponent.onExit() == true); +} + +TEST_CASE("EventBusCliComponent: show_event_buses prints the configuration and the counters") +{ + ebCliBackend ebCliServer; + REQUIRE(ebCliServer.running()); + ebCliServer.answerWith(ebCliOneBusBody); + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand(UserDto{}, "show_event_buses", {}); + const std::string ebCliPrinted = ebCliOut.str(); + + REQUIRE(ebCliServer.paths.size() == 1); + REQUIRE(ebCliServer.paths[0] == "/eventBus/buses"); + REQUIRE(ebCliServer.methods[0] == "GET"); + REQUIRE(ebCliPrinted.find("BusCapacity") != std::string::npos); + REQUIRE(ebCliPrinted.find("orders") != std::string::npos); + REQUIRE(ebCliPrinted.find("Fail") != std::string::npos); + REQUIRE(ebCliPrinted.find("yes") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: show_event_bus asks for one bus") +{ + ebCliBackend ebCliServer; + ebCliServer.answerWith(R"({"NAME":"orders","AVAILABLE":true,"BUS_CAPACITY":8,"DROPPED":1})"); + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand(UserDto{}, "show_event_bus", { "--bus", "orders" }); + const std::string ebCliPrinted = ebCliOut.str(); + + REQUIRE(ebCliServer.paths.size() == 1); + REQUIRE(ebCliServer.paths[0] == "/eventBus/bus/orders"); + REQUIRE(ebCliPrinted.find("orders") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: show_event_bus reports an unknown bus") +{ + ebCliBackend ebCliServer; + ebCliServer.setStatus(404); + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand(UserDto{}, "show_event_bus", { "-b", "unknown" }); + + REQUIRE(ebCliOut.str().find("ERROR") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: publish_event posts the event and prints the result") +{ + ebCliBackend ebCliServer; + ebCliServer.answerWith(R"({"BUS":"orders","PRIORITY":"High","PUBLISHED":true,"SEQUENCE":7,)" + R"("TARGET":"BUS","TOPIC":"orders.created","TYPE":"OrderCreated"})"); + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand(UserDto{}, + "publish_event", + { "--bus", + "orders", + "--topic", + "orders.created", + "--type", + "OrderCreated", + "--payload", + "42", + "--priority", + "High" }); + const std::string ebCliPrinted = ebCliOut.str(); + + REQUIRE(ebCliServer.paths.size() == 1); + REQUIRE(ebCliServer.paths[0] == "/eventBus/publish/orders"); + REQUIRE(ebCliServer.methods[0] == "POST"); + REQUIRE(ebCliServer.bodies[0].find("\"TOPIC\":\"orders.created\"") != std::string::npos); + REQUIRE(ebCliServer.bodies[0].find("\"TYPE\":\"OrderCreated\"") != std::string::npos); + REQUIRE(ebCliServer.bodies[0].find("\"PAYLOAD\":\"42\"") != std::string::npos); + REQUIRE(ebCliServer.bodies[0].find("\"PRIORITY\":\"High\"") != std::string::npos); + REQUIRE(ebCliServer.bodies[0].find("\"TARGET\":\"BUS\"") != std::string::npos); + REQUIRE(ebCliPrinted.find("Published event orders.created/OrderCreated on event bus orders") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: publish_event --to-subscribers broadcasts") +{ + ebCliBackend ebCliServer; + ebCliServer.answerWith(R"({"PUBLISHED":true})"); + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand( + UserDto{}, "publish_event", { "-b", "orders", "-t", "orders.created", "-ty", "OrderCreated", "-ts" }); + + REQUIRE(ebCliServer.bodies[0].find("\"TARGET\":\"SUBSCRIBERS\"") != std::string::npos); + REQUIRE(ebCliOut.str().find("Published event") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: publish_event without a topic prints an error") +{ + ebCliBackend ebCliServer; + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCerrCapture ebCliErr; + ebCliComponent.onCommand(UserDto{}, "publish_event", { "--bus", "orders", "--type", "OrderCreated" }); + + REQUIRE(ebCliServer.paths.empty()); + REQUIRE(ebCliErr.str().find("ERROR") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: publish_event reports a dropped event") +{ + ebCliBackend ebCliServer; + ebCliServer.answerWith(R"({"PUBLISHED":false})"); + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand(UserDto{}, "publish_event", { "-b", "orders", "-t", "t", "-ty", "T" }); + + REQUIRE(ebCliOut.str().find("was dropped") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: subscribe_event posts the subscription") +{ + ebCliBackend ebCliServer; + ebCliServer.answerWith(R"({"BUS":"orders","SUBSCRIBER":"worker","SUBSCRIBED":true,"TOPIC":"orders.created"})"); + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand( + UserDto{}, "subscribe_event", { "--bus", "orders", "--topic", "orders.created", "--subscriber", "worker" }); + + REQUIRE(ebCliServer.paths.size() == 1); + REQUIRE(ebCliServer.paths[0] == "/eventBus/subscribe/orders"); + REQUIRE(ebCliServer.methods[0] == "POST"); + REQUIRE(ebCliServer.bodies[0].find("\"SUBSCRIBER\":\"worker\"") != std::string::npos); + REQUIRE(ebCliOut.str().find("Subscribed worker to orders.created on event bus orders") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: unsubscribe_event deletes the subscription") +{ + ebCliBackend ebCliServer; + ebCliServer.answerWith(R"({"BUS":"orders","SUBSCRIBER":"worker","TOPIC":"orders.created","UNSUBSCRIBED":true})"); + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand(UserDto{}, "unsubscribe_event", { "-b", "orders", "-t", "orders.created", "-s", "worker" }); + + REQUIRE(ebCliServer.paths.size() == 1); + REQUIRE(ebCliServer.paths[0] == "/eventBus/subscribe/orders"); + REQUIRE(ebCliServer.methods[0] == "DELETE"); + REQUIRE(ebCliOut.str().find("Unsubscribed worker from orders.created on event bus orders") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: event_queue_depth prints the depth") +{ + ebCliBackend ebCliServer; + ebCliServer.answerWith(R"({"BUS":"orders","DEPTH":2,"SUBSCRIBER":"worker"})"); + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand(UserDto{}, "event_queue_depth", { "--bus", "orders", "--subscriber", "worker" }); + + REQUIRE(ebCliServer.paths.size() == 1); + REQUIRE(ebCliServer.paths[0] == "/eventBus/depth/orders/worker"); + REQUIRE(ebCliOut.str().find("Depth of subscriber worker on event bus orders: 2") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: reset_event_bus posts the reset") +{ + ebCliBackend ebCliServer; + ebCliServer.answerWith(R"({"BUS":"orders","RESET":true})"); + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand(UserDto{}, "reset_event_bus", { "-b", "orders" }); + + REQUIRE(ebCliServer.paths.size() == 1); + REQUIRE(ebCliServer.paths[0] == "/eventBus/reset/orders"); + REQUIRE(ebCliServer.methods[0] == "POST"); + REQUIRE(ebCliOut.str().find("Reset event bus orders") != std::string::npos); +} + +TEST_CASE("EventBusCliComponent: an unknown command never reaches the api") +{ + ebCliBackend ebCliServer; + auto ebCliApi = std::make_shared(ebCliProviderOf(ebCliServer.port())); + EventBusCliComponent ebCliComponent(ebCliApi); + + ebCliCoutCapture ebCliOut; + ebCliComponent.onCommand(UserDto{}, "show_buses", { "-b", "ignored" }); + + REQUIRE(ebCliServer.paths.empty()); + REQUIRE(ebCliOut.str().empty()); +} +}// namespace lintel diff --git a/tests/cli/HistoryCliComponentTest.cpp b/tests/cli/HistoryCliComponentTest.cpp index 8dae5457..26c55bf3 100644 --- a/tests/cli/HistoryCliComponentTest.cpp +++ b/tests/cli/HistoryCliComponentTest.cpp @@ -204,6 +204,9 @@ TEST_CASE("HistoryCliComponent: onHelp prints the header and every command with REQUIRE(histCliHelp.find("--process-name, -p") != std::string::npos); REQUIRE(histCliHelp.find("--service-name, -s") != std::string::npos); REQUIRE(histCliHelp.find("--label, -l") != std::string::npos); + // the argument description must not end with a stray closing parenthesis + REQUIRE(histCliHelp.find("Service Name of history entries") != std::string::npos); + REQUIRE(histCliHelp.find("history entries)") == std::string::npos); } TEST_CASE("HistoryCliComponent: printCommandList lists the registered commands") diff --git a/tests/cli/MessageQueueCliComponentTest.cpp b/tests/cli/MessageQueueCliComponentTest.cpp index 16e984ad..fe947d04 100644 --- a/tests/cli/MessageQueueCliComponentTest.cpp +++ b/tests/cli/MessageQueueCliComponentTest.cpp @@ -231,10 +231,13 @@ TEST_CASE("MessageQueueCliComponent: allCommandsOf returns the registered comman std::find(mqCliCommands.begin(), mqCliCommands.end(), "show_message_queues") != mqCliCommands.end(); const bool mqCliHasPurgeMessageQueue = std::find(mqCliCommands.begin(), mqCliCommands.end(), "purge_message_queue") != mqCliCommands.end(); + const bool mqCliHasRecoverMessageQueues = + std::find(mqCliCommands.begin(), mqCliCommands.end(), "recover_message_queues") != mqCliCommands.end(); - REQUIRE(mqCliCommands.size() == 2); + REQUIRE(mqCliCommands.size() == 3); REQUIRE(mqCliHasShowMessageQueues); REQUIRE(mqCliHasPurgeMessageQueue); + REQUIRE(mqCliHasRecoverMessageQueues); } TEST_CASE("MessageQueueCliComponent: the component registers itself under name and alias") @@ -469,4 +472,36 @@ TEST_CASE("MessageQueueCliComponent: purge_message_queue reports a rejected purg REQUIRE(mqCliPrinted.find("ERROR") != std::string::npos); REQUIRE(mqCliPrinted.find("unknown") != std::string::npos); } + +TEST_CASE("MessageQueueCliComponent: recover_message_queues posts and prints the reclaimed count") +{ + mqCliBackend mqCliServer; + mqCliServer.answerWith(R"({"RECOVERED":2})"); + auto mqCliApi = std::make_shared(mqCliProviderOf(mqCliServer.port())); + MessageQueueCliComponent mqCliComponent(mqCliApi); + + mqCliCoutCapture mqCliOut; + mqCliComponent.onCommand(UserDto{}, "recover_message_queues", {}); + const std::string mqCliPrinted = mqCliOut.str(); + + REQUIRE(mqCliServer.paths.size() == 1); + REQUIRE(mqCliServer.paths[0] == "/messageQueue/recover"); + REQUIRE(mqCliServer.methods[0] == "POST"); + REQUIRE(mqCliPrinted.find("Recovered 2") != std::string::npos); +} + +TEST_CASE("MessageQueueCliComponent: recover_message_queues reports a rejected recover") +{ + mqCliBackend mqCliServer; + mqCliServer.answerWith(R"({"RECOVERED":0})"); + mqCliServer.setStatus(403); + auto mqCliApi = std::make_shared(mqCliProviderOf(mqCliServer.port())); + MessageQueueCliComponent mqCliComponent(mqCliApi); + + mqCliCoutCapture mqCliOut; + mqCliComponent.onCommand(UserDto{}, "recover_message_queues", {}); + const std::string mqCliPrinted = mqCliOut.str(); + + REQUIRE(mqCliPrinted.find("ERROR") != std::string::npos); +} }// namespace lintel \ No newline at end of file diff --git a/tests/cli/PropertyCliComponentTest.cpp b/tests/cli/PropertyCliComponentTest.cpp index ce0eee91..9200aa4b 100644 --- a/tests/cli/PropertyCliComponentTest.cpp +++ b/tests/cli/PropertyCliComponentTest.cpp @@ -195,6 +195,11 @@ TEST_CASE("PropertyCliComponent: onHelp prints the header and every command") REQUIRE(help.find("show_properties") != std::string::npos); REQUIRE(help.find("show_property") != std::string::npos); REQUIRE(help.find("update_property") != std::string::npos); + // all three commands must use the same dash spelling for the option + REQUIRE(help.find("--process-name, -p") != std::string::npos); + REQUIRE(help.find("--class-name, -c") != std::string::npos); + REQUIRE(help.find("--process_name") == std::string::npos); + REQUIRE(help.find("--class_name") == std::string::npos); } TEST_CASE("PropertyCliComponent: printCommandList lists every registered command") @@ -288,7 +293,7 @@ TEST_CASE("PropertyCliComponent: show_property prints the requested property") propCliCoutCapture out; component.onCommand(UserDto{}, "show_property", - { "--process_name", "proc", "--class_name", "cls", "--instance-name", "inst", "--name", "myProp" }); + { "--process-name", "proc", "--class-name", "cls", "--instance-name", "inst", "--name", "myProp" }); const std::string printed = out.str(); REQUIRE(server.paths.size() == 1); @@ -308,7 +313,7 @@ TEST_CASE("PropertyCliComponent: show_property reports an absent property on std propCliCerrCapture err; component.onCommand(UserDto{}, "show_property", - { "--process_name", "proc", "--class_name", "cls", "--instance-name", "inst", "--name", "missing" }); + { "--process-name", "proc", "--class-name", "cls", "--instance-name", "inst", "--name", "missing" }); REQUIRE(err.str().find("not preset") != std::string::npos); } @@ -324,9 +329,9 @@ TEST_CASE("PropertyCliComponent: update_property updates and prints before/after propCliCoutCapture out; component.onCommand(UserDto{}, "update_property", - { "--process_name", + { "--process-name", "proc", - "--class_name", + "--class-name", "cls", "--instance-name", "inst", @@ -356,9 +361,9 @@ TEST_CASE("PropertyCliComponent: update_property reports an absent property on s propCliCerrCapture err; component.onCommand(UserDto{}, "update_property", - { "--process_name", + { "--process-name", "proc", - "--class_name", + "--class-name", "cls", "--instance-name", "inst", diff --git a/tests/cli/SharedMemoryCliComponentTest.cpp b/tests/cli/SharedMemoryCliComponentTest.cpp index 26f30645..c052601b 100644 --- a/tests/cli/SharedMemoryCliComponentTest.cpp +++ b/tests/cli/SharedMemoryCliComponentTest.cpp @@ -2,6 +2,8 @@ #include #include +#include +#include #include #include #include @@ -209,9 +211,14 @@ TEST_CASE("SharedMemoryCliComponent: onHelp prints the header and every command REQUIRE(shmCliHelp.find("shrink_segment") != std::string::npos); REQUIRE(shmCliHelp.find("grow_segment") != std::string::npos); REQUIRE(shmCliHelp.find("export_repository") != std::string::npos); + REQUIRE(shmCliHelp.find("recover_segments") != std::string::npos); REQUIRE(shmCliHelp.find("--segment-name, -s") != std::string::npos); REQUIRE(shmCliHelp.find("--grow-size, -g") != std::string::npos); REQUIRE(shmCliHelp.find("--uuid, -u") != std::string::npos); + REQUIRE(shmCliHelp.find("--file, -f") != std::string::npos); + // grow_segment must not be described as the opposite operation + REQUIRE(shmCliHelp.find("Grow segment") != std::string::npos); + REQUIRE(shmCliHelp.find("Shrink segment") != std::string::npos); } TEST_CASE("SharedMemoryCliComponent: printCommandList lists every registered command") @@ -228,6 +235,7 @@ TEST_CASE("SharedMemoryCliComponent: printCommandList lists every registered com REQUIRE(shmCliList.find("shrink_segment") != std::string::npos); REQUIRE(shmCliList.find("grow_segment") != std::string::npos); REQUIRE(shmCliList.find("export_repository") != std::string::npos); + REQUIRE(shmCliList.find("recover_segments") != std::string::npos); } TEST_CASE("SharedMemoryCliComponent: printCommandList appends the sub-menu aliases") @@ -244,20 +252,111 @@ TEST_CASE("SharedMemoryCliComponent: printCommandList appends the sub-menu alias REQUIRE(shmCliList.find("help") != std::string::npos); } -TEST_CASE("SharedMemoryCliComponent: allCommandsOf returns the five registered commands") +TEST_CASE("SharedMemoryCliComponent: show_stats forwards the segment name and prints the statistics") +{ + shmCliBackend shmCliServer; + shmCliServer.answerWith(R"({"NAME":"stats-seg","ALLOCATED_BYTES":1024,"CAPACITY_BYTES":4096,)" + R"("FREE_BYTES":512,"GROW_COUNT":2,"SHRINK_COUNT":1,"RECOVERY_COUNT":3,"GENERATION":7})"); + auto shmCliApi = std::make_shared(shmCliProviderOf(shmCliServer.port())); + SharedMemoryCliComponent shmCliComponent(shmCliApi); + + shmCliCoutCapture shmCliOut; + shmCliComponent.onCommand(UserDto{}, "show_stats", { "-s", "stats-seg" }); + const std::string shmCliPrinted = shmCliOut.str(); + + REQUIRE(shmCliServer.paths.size() == 1); + REQUIRE(shmCliServer.paths[0] == "/shm/stats/stats-seg"); + REQUIRE(shmCliServer.methods[0] == "GET"); + REQUIRE(shmCliPrinted.find("stats-seg") != std::string::npos); + REQUIRE(shmCliPrinted.find("AllocatedBytes") != std::string::npos); + REQUIRE(shmCliPrinted.find("1024") != std::string::npos); +} + +TEST_CASE("SharedMemoryCliComponent: show_stats without a segment name prints an error") +{ + shmCliBackend shmCliServer; + auto shmCliApi = std::make_shared(shmCliProviderOf(shmCliServer.port())); + SharedMemoryCliComponent shmCliComponent(shmCliApi); + + shmCliCoutCapture shmCliOut; + shmCliComponent.onCommand(UserDto{}, "show_stats", {}); + const std::string shmCliPrinted = shmCliOut.str(); + + REQUIRE(shmCliServer.paths.empty()); + REQUIRE(shmCliPrinted.find("ERROR") != std::string::npos); + REQUIRE(shmCliPrinted.find("please set segment name") != std::string::npos); +} + +TEST_CASE("SharedMemoryCliComponent: cleanup posts to the cleanup endpoint and prints the counts") +{ + shmCliBackend shmCliServer; + shmCliServer.answerWith(R"({"DRY_RUN":false,"SCANNED":3,"ORPHANS":1,"REMOVED":1,"FAILED":0})"); + auto shmCliApi = std::make_shared(shmCliProviderOf(shmCliServer.port())); + SharedMemoryCliComponent shmCliComponent(shmCliApi); + + shmCliCoutCapture shmCliOut; + shmCliComponent.onCommand(UserDto{}, "cleanup", {}); + const std::string shmCliPrinted = shmCliOut.str(); + + REQUIRE(shmCliServer.paths.size() == 1); + REQUIRE(shmCliServer.paths[0] == "/shm/cleanup"); + REQUIRE(shmCliServer.methods[0] == "POST"); + REQUIRE(shmCliPrinted.find("1 removed") != std::string::npos); +} + +TEST_CASE("SharedMemoryCliComponent: recover_segments posts to the recover endpoint and prints the count") +{ + shmCliBackend shmCliServer; + shmCliServer.answerWith(R"({"RECOVERED":2})"); + auto shmCliApi = std::make_shared(shmCliProviderOf(shmCliServer.port())); + SharedMemoryCliComponent shmCliComponent(shmCliApi); + + shmCliCoutCapture shmCliOut; + shmCliComponent.onCommand(UserDto{}, "recover_segments", {}); + const std::string shmCliPrinted = shmCliOut.str(); + + REQUIRE(shmCliServer.paths.size() == 1); + REQUIRE(shmCliServer.paths[0] == "/shm/recover"); + REQUIRE(shmCliServer.methods[0] == "POST"); + REQUIRE(shmCliPrinted.find("Recovered 2 orphaned remap locks") != std::string::npos); +} + +TEST_CASE("SharedMemoryCliComponent: recover_segments reports a rejected recover") +{ + shmCliBackend shmCliServer; + shmCliServer.answerWithStatus(500, "{}"); + auto shmCliApi = std::make_shared(shmCliProviderOf(shmCliServer.port())); + SharedMemoryCliComponent shmCliComponent(shmCliApi); + + shmCliCoutCapture shmCliOut; + shmCliComponent.onCommand(UserDto{}, "recover_segments", {}); + const std::string shmCliPrinted = shmCliOut.str(); + + REQUIRE(shmCliServer.paths.size() == 1); + REQUIRE(shmCliPrinted.find("ERROR") != std::string::npos); + REQUIRE(shmCliPrinted.find("recover was rejected") != std::string::npos); +} + +TEST_CASE("SharedMemoryCliComponent: allCommandsOf returns the eight registered commands") { auto shmCliApi = std::make_shared(shmCliProviderOf(1)); SharedMemoryCliComponent shmCliComponent(shmCliApi); const std::vector shmCliCommands = shmCliComponent.allCommandsOf(); bool shmCliHasAll = true; - for (const char *shmCliExpected : - { "show_segments", "show_repositories", "shrink_segment", "grow_segment", "export_repository" }) { + for (const char *shmCliExpected : { "show_segments", + "show_repositories", + "show_stats", + "cleanup", + "recover_segments", + "shrink_segment", + "grow_segment", + "export_repository" }) { shmCliHasAll = shmCliHasAll && std::find(shmCliCommands.begin(), shmCliCommands.end(), shmCliExpected) != shmCliCommands.end(); } - REQUIRE(shmCliCommands.size() == 5); + REQUIRE(shmCliCommands.size() == 8); REQUIRE(shmCliHasAll); } @@ -483,6 +582,40 @@ TEST_CASE("SharedMemoryCliComponent: export_repository asks the api for the uuid REQUIRE(shmCliPrinted.find("exported") != std::string::npos); } +TEST_CASE("SharedMemoryCliComponent: export_repository writes the export to the given file") +{ + shmCliBackend shmCliServer; + shmCliServer.answerWith("{\"exported\":1}"); + auto shmCliApi = std::make_shared(shmCliProviderOf(shmCliServer.port())); + SharedMemoryCliComponent shmCliComponent(shmCliApi); + + const std::filesystem::path shmCliTarget = + std::filesystem::temp_directory_path() / "lintel_shm_export_cli_component_test.json"; + std::filesystem::remove(shmCliTarget); + + shmCliCoutCapture shmCliOut; + shmCliComponent.onCommand( + UserDto{}, "export_repository", { "--uuid", "export-uuid", "--file", shmCliTarget.string() }); + const std::string shmCliPrinted = shmCliOut.str(); + + REQUIRE(shmCliServer.paths.size() == 1); + REQUIRE(shmCliServer.paths[0] == "/shm/repository/export-uuid"); + REQUIRE(shmCliPrinted.find("Exported SharedMemory to >") != std::string::npos); + + REQUIRE(std::filesystem::exists(shmCliTarget)); + { + // scoped: Windows refuses to delete a file that is still open + std::ifstream shmCliFile(shmCliTarget); + std::string shmCliContent; + std::getline(shmCliFile, shmCliContent); + REQUIRE(shmCliContent.find("exported") != std::string::npos); + } + + std::error_code shmCliRemoveError; + std::filesystem::remove(shmCliTarget, shmCliRemoveError); + REQUIRE_FALSE(shmCliRemoveError); +} + TEST_CASE("SharedMemoryCliComponent: an unknown command never reaches the api") { shmCliBackend shmCliServer; @@ -526,6 +659,7 @@ TEST_CASE("SharedMemoryApi: an unauthorized response throws for every endpoint") REQUIRE_THROWS_AS(shmCliApi->allRepositoriesOf(".*", ".*"), HttpUnauthorizedException); REQUIRE_THROWS_AS(shmCliApi->growOf("seg", "1024"), HttpUnauthorizedException); REQUIRE_THROWS_AS(shmCliApi->repositoryOf("uuid"), HttpUnauthorizedException); + REQUIRE_THROWS_AS(shmCliApi->recoverOf(), HttpUnauthorizedException); } TEST_CASE("SharedMemoryApi: a server error is tolerated per endpoint") @@ -539,6 +673,7 @@ TEST_CASE("SharedMemoryApi: a server error is tolerated per endpoint") REQUIRE(shmCliApi->allRepositoriesOf(".*", ".*").empty()); REQUIRE_NOTHROW(shmCliApi->growOf("seg", "1024")); REQUIRE(shmCliApi->repositoryOf("uuid").empty()); + REQUIRE_FALSE(shmCliApi->recoverOf().has_value()); } TEST_CASE("SharedMemoryApi: a malformed body is tolerated") @@ -549,6 +684,7 @@ TEST_CASE("SharedMemoryApi: a malformed body is tolerated") REQUIRE(shmCliApi->allSegmentsOf(".*").empty()); REQUIRE(shmCliApi->allRepositoriesOf(".*", ".*").empty()); + REQUIRE_FALSE(shmCliApi->recoverOf().has_value()); } TEST_CASE("SharedMemoryApi: a missing server is tolerated per endpoint") @@ -561,5 +697,6 @@ TEST_CASE("SharedMemoryApi: a missing server is tolerated per endpoint") REQUIRE(shmCliApi->allRepositoriesOf(".*", ".*").empty()); REQUIRE_NOTHROW(shmCliApi->growOf("seg", "1024")); REQUIRE(shmCliApi->repositoryOf("uuid").empty()); + REQUIRE_FALSE(shmCliApi->recoverOf().has_value()); } }// namespace lintel \ No newline at end of file diff --git a/tests/core/CoreUtilsTest.cpp b/tests/core/CoreUtilsTest.cpp index f993e7fb..dbfd72c3 100644 --- a/tests/core/CoreUtilsTest.cpp +++ b/tests/core/CoreUtilsTest.cpp @@ -1,3 +1,4 @@ +#include #include #include #include @@ -8,6 +9,7 @@ #include #include +#include #include #include "../helpers/PlatformCompat.h" @@ -119,6 +121,23 @@ TEST_CASE("MemorySize: deserialize TB") { REQUIRE(MemorySize::deserialize("1TB") TEST_CASE("MemorySize: deserialize byte suffix") { REQUIRE(MemorySize::deserialize("512B") == 512); } +TEST_CASE("MemorySize: deserialize rejects invalid input") +{ + REQUIRE_THROWS_AS(MemorySize::deserialize(""), std::invalid_argument); + REQUIRE_THROWS_AS(MemorySize::deserialize("abc"), std::invalid_argument); + // trailing garbage behind the number + REQUIRE_THROWS_AS(MemorySize::deserialize("12XB"), std::invalid_argument); + REQUIRE_THROWS_AS(MemorySize::deserialize("1k5MB"), std::invalid_argument); +} + +TEST_CASE("MemorySize: deserialize rejects overflow") +{ + // more digits than the numeric type can hold + REQUIRE_THROWS_AS(MemorySize::deserialize("99999999999999999999999999MB"), std::invalid_argument); + // fits the numeric type, but the unit multiplier overflows + REQUIRE_THROWS_AS(MemorySize::deserialize("18446744073709551615PB"), std::invalid_argument); +} + TEST_CASE("MemorySize: serialize bytes") { std::string result = MemorySize::serialize(500); @@ -272,4 +291,25 @@ TEST_CASE("Cryption: verifyOf supports legacy SHA-512 hash") REQUIRE(Cryption::verifyOf("test", legacy)); REQUIRE_FALSE(Cryption::verifyOf("wrong", legacy)); } + +TEST_CASE("Platform: expandHomeDirectory only expands a leading tilde") +{ + const std::string home = homeDirectory(); + if (home.empty()) { SKIP("no home directory in this environment"); } + + REQUIRE(expandHomeDirectory("~") == home); + REQUIRE(expandHomeDirectory("~/config.xml") == home + "/config.xml"); + REQUIRE(expandHomeDirectory("~\\config.xml") == home + "\\config.xml"); + + // A tilde that is not the first character is a literal. Windows short names + // ("C:\Users\RUNNER~1\Temp\x.json") contain one, and replacing it + // everywhere turned that path into "C:\Users\RUNNERC:\Users\...". + const std::string shortName = "C:\\Users\\RUNNER~1\\Temp\\x.json"; + REQUIRE(expandHomeDirectory(shortName) == shortName); + REQUIRE(expandHomeDirectory("/tmp/RUNNER~1/x.json") == "/tmp/RUNNER~1/x.json"); + // a leading tilde followed by a name is another user's home, not ours + REQUIRE(expandHomeDirectory("~runner/x") == "~runner/x"); + REQUIRE(expandHomeDirectory("") == ""); +} + }// namespace lintel diff --git a/tests/core/LoggerServiceTest.cpp b/tests/core/LoggerServiceTest.cpp index 337585e3..9cb9e8b1 100644 --- a/tests/core/LoggerServiceTest.cpp +++ b/tests/core/LoggerServiceTest.cpp @@ -1,6 +1,7 @@ #include #include #include +#include #include #include #include @@ -143,7 +144,7 @@ TEST_CASE("LoggerService: the process name is matched as a regular expression", { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "regex matched {}", 1); + service.log(std::source_location::current(), LogLevel::info, "regex matched {}", 1); } REQUIRE(loggerContains(loggerFileContent(file), "regex matched 1")); @@ -160,7 +161,7 @@ TEST_CASE("LoggerService: an empty sink file name falls back to .l REQUIRE_FALSE(std::filesystem::exists(file)); { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "fallback name"); + service.log(std::source_location::current(), LogLevel::info, "fallback name"); } REQUIRE(std::filesystem::exists(file)); @@ -177,7 +178,7 @@ TEST_CASE("LoggerService: a configured sink file name is used", "[logger_service { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "named file"); + service.log(std::source_location::current(), LogLevel::info, "named file"); } REQUIRE(std::filesystem::exists(file)); @@ -195,7 +196,7 @@ TEST_CASE("LoggerService: create sub directories puts the log into a per process { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "in sub directory"); + service.log(std::source_location::current(), LogLevel::info, "in sub directory"); } REQUIRE(std::filesystem::exists(file)); @@ -236,7 +237,7 @@ TEST_CASE("LoggerService: a daily sink creates a dated log file", "[logger_servi { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "daily entry"); + service.log(std::source_location::current(), LogLevel::info, "daily entry"); } // a daily sink names its file after the current date, so only the shape of the name is stable @@ -257,9 +258,9 @@ TEST_CASE("LoggerService: messages below the logger level are dropped", "[logger { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::debug, "debug marker"); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "info marker"); - service.log(__FILE__, __LINE__, "test", LogLevel::error, "error marker"); + service.log(std::source_location::current(), LogLevel::debug, "debug marker"); + service.log(std::source_location::current(), LogLevel::info, "info marker"); + service.log(std::source_location::current(), LogLevel::error, "error marker"); } const std::string content = loggerFileContent(file); @@ -278,8 +279,8 @@ TEST_CASE("LoggerService: messages below the sink level are dropped", "[logger_s { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::warn, "warn marker"); - service.log(__FILE__, __LINE__, "test", LogLevel::error, "error marker"); + service.log(std::source_location::current(), LogLevel::warn, "warn marker"); + service.log(std::source_location::current(), LogLevel::error, "error marker"); } const std::string content = loggerFileContent(file); @@ -299,7 +300,7 @@ TEST_CASE("LoggerService: a configured sink pattern replaces the default pattern { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::error, "only the message"); + service.log(std::source_location::current(), LogLevel::error, "only the message"); } const std::string content = loggerFileContent(file); @@ -315,7 +316,7 @@ TEST_CASE("LoggerService: a broken format string is reported instead of throwing { LoggerService service(processName, configuration); - REQUIRE_NOTHROW(service.log(__FILE__, __LINE__, "test", LogLevel::error, "missing {}")); + REQUIRE_NOTHROW(service.log(std::source_location::current(), LogLevel::error, "missing {}")); } const std::string content = loggerFileContent(file); @@ -336,7 +337,7 @@ TEST_CASE("LoggerService: a syslog sink is skipped next to a file sink", "[logge { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "next to syslog"); + service.log(std::source_location::current(), LogLevel::info, "next to syslog"); } REQUIRE(loggerContains(loggerFileContent(file), "next to syslog")); @@ -354,7 +355,7 @@ TEST_CASE("LoggerService: a console sink next to a file sink keeps the file sink { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "with console"); + service.log(std::source_location::current(), LogLevel::info, "with console"); } REQUIRE(loggerContains(loggerFileContent(file), "with console")); @@ -373,7 +374,7 @@ TEST_CASE("LoggerService: the rotating sink rolls over when the size limit is ex { LoggerService service(processName, configuration); - for (int i = 0; i < 40; i++) { service.log(__FILE__, __LINE__, "test", LogLevel::info, "marker {}", i); } + for (int i = 0; i < 40; i++) { service.log(std::source_location::current(), LogLevel::info, "marker {}", i); } } REQUIRE(std::filesystem::exists(file)); @@ -423,10 +424,10 @@ TEST_CASE("LoggerService: every sink level string is mapped", "[logger_service]" { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::error, "level marker"); + service.log(std::source_location::current(), LogLevel::error, "level marker"); // one level below the configured floor: a sink whose level string was // dropped keeps spdlog's default (trace) and would log this as well - service.log(__FILE__, __LINE__, "test", LogLevel::warn, "below marker"); + service.log(std::source_location::current(), LogLevel::warn, "below marker"); } // the trace sink is the most permissive one and must have received the message @@ -450,7 +451,7 @@ TEST_CASE("LoggerService: every logger level string is mapped", "[logger_service INFO("logger level: " << level); auto configuration = loggerConfigurationOf(directory.path, "main", level, { loggerRotatingSink("app") }); LoggerService service(processName, configuration); - REQUIRE_NOTHROW(service.log(__FILE__, __LINE__, "test", LogLevel::error, "level {}", level)); + REQUIRE_NOTHROW(service.log(std::source_location::current(), LogLevel::error, "level {}", level)); } } @@ -481,7 +482,7 @@ TEST_CASE("LoggerService: a tcp sink is constructed and used", "[logger_service] { LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "tcp marker"); + service.log(std::source_location::current(), LogLevel::info, "tcp marker"); } ::close(serverFd); @@ -495,7 +496,7 @@ TEST_CASE("LoggerService: initSingleton installs a configured logger", "[logger_ auto configuration = loggerConfigurationOf(directory.path, "main", "trace", { loggerRotatingSink("app") }); LoggerService::initSingleton(std::make_shared("main"), configuration); - REQUIRE_NOTHROW(LoggerService::get().log(__FILE__, __LINE__, "test", LogLevel::error, "singleton")); + REQUIRE_NOTHROW(LoggerService::get().log(std::source_location::current(), LogLevel::error, "singleton")); // initSingleton2 falls back to the default logger and must not throw REQUIRE_NOTHROW(LoggerService::initSingleton2()); @@ -529,11 +530,11 @@ TEST_CASE("LoggerService: setLevel changes the effective level without a restart // below the configured level: dropped, so the diagnosis starts from what the // operator deployed and not from a silenced process - service.log(__FILE__, __LINE__, "test", LogLevel::debug, "before the runtime change"); + service.log(std::source_location::current(), LogLevel::debug, "before the runtime change"); REQUIRE_FALSE(loggerContains(loggerFileContent(file), "before the runtime change")); service.setLevel(LogLevel::debug); - service.log(__FILE__, __LINE__, "test", LogLevel::debug, "after the runtime change"); + service.log(std::source_location::current(), LogLevel::debug, "after the runtime change"); REQUIRE(loggerContains(loggerFileContent(file), "after the runtime change")); REQUIRE(service.levelOf() == LogLevel::debug); } @@ -547,7 +548,7 @@ TEST_CASE("LoggerService: a json logger writes one json object per message", "[l const auto file = directory.path / "app.log"; LoggerService service(processName, configuration); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "json message {}", 7); + service.log(std::source_location::current(), LogLevel::info, "json message {}", 7); const std::string content = loggerFileContent(file); // the shape a log shipper parses: one line, object start first, the level and @@ -568,13 +569,13 @@ TEST_CASE("LoggerService: the request correlation id is part of the message", "[ LoggerService::setRequestId(std::string()); REQUIRE(LoggerService::requestId().empty()); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "outside a request"); + service.log(std::source_location::current(), LogLevel::info, "outside a request"); REQUIRE_FALSE(loggerContains(loggerFileContent(file), "[req-123]")); { RequestIdGuard guard("req-123"); REQUIRE(LoggerService::requestId() == "req-123"); - service.log(__FILE__, __LINE__, "test", LogLevel::info, "inside a request"); + service.log(std::source_location::current(), LogLevel::info, "inside a request"); // ties the line to the X-Request-Id the server answered with REQUIRE(loggerContains(loggerFileContent(file), "[req-123] inside a request")); } diff --git a/tests/core/SqliteConnectionTest.cpp b/tests/core/SqliteConnectionTest.cpp index 0cfeeb58..86088285 100644 --- a/tests/core/SqliteConnectionTest.cpp +++ b/tests/core/SqliteConnectionTest.cpp @@ -26,14 +26,14 @@ namespace { { std::string m_dbPath; - PragmaFixture() : m_dbPath(kPragmaTestDb()) { std::remove(m_dbPath.c_str()); } + PragmaFixture() : m_dbPath(kPragmaTestDb()) { static_cast(std::remove(m_dbPath.c_str())); } ~PragmaFixture() { - std::remove(m_dbPath.c_str()); + static_cast(std::remove(m_dbPath.c_str())); // WAL mode creates -wal/-shm sidecar files next to the DB file - std::remove((m_dbPath + "-wal").c_str()); - std::remove((m_dbPath + "-shm").c_str()); + static_cast(std::remove((m_dbPath + "-wal").c_str())); + static_cast(std::remove((m_dbPath + "-shm").c_str())); } [[nodiscard]] std::shared_ptr connectionEntry() const @@ -101,6 +101,37 @@ TEST_CASE("SQLite: foreign_keys ON rejects violating inserts", "[sqlite][pragma] REQUIRE_THROWS_AS(statement.execute("insert into child (id, parent_id) values (2, 999)"), db::SQLException); } +TEST_CASE("SQLite: execute runs every statement of a script", "[sqlite][execute]") +{ + db::Connection connection(db::ConnectionType::SQLite, ":memory:"); + db::Statement statement(connection); + + // DDL plus DML in a single call: only the first statement would run if the prepare tail were ignored + statement.execute( + "create table multi (id integer primary key, name text);\n" + "insert into multi (id, name) values (1, 'first');\n" + "insert into multi (id, name) values (2, 'second');"); + + db::Result rows = statement.execute("select name from multi order by id"); + REQUIRE(rows.getSize() == 2); + REQUIRE(rows.of(0).of(0).getValue() == "first"); + REQUIRE(rows.of(1).of(0).getValue() == "second"); + + // the rows of all statements end up in the same result + db::Result combined = statement.execute("select 1 as value; select 2 as value"); + REQUIRE(combined.getSize() == 2); +} + +TEST_CASE("SQLite: execute reports a failure in a later statement", "[sqlite][execute]") +{ + db::Connection connection(db::ConnectionType::SQLite, ":memory:"); + db::Statement statement(connection); + statement.execute("create table later (id integer primary key)"); + + REQUIRE_THROWS_AS( + statement.execute("insert into later (id) values (1); insert into missing (id) values (2);"), db::SQLException); +} + TEST_CASE("SQLite: foreign_keys ON applies to DatabaseConnectionEntry connections", "[sqlite][pragma]") { PragmaFixture fixture; diff --git a/tests/core/StartupBuilderTest.cpp b/tests/core/StartupBuilderTest.cpp index b744bfb4..4a8b59b6 100644 --- a/tests/core/StartupBuilderTest.cpp +++ b/tests/core/StartupBuilderTest.cpp @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -84,6 +85,24 @@ namespace { explicit SbFeatureA(std::shared_ptr features) : SbCountingFeature(features, "SbFeatureA") {} }; + /** Feature derives its name from the enum value; this subclass only makes + * the template instantiable in a test. */ + class SbEnumNamedFeature : public Feature + { + public: + SbEnumNamedFeature(Features::Value value, std::shared_ptr features) : Feature(value, std::move(features)) + {} + void registerTypes(lintel::di::ContainerBuilder & /*builder*/) override {} + void initialize(std::shared_ptr /*container*/) override {} + }; + + /** Overwrite the stack region a just-returned constructor used. */ + void SbScribbleStack() + { + volatile std::byte sbStackBuffer[512]; + for (std::size_t i = 0; i < sizeof(sbStackBuffer); ++i) { sbStackBuffer[i] = std::byte{ 0xA5 }; } + } + class SbFeatureB : public SbCountingFeature { public: @@ -480,6 +499,20 @@ TEST_CASE("StartupBuilder::start shuts the services down in shutdown priority or std::filesystem::remove_all(configDirectory, errorCode); } +TEST_CASE("Feature::getName owns the name it returns", "[startup_builder]") +{ + // Regression: the name used to be a std::string_view initialised from a + // temporary std::string, so getName() handed out freed stack bytes once the + // constructor returned. StartupBuilder::verifyFeatureDependencies() then + // compared that garbage against the configured feature names and aborted the + // process at startup. Scribbling over the stack makes the dangling view + // visible even without a sanitizer. + auto features = std::make_shared(); + SbEnumNamedFeature feature(Features::Value::Property, features); + SbScribbleStack(); + REQUIRE(feature.getName() == "Property"); +} + TEST_CASE("lintel features declare their layer dependencies", "[startup_builder]") { // Guards the concrete declarations: StartupBuilder only sees them through diff --git a/tests/features/MessageQueueShmCoreTest.cpp b/tests/features/MessageQueueShmCoreTest.cpp index 4ee60674..4b0438c7 100644 --- a/tests/features/MessageQueueShmCoreTest.cpp +++ b/tests/features/MessageQueueShmCoreTest.cpp @@ -1,3 +1,4 @@ +#include #include #include "../helpers/PlatformCompat.h" @@ -6,6 +7,7 @@ #include #include #include +#include #include #include #include @@ -14,10 +16,15 @@ #include #include +#if !defined(_WIN32) +#include // waitpid (the cross-process test needs fork) +#endif + #include "lintel/core/configuration/Configuration.h" #include "lintel/core/configuration/EnvironmentConfiguration.h" #include "lintel/core/models/ProcessName.h" #include "lintel/core/models/SharedMemorySegment.h" +#include "lintel/core/shm/BoostSegmentAllocator.h" #include "lintel/core/utils/TypeName.h" #include "lintel/features/base/configuration/SharedMemorySegmentComponent.h" #include "lintel/features/base/configuration/SharedMemorySegmentEntry.h" @@ -52,9 +59,11 @@ namespace { std::atomic g_queueCounter{ 0 }; + /** Queue names stay short: the shared memory object that holds the queue + * header is named "lintel_mq_" + queue name and limited to 32 characters. */ std::string uniqueQueueName() { - return "shm_queue_core_" + std::to_string(::getpid()) + "_" + std::to_string(++g_queueCounter); + return "shm_q_" + std::to_string(::getpid()) + "_" + std::to_string(++g_queueCounter); } struct TestMessage @@ -113,6 +122,20 @@ namespace { std::string m_name; }; + /** Allocator that cannot serve a single request. + * + * The cross-process test hands an accessor that captured this allocator to + * the queue: everything the queue does in the segment has to go through the + * name binding instead, exactly like an accessor that was copied into the + * segment by another process and carries a pointer into that process. */ + class DeadEndAllocator : public ShmSegmentAccessor::Allocator + { + public: + void *allocate(std::size_t /*bytes*/) override { return nullptr; } + + void deallocate(void * /*pointer*/) override {} + }; + }// namespace TEST_CASE("MessageQueueShmCore sends and receives FIFO") @@ -320,6 +343,65 @@ TEST_CASE("MessageQueueShmCore is visible to a second core of the same segment") REQUIRE_THROWS_AS((MessageQueueShmCore(name, 8, sizeof(TestMessage), accessor)), std::invalid_argument); } +#if !defined(_WIN32) +TEST_CASE("MessageQueueShmCore is found by a forked second process") +{ + QueueFixture fixture; + const std::string name = uniqueQueueName(); + QueueCleanup cleanup(name); + + // The parent creates the queue and therefore owns its ring. + MessageQueueShmCore queue(name, 4, sizeof(TestMessage), fixture.accessor()); + REQUIRE(queue.numberMessagesOf() == 0); + + const pid_t pid = ::fork(); + REQUIRE(pid >= 0); + if (pid == 0) { + // Child: act like a second process that only knows the segment name. The + // fork copied the parent's process local queue cache, so drop it first - + // otherwise the child would find the parent's state and the test would not + // prove that the queue is discovered through the segment. + MessageQueueShmCore::removeOf(name); + int result = 1; + try { + // A fresh mapping and a fresh allocator: nothing is shared with the + // parent except the segment file itself. The accessor deliberately + // captured an allocator that cannot allocate - only the name binding + // below makes it usable, just as in a process that holds an accessor + // copied into the segment by somebody else. + boost::interprocess::managed_mapped_file mapping(boost::interprocess::open_only, fixture.path.string().c_str()); + BoostSegmentAllocator allocator(*mapping.get_segment_manager()); + DeadEndAllocator deadEnd; + const ShmSegmentAccessor accessor(fixture.segmentName, deadEnd); + ShmSegmentAccessor::bind(fixture.segmentName, allocator); + // The same capacity and slot size have to be accepted (no layout + // conflict) and the message has to land in the ring of the parent. + MessageQueueShmCore second(name, 4, sizeof(TestMessage), accessor); + const TestMessage message{ 4711, 12 }; + result = second.trySendOf(&message, sizeof(message)) ? 0 : 1; + } catch (...) { + result = 1; + } + // _exit, not return: the inherited spdlog and history state of the parent + // must not run its destructors (and take locks) in the child. + ::_exit(result); + } + + int status = 0; + REQUIRE(::waitpid(pid, &status, 0) == pid); + REQUIRE(WIFEXITED(status)); + REQUIRE(WEXITSTATUS(status) == 0); + + // Generous timeout: the send of the child is already done, the receive only + // has to observe it in the shared ring. + TestMessage received{}; + REQUIRE(queue.receiveOfWithTimeout(&received, sizeof(received), std::chrono::milliseconds(5000))); + REQUIRE(received.value == 4711); + REQUIRE(received.sequence == 12); + REQUIRE(queue.numberMessagesOf() == 0); +} +#endif// !_WIN32 + TEST_CASE("MessageQueueShmCore removeOf drops the queue association") { QueueFixture fixture; diff --git a/tests/features/SharedMemoryRepositoryTest.cpp b/tests/features/SharedMemoryRepositoryTest.cpp index 2b752aae..458b0121 100644 --- a/tests/features/SharedMemoryRepositoryTest.cpp +++ b/tests/features/SharedMemoryRepositoryTest.cpp @@ -2,10 +2,13 @@ #include "../helpers/PlatformCompat.h" +#include #include #include +#include #include #include +#include #include #include #include @@ -15,6 +18,7 @@ #include "lintel/core/models/ProcessName.h" #include "lintel/core/models/SharedMemorySegment.h" #include "lintel/core/shm/SharedMemoryRepository.h" +#include "lintel/core/shm/SharedMemoryRepositoryRegistry.h" #include "lintel/core/shm/containers/ArrayShm.h" #include "lintel/core/shm/containers/QueueShm.h" #include "lintel/core/shm/containers/RingBufferShm.h" @@ -23,7 +27,6 @@ #include "lintel/features/base/configuration/SharedMemorySegmentEntry.h" #include "lintel/features/base/repositories/SharedMemoryRepository.h" #include "lintel/features/base/services/SchedulerService.h" -#include "lintel/features/base/services/SharedMemoryRepositoryRegistry.h" #include "lintel/features/base/services/SharedMemorySegmentManager.h" #include "lintel/features/base/services/SharedMemoryService.h" #include "lintel/features/base/services/ShmConstructors.h" @@ -149,6 +152,42 @@ namespace { int m_counter = 0; }; + /** Bump allocator over one externally owned buffer. The whole "segment" is + * therefore contiguous memory that can be copied to another address in one + * memcpy - the situation of a segment mapped at a different virtual address + * by another process. */ + class RegionAllocator : public ShmSegmentAccessor::Allocator + { + public: + RegionAllocator(std::byte *base, std::size_t size) : m_base(base), m_size(size) {} + + void *allocate(std::size_t bytes) override + { + if (m_used > m_size) { return nullptr; } + void *candidate = m_base + m_used; + std::size_t space = m_size - m_used; + // Cache line alignment, like the segment allocator gives the slot array. + if (std::align(RingBufferShm::CACHE_LINE, bytes, candidate, space) == nullptr) { return nullptr; } + m_used = static_cast(static_cast(candidate) - m_base) + bytes; + return candidate; + } + + void deallocate(void * /*pointer*/) override {} + + private: + std::byte *m_base; + std::size_t m_size; + std::size_t m_used = 0; + }; + + /** Bytes of one simulated segment mapping; cache line aligned so a relocated + * ring still holds its cache line aligned slots. */ + struct alignas(RingBufferShm::CACHE_LINE) SegmentRegion + { + static constexpr std::size_t SIZE = 4096; + std::byte bytes[SIZE]{}; + }; + std::string repositoryToJson(const JsonSerializable &repository) { rapidjson::StringBuffer buffer; @@ -389,6 +428,41 @@ TEST_CASE("RingBufferShm in shared memory pushes and pops in order") REQUIRE_FALSE(popped.has_value()); } +TEST_CASE("RingBufferShm addresses its slots relative to the ring object") +{ + using Ring = RingBufferShm; + + // Emulate the second mapping of a segment: ring and slot array live in one + // contiguous region, that region is copied to another address and the + // original is wiped. An address cached inside the ring would keep pointing + // at the wiped region; the offset travels with the copy. + auto firstRegion = std::make_unique(); + RegionAllocator firstAllocator(firstRegion->bytes, SegmentRegion::SIZE); + auto *ring = static_cast(firstAllocator.allocate(sizeof(Ring))); + REQUIRE(ring != nullptr); + ::new (static_cast(ring)) Ring(4, ShmSegmentAccessor(firstAllocator)); + REQUIRE(ring->push(7)); + REQUIRE(ring->push(8)); + + const std::ptrdiff_t ringOffset = reinterpret_cast(ring) - firstRegion->bytes; + auto secondRegion = std::make_unique(); + std::memcpy(secondRegion->bytes, firstRegion->bytes, SegmentRegion::SIZE); + + // The original mapping is gone: only the relocated bytes are still valid. + std::fill_n(firstRegion->bytes, SegmentRegion::SIZE, std::byte{ 0 }); + + auto *relocated = std::launder(reinterpret_cast(secondRegion->bytes + ringOffset)); + int value = 0; + REQUIRE(relocated->pop(value)); + REQUIRE(value == 7); + REQUIRE(relocated->push(9)); + REQUIRE(relocated->pop(value)); + REQUIRE(value == 8); + REQUIRE(relocated->pop(value)); + REQUIRE(value == 9); + REQUIRE(relocated->empty()); +} + TEST_CASE("QueueShm in shared memory is FIFO") { ShmRepositoryFixture fixture; @@ -446,4 +520,54 @@ TEST_CASE("SharedMemoryRepositoryRegistry keeps typed repositories") REQUIRE_FALSE(registry.remove("test-registry-cell")); } +TEST_CASE("shm::SharedMemoryRepository::of reuses one instance per segment and name") +{ + ShmRepositoryFixture fixture; + auto ®istry = SharedMemoryRepositoryRegistry::instance(); + const std::string cellKey = fixture.segment->getName() + ":of-cell"; + const std::string otherKey = fixture.segment->getName() + ":of-other"; + REQUIRE_FALSE(registry.contains(cellKey)); + + auto first = + shm::SharedMemoryRepository::of(fixture.service, fixture.segment, "of-cell", CellPayload{ 5, 6 }); + REQUIRE(registry.contains(cellKey)); + REQUIRE(first->get().m_value == 5); + + // same segment and name: the registered instance comes back, the constructor + // arguments of the second call are not used again + auto second = + shm::SharedMemoryRepository::of(fixture.service, fixture.segment, "of-cell", CellPayload{ 9, 9 }); + REQUIRE(first.get() == second.get()); + REQUIRE(second->get().m_value == 5); + + // another name in the same segment is another instance + auto other = + shm::SharedMemoryRepository::of(fixture.service, fixture.segment, "of-other", CellPayload{ 1, 2 }); + REQUIRE(other.get() != first.get()); + REQUIRE(other->get().m_value == 1); + + // the key belongs to one payload type: a second type is refused instead of + // being handed a reinterpreted instance + REQUIRE_THROWS_AS( + (shm::SharedMemoryRepository::of(fixture.service, fixture.segment, "of-cell", 1)), std::invalid_argument); + + // remove() releases the entry; a later factory call builds a new instance + second.reset(); + const long referencesBeforeRemove = first.use_count();// first + the registry + REQUIRE(registry.remove(cellKey)); + REQUIRE_FALSE(registry.contains(cellKey)); + REQUIRE(first.use_count() == referencesBeforeRemove - 1); + auto third = + shm::SharedMemoryRepository::of(fixture.service, fixture.segment, "of-cell", CellPayload{ 7, 8 }); + REQUIRE(third.get() != first.get()); + // the binding is new, the cell is not: it still lives in the segment with the + // value the first construction stored + REQUIRE(third->get().m_value == 5); + + // do not leave entries behind: they would keep the fixture's service (and + // with it the segment mapping) alive beyond the test + REQUIRE(registry.remove(cellKey)); + REQUIRE(registry.remove(otherKey)); +} + }// namespace lintel diff --git a/tests/features/SharedMemoryServiceTest.cpp b/tests/features/SharedMemoryServiceTest.cpp index b7c0f7a0..59579aec 100644 --- a/tests/features/SharedMemoryServiceTest.cpp +++ b/tests/features/SharedMemoryServiceTest.cpp @@ -18,6 +18,7 @@ #include "lintel/core/models/ProcessName.h" #include "lintel/core/models/SharedMemorySegment.h" #include "lintel/core/services/ReadinessRegistry.h" +#include "lintel/core/shm/SegmentLayoutPolicy.h" #include "lintel/core/utils/TypeName.h" #include "lintel/features/base/configuration/SharedMemorySegmentComponent.h" #include "lintel/features/base/configuration/SharedMemorySegmentEntry.h" @@ -545,4 +546,195 @@ TEST_CASE("SharedMemory lock recovery: a holder that releases during the probe k same.post(); } +TEST_CASE("SharedMemoryService refuses a segment written with an older layout") +{ + // Default policy: nothing is discarded, the operator has to decide. The + // exception names the file so the decision can be made deliberately. + ScopedEnvironmentVariable configDirectory{ "CONFIG_DIRECTORY", "/tmp/nonexistent_shm_service_cfg" }; + const std::filesystem::path path = uniqueSegmentPath(); + const std::string name = uniqueSegmentName(); + + { + boost::interprocess::managed_mapped_file old(boost::interprocess::create_only, path.c_str(), kInitialSize); + old.construct("stale_object")(42); + } + + auto segment = std::make_shared(path, name, kInitialSize); + auto config = std::make_shared( + std::vector>{}, std::make_shared()); + config->setEntries( + { std::make_shared(type_name(), segment) }); + auto segmentManager = std::make_shared(config); + auto scheduler = std::make_shared(std::make_shared("shm-service-test")); + + REQUIRE_THROWS_AS( + std::make_shared(segmentManager, scheduler, std::make_shared("shm-service-test")), + std::runtime_error); + + // the file is untouched: the operator still has it for a migration + { + boost::interprocess::managed_mapped_file current(boost::interprocess::open_only, path.c_str()); + REQUIRE(current.find("stale_object").first != nullptr); + } + + scheduler.reset(); + std::filesystem::remove(path); + boost::interprocess::named_semaphore::remove(shm::remapSemaphoreNameOf(name).c_str()); + SegmentLockOwner::removeOf(name); +} + +TEST_CASE("SharedMemoryService recreates a segment when the policy allows it") +{ + // LINTEL_SHM_RECREATE=1 is the explicit opt-in for a developer machine; the + // contents are discarded, not migrated. + ScopedEnvironmentVariable configDirectory{ "CONFIG_DIRECTORY", "/tmp/nonexistent_shm_service_cfg" }; + ScopedEnvironmentVariable recreate{ "LINTEL_SHM_RECREATE", "1" }; + const std::filesystem::path path = uniqueSegmentPath(); + const std::string name = uniqueSegmentName(); + + { + // exactly what an older build leaves behind: named objects, no marker + boost::interprocess::managed_mapped_file old(boost::interprocess::create_only, path.c_str(), kInitialSize); + old.construct("stale_object")(42); + REQUIRE(old.find("stale_object").first != nullptr); + } + + auto segment = std::make_shared(path, name, kInitialSize); + auto config = std::make_shared( + std::vector>{}, std::make_shared()); + config->setEntries( + { std::make_shared(type_name(), segment) }); + auto segmentManager = std::make_shared(config); + auto scheduler = std::make_shared(std::make_shared("shm-service-test")); + + auto service = + std::make_shared(segmentManager, scheduler, std::make_shared("shm-service-test")); + + { + boost::interprocess::managed_mapped_file current(boost::interprocess::open_only, path.c_str()); + REQUIRE(current.find("stale_object").first == nullptr); + const auto marker = current.find("lintel_segment_layout"); + REQUIRE(marker.first != nullptr); + REQUIRE(*marker.first == 2); + current.construct("survivor")(7); + } + + // a segment that already carries the current marker is kept, not recreated + auto reopened = + std::make_shared(segmentManager, scheduler, std::make_shared("shm-service-test")); + { + boost::interprocess::managed_mapped_file current(boost::interprocess::open_only, path.c_str()); + REQUIRE(current.find("survivor").first != nullptr); + } + + reopened.reset(); + service.reset(); + scheduler.reset(); + std::filesystem::remove(path); + boost::interprocess::named_semaphore::remove(shm::remapSemaphoreNameOf(name).c_str()); + SegmentLockOwner::removeOf(name); +} + +TEST_CASE("SharedMemoryService lets a registered migrator decide what happens") +{ + // The migrator runs while the stale segment is still on disk, so it can read + // the payloads and carry them somewhere durable before anything is removed. + ScopedEnvironmentVariable configDirectory{ "CONFIG_DIRECTORY", "/tmp/nonexistent_shm_service_cfg" }; + const std::filesystem::path path = uniqueSegmentPath(); + const std::string name = uniqueSegmentName(); + + { + boost::interprocess::managed_mapped_file old(boost::interprocess::create_only, path.c_str(), kInitialSize); + old.construct("stale_object")(42); + } + + struct Seen + { + std::filesystem::path path; + std::uint64_t found = 0; + std::uint64_t current = 0; + int value = 0; + }; + Seen seen; + shm::SegmentLayoutPolicy::setMigrator( + [&seen](const std::filesystem::path &stale, std::uint64_t found, std::uint64_t current) { + seen.path = stale; + seen.found = found; + seen.current = current; + // the old segment is still intact, so its payload can be read + boost::interprocess::managed_mapped_file old(boost::interprocess::open_only, stale.c_str()); + const auto object = old.find("stale_object"); + if (object.first == nullptr) { return false; } + seen.value = *object.first; + return true;// "carried over, you may start empty" + }); + + auto segment = std::make_shared(path, name, kInitialSize); + auto config = std::make_shared( + std::vector>{}, std::make_shared()); + config->setEntries( + { std::make_shared(type_name(), segment) }); + auto segmentManager = std::make_shared(config); + auto scheduler = std::make_shared(std::make_shared("shm-service-test")); + + auto service = + std::make_shared(segmentManager, scheduler, std::make_shared("shm-service-test")); + + REQUIRE(seen.path == path); + REQUIRE(seen.found == 0);// a segment from before the marker existed + REQUIRE(seen.current == 2); + REQUIRE(seen.value == 42); + + { + boost::interprocess::managed_mapped_file current(boost::interprocess::open_only, path.c_str()); + REQUIRE(current.find("stale_object").first == nullptr); + } + + service.reset(); + scheduler.reset(); + std::filesystem::remove(path); + boost::interprocess::named_semaphore::remove(shm::remapSemaphoreNameOf(name).c_str()); + SegmentLockOwner::removeOf(name); + shm::SegmentLayoutPolicy::reset(); +} + +TEST_CASE("SharedMemoryService aborts when the migrator refuses") +{ + ScopedEnvironmentVariable configDirectory{ "CONFIG_DIRECTORY", "/tmp/nonexistent_shm_service_cfg" }; + const std::filesystem::path path = uniqueSegmentPath(); + const std::string name = uniqueSegmentName(); + + { + boost::interprocess::managed_mapped_file old(boost::interprocess::create_only, path.c_str(), kInitialSize); + old.construct("stale_object")(42); + } + + shm::SegmentLayoutPolicy::setMigrator( + [](const std::filesystem::path & /*stale*/, std::uint64_t /*found*/, std::uint64_t /*current*/) { return false; }); + + auto segment = std::make_shared(path, name, kInitialSize); + auto config = std::make_shared( + std::vector>{}, std::make_shared()); + config->setEntries( + { std::make_shared(type_name(), segment) }); + auto segmentManager = std::make_shared(config); + auto scheduler = std::make_shared(std::make_shared("shm-service-test")); + + REQUIRE_THROWS_WITH( + std::make_shared(segmentManager, scheduler, std::make_shared("shm-service-test")), + Catch::Matchers::ContainsSubstring("migrator refused")); + + { + // nothing was removed: the operator can still migrate offline + boost::interprocess::managed_mapped_file current(boost::interprocess::open_only, path.c_str()); + REQUIRE(current.find("stale_object").first != nullptr); + } + + scheduler.reset(); + std::filesystem::remove(path); + boost::interprocess::named_semaphore::remove(shm::remapSemaphoreNameOf(name).c_str()); + SegmentLockOwner::removeOf(name); + shm::SegmentLayoutPolicy::reset(); +} + }// namespace lintel diff --git a/tests/http/EventBusControllerTest.cpp b/tests/http/EventBusControllerTest.cpp new file mode 100644 index 00000000..1514c323 --- /dev/null +++ b/tests/http/EventBusControllerTest.cpp @@ -0,0 +1,476 @@ +#include + +#include "../helpers/PlatformCompat.h" + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "lintel/core/configuration/Configuration.h" +#include "lintel/core/configuration/EnvironmentConfiguration.h" +#include "lintel/core/models/ProcessName.h" +#include "lintel/core/models/SharedMemorySegment.h" +#include "lintel/core/utils/TypeName.h" +#include "lintel/features/base/configuration/EventBusComponent.h" +#include "lintel/features/base/configuration/EventBusEntry.h" +#include "lintel/features/base/configuration/SharedMemorySegmentComponent.h" +#include "lintel/features/base/configuration/SharedMemorySegmentEntry.h" +#include "lintel/features/base/events/Event.h" +#include "lintel/features/base/events/EventBus.h" +#include "lintel/features/base/models/Group.h" +#include "lintel/features/base/models/User.h" +#include "lintel/features/base/services/EventBusService.h" +#include "lintel/features/base/services/SchedulerService.h" +#include "lintel/features/base/services/SharedMemorySegmentManager.h" +#include "lintel/features/base/services/SharedMemoryService.h" +#include "lintel/features/http/controllers/EventBusController.h" +#include "lintel/features/http/service/ContentType.h" +#include "lintel/features/http/service/HttpStatusCodes.h" + +#include "../helpers/ScopedEnvironmentVariable.h" +#include "../mocks/MockAuthService.h" + +namespace lintel { +namespace { + + // `ebc` prefix: unity builds merge several test files into one anonymous + // namespace, so a generic `adminGroup` or `fixture` would collide. + constexpr std::size_t ebcSegmentSize = 1u << 20; + constexpr char ebcBusName[] = "orders"; + + Group ebcAdminGroup{ "Admin-EventBus", {}, true }; + Group ebcUserGroup{ "User-EventBus", {}, true }; + Group ebcPublisherGroup{ "Publisher-EventBus", {}, true }; + + User ebcAdmin("EventBus", "Admin", User::Male, "eventbus-admin@test.com", "eventbusAdmin", "", { ebcAdminGroup }); + User ebcUser("EventBus", "User", User::Male, "eventbus-user@test.com", "eventbusUser", "", { ebcUserGroup }); + User ebcPublisher("EventBus", + "Publisher", + User::Male, + "eventbus-publisher@test.com", + "eventbusPublisher", + "", + { ebcPublisherGroup }); + + UserToken ebcTokenOf(const User &user) + { + return UserToken{ "127.0.0.1", + "token-id", + std::chrono::time_point_cast(std::chrono::system_clock::now()), + user }; + } + + std::filesystem::path ebcUniquePath() + { + static unsigned int counter = 0; + const auto path = + std::filesystem::temp_directory_path() + / ("eventbus_controller_test_" + std::to_string(::getpid()) + "_" + std::to_string(counter++) + ".bin"); + std::filesystem::remove(path); + return path; + } + + // Keep the segment name short: the remap semaphore appends "_remap_sem" and + // POSIX caps semaphore names at 31 characters. + std::string ebcUniqueName() + { + static unsigned int counter = 0; + return "ebctl_" + std::to_string(::getpid()) + "_" + std::to_string(counter++); + } + + /** + * Build a controller over a real EventBusService in a temporary shared memory + * segment. The service is cheap to construct and the handlers are exercised + * directly, so no HTTP server is needed; the status of a successful handler is + * set by the server in production, hence the assertions look at the body. + */ + struct EventBusControllerFixture + { + // must stay the first member: it has to be set before the configuration is built + ScopedEnvironmentVariable m_configDirectory{ "CONFIG_DIRECTORY", "/tmp/nonexistent_ebc_test_cfg" }; + std::filesystem::path m_path = ebcUniquePath(); + std::string m_segmentName = ebcUniqueName(); + std::shared_ptr m_segment; + std::shared_ptr m_scheduler; + std::shared_ptr m_segmentManager; + std::shared_ptr m_sharedMemoryService; + std::shared_ptr m_eventBusService; + std::shared_ptr m_authService = std::make_shared(); + // constructed in the body: the handlers capture `this`, so a copy of the + // controller would capture the destroyed temporary + std::unique_ptr m_controller; + + EventBusControllerFixture() + { + m_segment = std::make_shared(m_path, m_segmentName, ebcSegmentSize); + auto configuration = std::make_shared( + std::vector>{}, std::make_shared()); + auto busEntry = std::make_shared( + type_name(), ebcBusName, m_segmentName, EventBusCapacities{ 8, 8, 8, 8 }); + configuration->setEntries( + { std::make_shared(type_name(), m_segment), busEntry }); + m_segmentManager = std::make_shared(configuration); + m_scheduler = std::make_shared(std::make_shared("eventbus-controller-test")); + m_sharedMemoryService = std::make_shared( + m_segmentManager, m_scheduler, std::make_shared("eventbus-controller-test")); + m_eventBusService = std::make_shared(configuration, + std::make_shared("eventbus-controller-test"), + m_sharedMemoryService, + m_segmentManager); + m_controller = std::make_unique(m_authService, m_eventBusService); + } + + ~EventBusControllerFixture() + { + m_eventBusService.reset(); + m_sharedMemoryService.reset(); + m_scheduler.reset(); + std::filesystem::remove(m_path); + boost::interprocess::named_semaphore::remove((m_segmentName + "_remap_sem").c_str()); + } + + EventBusControllerFixture(const EventBusControllerFixture &) = delete; + EventBusControllerFixture &operator=(const EventBusControllerFixture &) = delete; + + /** @return the configured bus */ + std::shared_ptr bus() const { return m_eventBusService->of(ebcBusName); } + }; + + /** Build a request whose regex matches are the given path segments. */ + httplib::Request ebcRequestWithMatches(const std::vector &parts) + { + // The smatch stored in the request holds iterators into the subject string, + // so the subject must outlive the returned request. + static std::string subject; + subject.clear(); + std::string pattern; + for (std::size_t i = 0; i < parts.size(); ++i) { + if (i != 0) { + subject += "/"; + pattern += "/"; + } + subject += parts[i]; + pattern += "(.*)"; + } + httplib::Request request; + std::smatch matches; + std::regex_match(subject, matches, std::regex(pattern)); + request.matches = matches; + return request; + } + + ContentType ebcJson() { return ContentType("application/json"); } + + constexpr char ebcPublishBody[] = + R"({"TOPIC":"orders.created","TYPE":"OrderCreated","PAYLOAD":"42","PRIORITY":"High"})"; + +}// namespace + +TEST_CASE("EventBusController: reading needs a token") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + httplib::Request ebcReq; + httplib::Response ebcRes; + + ebcController.busesOfGet(ebcReq, ebcRes, ebcJson(), std::nullopt); + REQUIRE(ebcRes.status == 401); +} + +TEST_CASE("EventBusController: reading needs the user or admin group") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + httplib::Request ebcReq; + httplib::Response ebcRes; + + User ebcOutsider( + "EventBus", "Outsider", User::Male, "outsider@test.com", "outsider", "", { Group{ "Other", {}, true } }); + ebcController.busesOfGet(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcOutsider)); + REQUIRE(ebcRes.status == 401); +} + +TEST_CASE("EventBusController: everything but json is forbidden") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + httplib::Request ebcReq; + httplib::Response ebcRes; + + ebcController.busesOfGet(ebcReq, ebcRes, ContentType("text/plain"), ebcTokenOf(ebcUser)); + REQUIRE(ebcRes.status == 403); +} + +TEST_CASE("EventBusController: busesOfGet lists the configuration and the counters") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + httplib::Request ebcReq; + httplib::Response ebcRes; + + ebcController.busesOfGet(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcUser)); + + // the handler does not set the status on success - the server does + REQUIRE(ebcRes.body.find("\"NAME\":\"orders\"") != std::string::npos); + REQUIRE(ebcRes.body.find("\"AVAILABLE\":true") != std::string::npos); + REQUIRE(ebcRes.body.find("\"BUS_CAPACITY\":8") != std::string::npos); + REQUIRE(ebcRes.body.find("\"SUBSCRIBER_CAPACITY\":8") != std::string::npos); + REQUIRE(ebcRes.body.find("\"MAX_TOPICS\":8") != std::string::npos); + REQUIRE(ebcRes.body.find("\"PUB_POLICY\":\"Fail\"") != std::string::npos); + REQUIRE(ebcRes.body.find("\"DROPPED\":0") != std::string::npos); + REQUIRE(ebcRes.body.find("\"TOPIC_COUNT\":0") != std::string::npos); +} + +TEST_CASE("EventBusController: busOfGet serializes one bus") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "orders" }); + httplib::Response ebcRes; + + ebcController.busOfGet(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcAdmin)); + + REQUIRE(ebcRes.body.find("\"NAME\":\"orders\"") != std::string::npos); + REQUIRE(ebcRes.body.find("\"AVAILABLE\":true") != std::string::npos); +} + +TEST_CASE("EventBusController: an unknown bus is 404") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "unknown" }); + httplib::Response ebcRes; + + ebcController.busOfGet(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcAdmin)); + REQUIRE(ebcRes.status == 404); +} + +TEST_CASE("EventBusController: publishOfPost queues the event on the bus") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "orders" }); + ebcReq.body = ebcPublishBody; + httplib::Response ebcRes; + + // a member of the publisher group may publish + ebcController.publishOfPost(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcPublisher)); + + REQUIRE(ebcRes.body.find("\"PUBLISHED\":true") != std::string::npos); + REQUIRE(ebcRes.body.find("\"PRIORITY\":\"High\"") != std::string::npos); + REQUIRE(ebcRes.body.find("\"TARGET\":\"BUS\"") != std::string::npos); + REQUIRE(ebcRes.body.find("\"TOPIC\":\"orders.created\"") != std::string::npos); + + const std::optional ebcEvent = ebcFixture.bus()->receive(); + REQUIRE(ebcEvent.has_value()); + REQUIRE(std::string(ebcEvent->topic()) == "orders.created"); + REQUIRE(std::string(ebcEvent->type()) == "OrderCreated"); + REQUIRE(ebcEvent->priority() == EventPriority::High); + REQUIRE(ebcEvent->contentLength() == 2); +} + +TEST_CASE("EventBusController: publishing needs the publisher or admin group") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "orders" }); + ebcReq.body = ebcPublishBody; + + // a reader is not a publisher + httplib::Response ebcUserRes; + ebcController.publishOfPost(ebcReq, ebcUserRes, ebcJson(), ebcTokenOf(ebcUser)); + REQUIRE(ebcUserRes.status == 401); + REQUIRE(ebcFixture.bus()->isEmpty()); + + // no token at all + httplib::Response ebcAnonymousRes; + ebcController.publishOfPost(ebcReq, ebcAnonymousRes, ebcJson(), std::nullopt); + REQUIRE(ebcAnonymousRes.status == 401); + REQUIRE(ebcFixture.bus()->isEmpty()); +} + +TEST_CASE("EventBusController: an admin may publish") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "orders" }); + ebcReq.body = ebcPublishBody; + httplib::Response ebcRes; + + ebcController.publishOfPost(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcAdmin)); + + REQUIRE(ebcRes.body.find("\"PUBLISHED\":true") != std::string::npos); + REQUIRE(ebcRes.body.find("\"TOPIC\":\"orders.created\"") != std::string::npos); + REQUIRE(!ebcFixture.bus()->isEmpty()); +} + +TEST_CASE("EventBusController: a publisher may not read") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + httplib::Request ebcListReq; + httplib::Response ebcListRes; + ebcController.busesOfGet(ebcListReq, ebcListRes, ebcJson(), ebcTokenOf(ebcPublisher)); + REQUIRE(ebcListRes.status == 401); + + auto ebcBusReq = ebcRequestWithMatches({ "orders" }); + httplib::Response ebcBusRes; + ebcController.busOfGet(ebcBusReq, ebcBusRes, ebcJson(), ebcTokenOf(ebcPublisher)); + REQUIRE(ebcBusRes.status == 401); + + auto ebcDepthReq = ebcRequestWithMatches({ "orders", "worker" }); + httplib::Response ebcDepthRes; + ebcController.depthOfGet(ebcDepthReq, ebcDepthRes, ebcJson(), ebcTokenOf(ebcPublisher)); + REQUIRE(ebcDepthRes.status == 401); +} + +TEST_CASE("EventBusController: publishOfPost broadcasts to the subscribers") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + REQUIRE(ebcFixture.bus()->subscribe("orders.*", "worker")); + auto ebcReq = ebcRequestWithMatches({ "orders" }); + ebcReq.body = R"({"TOPIC":"orders.created","TYPE":"OrderCreated","TARGET":"SUBSCRIBERS"})"; + httplib::Response ebcRes; + + ebcController.publishOfPost(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcPublisher)); + + REQUIRE(ebcRes.body.find("\"PUBLISHED\":true") != std::string::npos); + REQUIRE(ebcRes.body.find("\"TARGET\":\"SUBSCRIBERS\"") != std::string::npos); + const std::optional ebcEvent = ebcFixture.bus()->receiveOf("worker"); + REQUIRE(ebcEvent.has_value()); + REQUIRE(std::string(ebcEvent->topic()) == "orders.created"); + REQUIRE(ebcEvent->priority() == EventPriority::Normal); +} + +TEST_CASE("EventBusController: publishOfPost rejects a topic that does not fit into an event") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "orders" }); + ebcReq.body = std::string(R"({"TOPIC":")") + std::string(Event::NAME_SIZE, 'x') + R"(","TYPE":"T"})"; + httplib::Response ebcRes; + + ebcController.publishOfPost(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcPublisher)); + REQUIRE(ebcRes.status == 400); +} + +TEST_CASE("EventBusController: publishOfPost rejects an unknown priority") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "orders" }); + ebcReq.body = R"({"TOPIC":"orders.created","TYPE":"T","PRIORITY":"urgent"})"; + httplib::Response ebcRes; + + ebcController.publishOfPost(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcPublisher)); + REQUIRE(ebcRes.status == 400); +} + +TEST_CASE("EventBusController: publishOfPost returns 404 for an unknown bus") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "unknown" }); + ebcReq.body = ebcPublishBody; + httplib::Response ebcRes; + + ebcController.publishOfPost(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcPublisher)); + REQUIRE(ebcRes.status == 404); +} + +TEST_CASE("EventBusController: subscribing needs the admin group") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "orders" }); + ebcReq.body = R"({"TOPIC":"orders.created","SUBSCRIBER":"worker"})"; + httplib::Response ebcRes; + + ebcController.subscribeOfPost(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcUser)); + REQUIRE(ebcRes.status == 401); +} + +TEST_CASE("EventBusController: subscribe and unsubscribe change the registry") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "orders" }); + ebcReq.body = R"({"TOPIC":"orders.created","SUBSCRIBER":"worker"})"; + httplib::Response ebcRes; + + ebcController.subscribeOfPost(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcAdmin)); + REQUIRE(ebcRes.body.find("\"SUBSCRIBED\":true") != std::string::npos); + REQUIRE(ebcFixture.bus()->isSubscribed("orders.created", "worker")); + // an exact topic is registered implicitly by the subscribe route + REQUIRE(ebcFixture.bus()->isTopicRegistered("orders.created")); + + auto ebcDeleteReq = ebcRequestWithMatches({ "orders" }); + ebcDeleteReq.body = ebcReq.body; + httplib::Response ebcDeleteRes; + ebcController.unsubscribeOfDelete(ebcDeleteReq, ebcDeleteRes, ebcJson(), ebcTokenOf(ebcAdmin)); + REQUIRE(ebcDeleteRes.body.find("\"UNSUBSCRIBED\":true") != std::string::npos); + REQUIRE(!ebcFixture.bus()->isSubscribed("orders.created", "worker")); +} + +TEST_CASE("EventBusController: depthOfGet reports the subscriber queue depth") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + REQUIRE(ebcFixture.bus()->subscribe("orders.*", "worker")); + Event ebcEvent("orders.created", "OrderCreated"); + REQUIRE(ebcFixture.bus()->publishToSubscribers(ebcEvent)); + REQUIRE(ebcFixture.bus()->publishToSubscribers(ebcEvent)); + + auto ebcReq = ebcRequestWithMatches({ "orders", "worker" }); + httplib::Response ebcRes; + ebcController.depthOfGet(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcUser)); + + REQUIRE(ebcRes.body.find("\"DEPTH\":2") != std::string::npos); + REQUIRE(ebcRes.body.find("\"SUBSCRIBER\":\"worker\"") != std::string::npos); +} + +TEST_CASE("EventBusController: depthOfGet returns 404 for an unknown bus") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "unknown", "worker" }); + httplib::Response ebcRes; + + ebcController.depthOfGet(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcUser)); + REQUIRE(ebcRes.status == 404); +} + +TEST_CASE("EventBusController: resetting needs the admin group") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + auto ebcReq = ebcRequestWithMatches({ "orders" }); + httplib::Response ebcRes; + + ebcController.resetOfPost(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcUser)); + REQUIRE(ebcRes.status == 401); +} + +TEST_CASE("EventBusController: resetOfPost drops the queued events") +{ + EventBusControllerFixture ebcFixture; + EventBusController &ebcController = *ebcFixture.m_controller; + Event ebcEvent("orders.created", "OrderCreated"); + REQUIRE(ebcFixture.bus()->publish(ebcEvent)); + + auto ebcReq = ebcRequestWithMatches({ "orders" }); + httplib::Response ebcRes; + ebcController.resetOfPost(ebcReq, ebcRes, ebcJson(), ebcTokenOf(ebcAdmin)); + + REQUIRE(ebcRes.body.find("\"RESET\":true") != std::string::npos); + REQUIRE(ebcFixture.bus()->isEmpty()); +} +}// namespace lintel diff --git a/tests/http/HistoryControllerTest.cpp b/tests/http/HistoryControllerTest.cpp index 8813b0e8..6579d46c 100644 --- a/tests/http/HistoryControllerTest.cpp +++ b/tests/http/HistoryControllerTest.cpp @@ -16,7 +16,6 @@ #include #include #include -#include #include #include #include @@ -92,7 +91,6 @@ namespace { std::shared_ptr m_envConfig; std::shared_ptr m_configuration; std::shared_ptr m_connectionConfigurations; - std::shared_ptr m_groupRepository; std::shared_ptr m_historyRepository; std::shared_ptr m_authService; std::shared_ptr m_server = std::make_shared(); @@ -133,7 +131,6 @@ namespace { type_name(), m_dbPath, "", "", db::ConnectionType::SQLite, "default", 0, "", true); m_configuration->setEntries(std::vector>{ entry }); m_connectionConfigurations = std::make_shared(m_configuration); - m_groupRepository = std::make_shared(m_connectionConfigurations); m_historyRepository = std::make_shared(m_connectionConfigurations); m_authService = std::make_shared(); } @@ -149,10 +146,7 @@ namespace { HistoryControllerFixture &operator=(const HistoryControllerFixture &) = delete; /** @return a controller wired to the real SQLite history repository */ - HistoryController makeController() const - { - return HistoryController(m_authService, m_groupRepository, m_historyRepository); - } + HistoryController makeController() const { return HistoryController(m_authService, m_historyRepository); } /** Register the routes of the controller, bind an ephemeral port and serve */ void startServing(Controller &controller) diff --git a/tests/http/MessageQueueControllerTest.cpp b/tests/http/MessageQueueControllerTest.cpp index f57ba4e2..939dc7f1 100644 --- a/tests/http/MessageQueueControllerTest.cpp +++ b/tests/http/MessageQueueControllerTest.cpp @@ -529,4 +529,43 @@ TEST_CASE("MessageQueueController::purgeOfPost refuses a pattern that only match REQUIRE(response != nullptr); REQUIRE(response->status == 404); } + +TEST_CASE("MessageQueueController::recoverOfPost reclaims the orphaned entries", "[messagequeue_controller]") +{ + MessageQueueControllerFixture fixture; + auto controller = fixture.makeController(); + fixture.startServing(controller); + REQUIRE(fixture.isReady()); + + // MessageQueueEntry is copyable but not assignable (const members), so no initializer-list assign + fixture.m_repository->orphansToFind.push_back(mqcEntry("worker", "mqc-orphan", 8)); + REQUIRE_CALL(*fixture.m_authService, onAccessOf(ANY(const UserTokenLogin &))) + .TIMES(1) + .LR_RETURN(mqcTokenOf(mqcAdmin, mqcTokenAdmin)); + + auto response = fixture.m_client->Post("/messageQueue/recover", mqcAuthHeaders(mqcTokenAdmin)); + REQUIRE(response != nullptr); + REQUIRE(response->status == 200); + REQUIRE(response->body == R"({"RECOVERED":1})"); + REQUIRE(fixture.m_repository->reclaimedQueues == std::vector{ "mqc-orphan" }); +} + +TEST_CASE("MessageQueueController::recoverOfPost returns 401 for a non-admin user", "[messagequeue_controller]") +{ + MessageQueueControllerFixture fixture; + auto controller = fixture.makeController(); + fixture.startServing(controller); + REQUIRE(fixture.isReady()); + + // MessageQueueEntry is copyable but not assignable (const members), so no initializer-list assign + fixture.m_repository->orphansToFind.push_back(mqcEntry("worker", "mqc-orphan", 8)); + REQUIRE_CALL(*fixture.m_authService, onAccessOf(ANY(const UserTokenLogin &))) + .TIMES(1) + .LR_RETURN(mqcTokenOf(mqcOutsider, mqcTokenOutsider)); + + auto response = fixture.m_client->Post("/messageQueue/recover", mqcAuthHeaders(mqcTokenOutsider)); + REQUIRE(response != nullptr); + REQUIRE(response->status == 401); + REQUIRE(fixture.m_repository->reclaimedQueues.empty()); +} }// namespace lintel diff --git a/tests/http/SharedMemoryControllerMockTest.cpp b/tests/http/SharedMemoryControllerMockTest.cpp index 988cd09e..729022cd 100644 --- a/tests/http/SharedMemoryControllerMockTest.cpp +++ b/tests/http/SharedMemoryControllerMockTest.cpp @@ -15,7 +15,9 @@ #include "lintel/core/models/SharedMemorySegment.h" #include "lintel/core/models/SharedMemorySegmentInfo.h" #include "lintel/features/base/models/Group.h" +#include "lintel/features/base/models/SharedMemorySegmentStats.h" #include "lintel/features/base/models/User.h" +#include "lintel/features/base/services/SharedMemoryCleanupService.h" #include "lintel/features/http/controllers/SharedMemoryController.h" namespace lintel { @@ -44,6 +46,14 @@ namespace { using SharedMemoryController::SharedMemoryController; }; + /** MockSharedMemoryService inherits the interface default of recover() (0); + * this variant makes the call observable for the /shm/recover handler. */ + class ShmRecoverMockService : public MockSharedMemoryService + { + public: + MAKE_MOCK0(recover, std::size_t(), override); + }; + class ShmCtlFakeRepository : public SharedMemoryRepository { public: @@ -100,7 +110,7 @@ TEST_CASE("SharedMemoryController can be constructed with mocks") auto mockAuth = std::make_shared(); auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - REQUIRE_NOTHROW(TestSharedMemoryController(mockAuth, mockShmService, mockShmManager, {})); + REQUIRE_NOTHROW(TestSharedMemoryController(mockAuth, mockShmService, mockShmManager, {}, nullptr)); } TEST_CASE("SharedMemoryController::allSegmentsOfGet calls allOf and showStateOf") @@ -117,7 +127,7 @@ TEST_CASE("SharedMemoryController::allSegmentsOfGet calls allOf and showStateOf" .LR_RETURN(std::vector>{ segment }); REQUIRE_CALL(*mockShmService, showStateOf(segment)).TIMES(1).LR_RETURN(info); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); httplib::Request request; std::smatch m; @@ -142,7 +152,7 @@ TEST_CASE("SharedMemoryController::shrinkSegmentOfPut calls of and shrinkOf") REQUIRE_CALL(*mockShmManager, of("test")).TIMES(1).LR_RETURN(segment); REQUIRE_CALL(*mockShmService, shrinkOf(segment)).TIMES(1); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); httplib::Request request; std::smatch m; @@ -167,7 +177,7 @@ TEST_CASE("SharedMemoryController::growSegmentOfPut calls of and growOf") REQUIRE_CALL(*mockShmManager, of("test")).TIMES(1).LR_RETURN(segment); REQUIRE_CALL(*mockShmService, growOf(segment, 4096)).TIMES(1); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); httplib::Request request; std::smatch m; @@ -187,7 +197,7 @@ TEST_CASE("SharedMemoryController returns 401 for unauthenticated requests") auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); httplib::Request request; httplib::Response response; @@ -205,7 +215,7 @@ TEST_CASE("SharedMemoryController::shrinkSegmentOfPut returns 403 for unknown se REQUIRE_CALL(*mockShmManager, of("unknown")).TIMES(1).LR_RETURN(nullptr); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); httplib::Request request; std::smatch m; @@ -227,7 +237,7 @@ TEST_CASE("SharedMemoryController::allSegmentsOfGet returns 403 for non-json") auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "test" }); httplib::Response response; @@ -243,7 +253,7 @@ TEST_CASE("SharedMemoryController::allSegmentsOfGet returns 401 for user without auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "test" }); httplib::Response response; @@ -262,7 +272,7 @@ TEST_CASE("SharedMemoryController::allRepositoriesOfGet lists matching repositor auto segment = std::make_shared("/tmp/test", "mySeg", 1024); auto repo = std::make_shared(segment, "myRepo", "uuid-1"); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, { repo }); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, { repo }, nullptr); auto request = requestWithMatches({ "myRepo.*", "mySeg.*" }); httplib::Response response; @@ -281,7 +291,7 @@ TEST_CASE("SharedMemoryController::allRepositoriesOfGet skips non-matching repos auto segment = std::make_shared("/tmp/test", "mySeg", 1024); auto repo = std::make_shared(segment, "myRepo", "uuid-1"); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, { repo }); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, { repo }, nullptr); auto request = requestWithMatches({ "nope.*", "nope.*" }); httplib::Response response; @@ -297,7 +307,7 @@ TEST_CASE("SharedMemoryController::allRepositoriesOfGet returns 400 for invalid auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "[", ".*" }); httplib::Response response; @@ -313,7 +323,7 @@ TEST_CASE("SharedMemoryController::allRepositoriesOfGet returns 403 for non-json auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ ".*", ".*" }); httplib::Response response; @@ -329,7 +339,7 @@ TEST_CASE("SharedMemoryController::allRepositoriesOfGet returns 401 without user auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ ".*", ".*" }); httplib::Response response; @@ -348,7 +358,7 @@ TEST_CASE("SharedMemoryController::exportRepositoryOfGet serializes a known repo auto segment = std::make_shared("/tmp/test", "mySeg", 1024); auto repo = std::make_shared(segment, "myRepo", "uuid-1"); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, { repo }); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, { repo }, nullptr); auto request = requestWithMatches({ "uuid-1" }); httplib::Response response; @@ -364,7 +374,7 @@ TEST_CASE("SharedMemoryController::exportRepositoryOfGet returns 404 for unknown auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "missing" }); httplib::Response response; @@ -380,7 +390,7 @@ TEST_CASE("SharedMemoryController::exportRepositoryOfGet returns 403 for non-jso auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "uuid-1" }); httplib::Response response; @@ -396,7 +406,7 @@ TEST_CASE("SharedMemoryController::exportRepositoryOfGet returns 401 without use auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "uuid-1" }); httplib::Response response; @@ -412,7 +422,7 @@ TEST_CASE("SharedMemoryController::growSegmentOfPut returns 400 for invalid size auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "test", "notasize" }); httplib::Response response; @@ -430,7 +440,7 @@ TEST_CASE("SharedMemoryController::growSegmentOfPut returns 403 for unknown segm REQUIRE_CALL(*mockShmManager, of("unknown")).TIMES(1).LR_RETURN(nullptr); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "unknown", "4096" }); httplib::Response response; @@ -446,7 +456,7 @@ TEST_CASE("SharedMemoryController::exportRepositoryOfGet returns 401 for a user auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "uuid-1" }); httplib::Response response; @@ -462,7 +472,7 @@ TEST_CASE("SharedMemoryController::allRepositoriesOfGet returns 401 for a user w auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ ".*", ".*" }); httplib::Response response; @@ -478,7 +488,7 @@ TEST_CASE("SharedMemoryController::shrinkSegmentOfPut returns 401 without a user auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "test" }); httplib::Response response; @@ -494,7 +504,7 @@ TEST_CASE("SharedMemoryController::shrinkSegmentOfPut returns 401 for a user wit auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "test" }); httplib::Response response; @@ -510,7 +520,7 @@ TEST_CASE("SharedMemoryController::shrinkSegmentOfPut returns 403 for non-json") auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "test" }); httplib::Response response; @@ -526,7 +536,7 @@ TEST_CASE("SharedMemoryController::growSegmentOfPut returns 401 without a user") auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "test", "4096" }); httplib::Response response; @@ -542,7 +552,7 @@ TEST_CASE("SharedMemoryController::growSegmentOfPut returns 401 for a user witho auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "test", "4096" }); httplib::Response response; @@ -558,7 +568,7 @@ TEST_CASE("SharedMemoryController::growSegmentOfPut returns 403 for non-json") auto mockShmService = std::make_shared(); auto mockShmManager = std::make_shared(); - TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}); + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); auto request = requestWithMatches({ "test", "4096" }); httplib::Response response; @@ -567,4 +577,181 @@ TEST_CASE("SharedMemoryController::growSegmentOfPut returns 403 for non-json") controller.growSegmentOfPut(request, response, contentType, makeToken()); REQUIRE(response.status == 403); } + +TEST_CASE("SharedMemoryController::statsOfGet serializes the statistics of one segment") +{ + auto mockAuth = std::make_shared(); + auto mockShmService = std::make_shared(); + auto mockShmManager = std::make_shared(); + + auto segment = std::make_shared("/tmp/test", "test", 1024); + SharedMemorySegmentStats stats; + stats.name = "test"; + stats.allocatedBytes = 1024; + stats.capacityBytes = 4096; + stats.freeBytes = 512; + stats.growCount = 2; + stats.shrinkCount = 1; + stats.recoveryCount = 3; + stats.generation = 7; + + REQUIRE_CALL(*mockShmManager, of("test")).TIMES(1).LR_RETURN(segment); + REQUIRE_CALL(*mockShmService, statsOf(segment)).TIMES(1).LR_RETURN(stats); + + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); + + auto request = requestWithMatches({ "test" }); + httplib::Response response; + ContentType contentType("application/json"); + + controller.statsOfGet(request, response, contentType, makeToken()); + // the handler does not set the status on success - the server does + REQUIRE(response.body.find("\"NAME\":\"test\"") != std::string::npos); + REQUIRE(response.body.find("\"ALLOCATED_BYTES\":1024") != std::string::npos); + REQUIRE(response.body.find("\"GENERATION\":7") != std::string::npos); +} + +TEST_CASE("SharedMemoryController::statsOfGet returns 404 for an unknown segment") +{ + auto mockAuth = std::make_shared(); + auto mockShmService = std::make_shared(); + auto mockShmManager = std::make_shared(); + + REQUIRE_CALL(*mockShmManager, of("unknown")).TIMES(1).LR_RETURN(nullptr); + FORBID_CALL(*mockShmService, statsOf(ANY(std::shared_ptr))); + + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); + + auto request = requestWithMatches({ "unknown" }); + httplib::Response response; + ContentType contentType("application/json"); + + controller.statsOfGet(request, response, contentType, makeToken()); + REQUIRE(response.status == 404); +} + +TEST_CASE("SharedMemoryController::statsOfGet returns 401 for a user without groups") +{ + auto mockAuth = std::make_shared(); + auto mockShmService = std::make_shared(); + auto mockShmManager = std::make_shared(); + + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); + + auto request = requestWithMatches({ "test" }); + httplib::Response response; + ContentType contentType("application/json"); + + controller.statsOfGet(request, response, contentType, tokenForGroups({ Group{ "Other", {}, true } })); + REQUIRE(response.status == 401); +} + +TEST_CASE("SharedMemoryController::statsOfGet returns 403 for non-json") +{ + auto mockAuth = std::make_shared(); + auto mockShmService = std::make_shared(); + auto mockShmManager = std::make_shared(); + + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); + + auto request = requestWithMatches({ "test" }); + httplib::Response response; + ContentType contentType("text/plain"); + + controller.statsOfGet(request, response, contentType, makeToken()); + REQUIRE(response.status == 403); +} + +TEST_CASE("SharedMemoryController::cleanupOfPost serializes the counts of the apply run") +{ + auto mockAuth = std::make_shared(); + auto mockShmService = std::make_shared(); + auto mockShmManager = std::make_shared(); + // an empty registry: the run scans nothing and finds nothing + auto cleanup = std::make_shared(nullptr, nullptr); + + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, cleanup); + + httplib::Request request; + httplib::Response response; + ContentType contentType("application/json"); + + controller.cleanupOfPost(request, response, contentType, makeToken()); + // the handler does not set the status on success - the server does + REQUIRE(response.body == R"({"DRY_RUN":false,"SCANNED":0,"ORPHANS":0,"REMOVED":0,"FAILED":0})"); +} + +TEST_CASE("SharedMemoryController::cleanupOfPost returns 401 for a non-admin user") +{ + auto mockAuth = std::make_shared(); + auto mockShmService = std::make_shared(); + auto mockShmManager = std::make_shared(); + auto cleanup = std::make_shared(nullptr, nullptr); + + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, cleanup); + + httplib::Request request; + httplib::Response response; + ContentType contentType("application/json"); + + controller.cleanupOfPost(request, response, contentType, tokenForGroups({ kUserGroup })); + REQUIRE(response.status == 401); +} + +TEST_CASE("SharedMemoryController::recoverOfPost answers with the number of reclaimed locks") +{ + auto mockAuth = std::make_shared(); + auto mockShmService = std::make_shared(); + auto mockShmManager = std::make_shared(); + + REQUIRE_CALL(*mockShmService, recover()).TIMES(1).LR_RETURN(std::size_t{ 2 }); + + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); + + httplib::Request request; + httplib::Response response; + ContentType contentType("application/json"); + + controller.recoverOfPost(request, response, contentType, makeToken()); + // the handler does not set the status on success - the server does + REQUIRE(response.body == R"({"RECOVERED":2})"); +} + +TEST_CASE("SharedMemoryController::recoverOfPost returns 401 for a non-admin user") +{ + auto mockAuth = std::make_shared(); + auto mockShmService = std::make_shared(); + auto mockShmManager = std::make_shared(); + + // recovery mutates the lock state of a segment: a user without the admin + // group must not reach the service at all + FORBID_CALL(*mockShmService, recover()); + + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); + + httplib::Request request; + httplib::Response response; + ContentType contentType("application/json"); + + controller.recoverOfPost(request, response, contentType, tokenForGroups({ kUserGroup })); + REQUIRE(response.status == 401); +} + +TEST_CASE("SharedMemoryController::recoverOfPost returns 403 for non-json") +{ + auto mockAuth = std::make_shared(); + auto mockShmService = std::make_shared(); + auto mockShmManager = std::make_shared(); + + FORBID_CALL(*mockShmService, recover()); + + TestSharedMemoryController controller(mockAuth, mockShmService, mockShmManager, {}, nullptr); + + httplib::Request request; + httplib::Response response; + ContentType contentType("text/plain"); + + controller.recoverOfPost(request, response, contentType, makeToken()); + REQUIRE(response.status == 403); +} }// namespace lintel diff --git a/tests/mocks/MockPropertyRepository.h b/tests/mocks/MockPropertyRepository.h index bba6d543..bb638bc3 100644 --- a/tests/mocks/MockPropertyRepository.h +++ b/tests/mocks/MockPropertyRepository.h @@ -16,12 +16,7 @@ class MockPropertyRepository : public PropertyRepository MAKE_MOCK1(save, void(std::shared_ptr), override); MAKE_MOCK1(deleteOf, void(const std::vector> &), override); MAKE_MOCK0(awake, std::vector>(), override); - MAKE_MOCK4(allOf, - std::vector>(const std::string &, - const std::string &, - const std::string &, - const std::string &), - override); + MAKE_MOCK1(allOf, std::vector>(const PropertyFilter &), override); }; }// namespace lintel diff --git a/tests/mocks/MockSharedMemoryService.h b/tests/mocks/MockSharedMemoryService.h index 22e626ce..bb18b1cb 100644 --- a/tests/mocks/MockSharedMemoryService.h +++ b/tests/mocks/MockSharedMemoryService.h @@ -5,12 +5,14 @@ #include "lintel/core/models/SharedMemorySegmentInfo.h" #include "lintel/core/services/ISharedMemoryService.h" +#include "lintel/features/base/models/SharedMemorySegmentStats.h" namespace lintel { class MockSharedMemoryService : public ISharedMemoryService { public: MAKE_CONST_MOCK1(showStateOf, SharedMemorySegmentInfo(const std::shared_ptr &), override); + MAKE_CONST_MOCK1(statsOf, SharedMemorySegmentStats(const std::shared_ptr &), override); MAKE_MOCK2(growOf, void(const std::shared_ptr &, std::size_t), override); MAKE_MOCK1(shrinkOf, void(const std::shared_ptr &), override); MAKE_MOCK0(onInitialize, void(), override); diff --git a/tests/property/DatabasePropertyRepositoryTest.cpp b/tests/property/DatabasePropertyRepositoryTest.cpp index 8015bf6b..a5826a8a 100644 --- a/tests/property/DatabasePropertyRepositoryTest.cpp +++ b/tests/property/DatabasePropertyRepositoryTest.cpp @@ -192,16 +192,16 @@ TEST_CASE("DatabasePropertyRepository::allOf filters by process, class, instance fixture.makeProperty("dprDelta", "__DEFAULT", "DprService", "otherProcess", 4) }); // all four filters are passed explicitly so that the individual LIKE clauses are pinned here; - // the defaults of the header are exercised by the regression test below - REQUIRE(fixture.m_repository->allOf(".*", ".*", ".*", ".*").size() == 4); - REQUIRE(fixture.m_repository->allOf("dprProcess", ".*", ".*", ".*").size() == 3); - REQUIRE(fixture.m_repository->allOf("dprProcess", "DprService", ".*", ".*").size() == 2); - REQUIRE(fixture.m_repository->allOf("dprProcess", "DprService", "second").size() == 1); - REQUIRE(fixture.m_repository->allOf("dprProcess", "DprService", ".*", "dprAlpha").size() == 1); - REQUIRE(fixture.m_repository->allOf("dprProcess", "DprService", ".*", "dprA.*").size() == 1); - REQUIRE(fixture.m_repository->allOf("nobody", ".*", ".*", ".*").empty()); - - const auto alpha = fixture.m_repository->allOf("dprProcess", "DprService", ".*", "dprAlpha"); + // the aggregate defaults are exercised by the regression test below + REQUIRE(fixture.m_repository->allOf(PropertyFilter{ ".*", ".*", ".*", ".*" }).size() == 4); + REQUIRE(fixture.m_repository->allOf(PropertyFilter{ "dprProcess", ".*", ".*", ".*" }).size() == 3); + REQUIRE(fixture.m_repository->allOf(PropertyFilter{ "dprProcess", "DprService", ".*", ".*" }).size() == 2); + REQUIRE(fixture.m_repository->allOf(PropertyFilter{ "dprProcess", "DprService", "second" }).size() == 1); + REQUIRE(fixture.m_repository->allOf(PropertyFilter{ "dprProcess", "DprService", ".*", "dprAlpha" }).size() == 1); + REQUIRE(fixture.m_repository->allOf(PropertyFilter{ "dprProcess", "DprService", ".*", "dprA.*" }).size() == 1); + REQUIRE(fixture.m_repository->allOf(PropertyFilter{ "nobody", ".*", ".*", ".*" }).empty()); + + const auto alpha = fixture.m_repository->allOf(PropertyFilter{ "dprProcess", "DprService", ".*", "dprAlpha" }); REQUIRE(alpha.size() == 1); REQUIRE(alpha[0]->toString() == "1"); } @@ -215,9 +215,11 @@ TEST_CASE("DatabasePropertyRepository::allOf matches every instance for the defa fixture.m_repository->save(fixture.makeProperty("dprAlpha", "__DEFAULT", "DprService", "dprProcess", 1)); fixture.m_repository->save(fixture.makeProperty("dprBeta", "second", "DprService", "dprProcess", 2)); - REQUIRE(fixture.m_repository->allOf("dprProcess", "DprService").size() == 2); - REQUIRE(fixture.m_repository->allOf("dprProcess", "DprService", ".*", ".*").size() == 2); - REQUIRE(fixture.m_repository->allOf().size() == 2); + REQUIRE( + fixture.m_repository->allOf(PropertyFilter{ .m_processName = "dprProcess", .m_className = "DprService" }).size() + == 2); + REQUIRE(fixture.m_repository->allOf(PropertyFilter{ "dprProcess", "DprService", ".*", ".*" }).size() == 2); + REQUIRE(fixture.m_repository->allOf(PropertyFilter{}).size() == 2); } TEST_CASE("DatabasePropertyRepository::deleteOf removes the given properties", "[database_property_repository]") @@ -287,9 +289,9 @@ TEST_CASE("DatabasePropertyRepository::allOf returns an empty result for a broke fixture.breakConnection(); std::vector> allOfResult; - REQUIRE_NOTHROW(allOfResult = fixture.m_repository->allOf()); + REQUIRE_NOTHROW(allOfResult = fixture.m_repository->allOf(PropertyFilter{})); REQUIRE(allOfResult.empty()); - REQUIRE(fixture.m_repository->allOf("dprProcess", "DprService", ".*", "dpr.*").empty()); + REQUIRE(fixture.m_repository->allOf(PropertyFilter{ "dprProcess", "DprService", ".*", "dpr.*" }).empty()); } TEST_CASE("DatabasePropertyRepository::deleteOf swallows a db::SQLException of a broken connection", diff --git a/tests/property/PropertyChangeTest.cpp b/tests/property/PropertyChangeTest.cpp index 47e4ed72..b25704aa 100644 --- a/tests/property/PropertyChangeTest.cpp +++ b/tests/property/PropertyChangeTest.cpp @@ -296,8 +296,8 @@ TEST_CASE("shared memory property repository round-trips saved properties") REQUIRE(awake.size() == 1); REQUIRE(awake.front()->getName() == "value"); - REQUIRE(fixture.repository->allOf("testProcess", "className", "instance", "value").size() == 1); - REQUIRE(fixture.repository->allOf("otherProcess", ".*", ".*", ".*").empty()); + REQUIRE(fixture.repository->allOf(PropertyFilter{ "testProcess", "className", "instance", "value" }).size() == 1); + REQUIRE(fixture.repository->allOf(PropertyFilter{ "otherProcess", ".*", ".*", ".*" }).empty()); // deliberate no-op migration hook fixture.repository->onMigrate(0); diff --git a/tests/property/PropertyServiceRepositoryTest.cpp b/tests/property/PropertyServiceRepositoryTest.cpp index 42819de3..2c7835a9 100644 --- a/tests/property/PropertyServiceRepositoryTest.cpp +++ b/tests/property/PropertyServiceRepositoryTest.cpp @@ -85,8 +85,7 @@ TEST_CASE("PropertyService: allOf delegates to the shadow repository when one is ALLOW_CALL(*shadowRepository, awake()).LR_RETURN(PropertyVector{}); ALLOW_CALL(*shadowRepository, save(ANY(std::vector>))); - ALLOW_CALL(*shadowRepository, allOf(trompeloeil::_, trompeloeil::_, trompeloeil::_, trompeloeil::_)) - .LR_RETURN(PropertyVector{ propSvcProperty("shadowed") }); + ALLOW_CALL(*shadowRepository, allOf(trompeloeil::_)).LR_RETURN(PropertyVector{ propSvcProperty("shadowed") }); PropertyService service({ shadowRepository }, {}); service.onAwake(); diff --git a/tests/services/ProcessServiceTest.cpp b/tests/services/ProcessServiceTest.cpp index 06261c46..3dbf8423 100644 --- a/tests/services/ProcessServiceTest.cpp +++ b/tests/services/ProcessServiceTest.cpp @@ -688,6 +688,34 @@ TEST_CASE("ProcessService restarts a whole process group", "[process_service]") #if !defined(_WIN32) // Windows has no SIGINT trap in a child process (SignalService::kill terminates the process outright), // so the scenario this test proves only exists on POSIX. +TEST_CASE("ProcessService::onShutdown stops the processes it started", "[process_service]") +{ + // Regression: the children were only stopped in the destructor, and the + // destructor never ran because a bootstrap plugin kept the DI container (and + // with it every service) alive. Shutdown now stops them itself. + ProcessServiceFixture fixture; + auto service = fixture.createService(); + + Process process = fixture.makeSleepProcess(); + service->startOf(process); + std::this_thread::sleep_for(std::chrono::milliseconds(150)); + + const auto before = service->allActiveOf(); + REQUIRE(before.size() == 2);// sleep + self + lintel::ProcessId childId = 0; + for (const auto &info : before) { + if (info.getProcessId() != service->currentOf().getProcessId()) { childId = info.getProcessId(); } + } + REQUIRE(childId > 0); + + service->onShutdown(); + + // the child must be gone, not merely marked stopped + for (const auto &info : service->allActiveOf()) { + if (info.getProcessId() == childId) { REQUIRE_FALSE(info.isRunning()); } + } +} + TEST_CASE("ProcessService destructor force-terminates a process that ignores SIGINT", "[process_service]") { ProcessServiceFixture fixture;