feat(plugins): table de vérité persistante, ramasse-miette forcé et supervision IPC/events - #279
Merged
Merged
Conversation
…upervision IPC/events
Activation/désactivation des plugins :
- PluginStateStore (JSON) persiste l'état actif/inactif, consulté par
PluginLoader.load_all() pour sauter les plugins désactivés au boot
- PluginSupervisor.enable()/disable(reason=) togglent en live + persistent
- registry_table() expose la table de vérité complète (disque + persisté +
état live), y compris les plugins désactivés ou jamais chargés
Ramasse-miette forcé au unload/disable (PluginResourceTracker) :
- jobs scheduler, health checks, abonnements events/hooks désinscrits de
force, indépendamment de la qualité des hooks on_stop/on_unload du plugin
- PluginRegistry.unregister() enfin appelé (existait, jamais invoqué)
- ctx.spawn_task() pour des tâches de fond suivies et annulées au unload
- routes HTTP démontées au unload (pas seulement au reload)
Centre de contrôle HTTP sur /plugins/ipc/* :
- GET /registry, POST /{name}/enable, POST /{name}/disable
- GET /audit (PluginSupervisor.ipc_audit()/ipc_stats())
- GET /events (EventBus.recent_emissions()/stats(), HookManager.recent_emissions())
Corrections (vérification reports/technical_debt_remediation_verification_2026-08-11.md) :
- propagate_services() cassait le reload d'un plugin TrustedBase (collision
sur un service protégé reçu en injection, pas exporté)
- SandboxProcessManager.call() ne recyclait pas le subprocess sur
IPCTimeoutError, seulement sur IPCProcessDead
- PermissionEngine(audit_cache_hits=False) pour sauter l'audit log sur
cache hit (optionnel, défaut inchangé)
- tests/conftest.py : préfixe xcore_test_ uniforme + fixture autouse de
nettoyage des dossiers temporaires orphelins en cas de crash dur
Contributor
Reviewer's GuideCette PR introduit un cycle de vie plugin supervisé de bout en bout : l’état enable/disable est persistant, les ressources et routes sont retirées de force à l’unload, et un centre HTTP expose l’état, les audits IPC et l’activité events/hooks. Elle ajoute aussi des protections de robustesse pour le reload TrustedBase, les timeouts sandbox, l’audit des permissions et le nettoyage des tests, avec une couverture unitaire et d’intégration dédiée. Sequence diagram for persistent plugin disable and restartsequenceDiagram
participant Admin
participant API as PluginAPI
participant Supervisor as PluginSupervisor
participant Store as PluginStateStore
participant Lifecycle as LifecycleManager
participant Loader as PluginLoader
Admin->>API: POST /{plugin_name}/disable
API->>Supervisor: disable(plugin_name, reason)
Supervisor->>Store: set_enabled(plugin_name, false, reason)
Store-->>Supervisor: state persisted
Supervisor->>Lifecycle: unload(plugin_name)
Lifecycle-->>Supervisor: plugin unloaded
Supervisor-->>API: status ok
API-->>Admin: disabled
Note over Loader,Store: Process restart
Loader->>Store: is_enabled(plugin_name)
Store-->>Loader: false
Loader-->>Loader: skip disabled plugin
Sequence diagram for forced plugin resource cleanupsequenceDiagram
participant Plugin
participant Context as PluginContext
participant Tracker as PluginResourceTracker
participant Lifecycle as LifecycleManager
participant Registry as PluginRegistry
Lifecycle->>Tracker: wrap scheduler, health, events, hooks
Lifecycle->>Context: assign _task_sink
Plugin->>Context: spawn_task(coro)
Context->>Context: track asyncio.Task
Plugin->>Tracker: register jobs, checks, subscriptions
Lifecycle->>Plugin: on_stop/on_unload()
Plugin-->>Lifecycle: success or error
Lifecycle->>Tracker: cleanup()
Tracker-->>Tracker: remove jobs, checks, events, hooks
Lifecycle->>Context: cancel spawned tasks
Lifecycle->>Registry: unregister(plugin_name)
Flow diagram for plugin unload route removalflowchart TD
Unload["Plugin unload or disable"] --> Event["plugin.*.unloaded"]
Event --> Handler["_on_plugin_unloaded(event)"]
Handler --> Unmount["_unmount_plugin_router(plugin_name)"]
Unmount --> Routes["Remove plugin paths from app.routes"]
Routes --> OpenAPI["Invalidate app.openapi_schema"]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've found 3 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="xcore/kernel/runtime/lifecycle.py" line_range="195-208" />
<code_context>
isolate_scheduler=tenancy.isolate_scheduler,
)
+ # Ramasse-miette forcé : on intercepte scheduler/health/events/hooks
+ # pour mémoriser ce que CE plugin y enregistre pendant on_load, afin
+ # de tout désinscrire au unload sans dépendre de on_unload/on_stop.
+ tracker = PluginResourceTracker(self.manifest.name)
+ ctx.events = tracker.wrap_events(ctx.events)
+ ctx.hooks = tracker.wrap_hooks(ctx.hooks)
+ ctx.health = tracker.wrap_health(ctx.health)
+ if ctx.services.get("scheduler") is not None:
+ ctx.services = dict(ctx.services)
+ ctx.services["scheduler"] = tracker.wrap_scheduler(
+ ctx.services["scheduler"]
+ )
+ self._resource_tracker = tracker
+ ctx._task_sink = self._spawned_tasks
+
if hasattr(self._instance, "_inject_context"):
</code_context>
<issue_to_address>
**issue (bug_risk):** Resources registered during `_inject_context` or `on_load` leak when loading fails, because the tracker is created and installed before those hooks but cleanup only runs from `_do_unload()`, which is not invoked for a failed load.
**Triggers:** When a plugin raises during context injection or `on_init`/`on_load`/`on_start`.
**Suggested fix:** Run tracker cleanup, task cancellation, registry cleanup, and module cleanup from the load-error path before propagating the load failure.
</issue_to_address>
### Comment 2
<location path="xcore/registry/state_store.py" line_range="46-61" />
<code_context>
+ if not self._path.exists():
+ return
+ try:
+ self._data = json.loads(self._path.read_text(encoding="utf-8")) or {}
+ except (json.JSONDecodeError, OSError) as e:
+ logger.warning(
+ "plugin state file unreadable, starting empty",
+ path=str(self._path),
+ error=str(e),
+ )
+ self._data = {}
+
+ def _persist(self) -> None:
+ self._path.parent.mkdir(parents=True, exist_ok=True)
+ tmp_path = self._path.with_suffix(f"{self._path.suffix}.tmp")
+ tmp_path.write_text(
+ json.dumps(self._data, indent=2, sort_keys=True), encoding="utf-8"
+ )
+ os.replace(tmp_path, self._path)
+
+ def is_enabled(self, name: str, default: bool = True) -> bool:
</code_context>
<issue_to_address>
**issue (bug_risk):** Concurrent enable/disable requests write through the same fixed temporary path without locking, so one request can overwrite another's in-memory state or cause `os.replace` to race on a temporary file; the persisted truth table is not concurrency-safe.
**Triggers:** When two HTTP enable/disable operations update plugin state concurrently.
**Suggested fix:** Serialize read-modify-write operations with a process-level lock and use a unique temporary filename per atomic write.
</issue_to_address>
### Comment 3
<location path="xcore/registry/state_store.py" line_range="46" />
<code_context>
+ self._data = json.loads(self._path.read_text(encoding="utf-8")) or {}
</code_context>
<issue_to_address>
**issue (bug_risk):** A syntactically valid JSON file containing a non-object, such as `[]` or `null` with the fallback not applying to the wrong shape, leaves `_data` with a non-dictionary value and later `is_enabled()` raises `AttributeError` on `.get`.
**Triggers:** When the state file is truncated or replaced with valid JSON of the wrong shape.
**Suggested fix:** Validate that the loaded value is a dictionary of plugin-state dictionaries and reset invalid shapes to `{}`.
```suggestion
data = json.loads(self._path.read_text(encoding="utf-8"))
self._data = (
data
if isinstance(data, dict)
and all(isinstance(value, dict) for value in data.values())
else {}
)
```
</issue_to_address>Sourcery assessment
Needs a human reviewer. 3 findings to address first, and this changes persistent plugin activation policy and runtime reachability: a faulty enable/disable decision could expose or withdraw plugin IPC and HTTP behavior, while unload cleanup could also remove resources or routes incorrectly. Reverting the code would not automatically undo the persisted state or effects that occurred while a plugin was enabled or disabled.
Blocking findings: xcore/kernel/runtime/lifecycle.py:208, xcore/registry/state_store.py:61, xcore/registry/state_store.py:46
Comment on lines
+195
to
+208
| # Ramasse-miette forcé : on intercepte scheduler/health/events/hooks | ||
| # pour mémoriser ce que CE plugin y enregistre pendant on_load, afin | ||
| # de tout désinscrire au unload sans dépendre de on_unload/on_stop. | ||
| tracker = PluginResourceTracker(self.manifest.name) | ||
| ctx.events = tracker.wrap_events(ctx.events) | ||
| ctx.hooks = tracker.wrap_hooks(ctx.hooks) | ||
| ctx.health = tracker.wrap_health(ctx.health) | ||
| if ctx.services.get("scheduler") is not None: | ||
| ctx.services = dict(ctx.services) | ||
| ctx.services["scheduler"] = tracker.wrap_scheduler( | ||
| ctx.services["scheduler"] | ||
| ) | ||
| self._resource_tracker = tracker | ||
| ctx._task_sink = self._spawned_tasks |
Contributor
There was a problem hiding this comment.
issue (bug_risk): Resources registered during _inject_context or on_load leak when loading fails, because the tracker is created and installed before those hooks but cleanup only runs from _do_unload(), which is not invoked for a failed load.
Triggers: When a plugin raises during context injection or on_init/on_load/on_start.
Suggested fix: Run tracker cleanup, task cancellation, registry cleanup, and module cleanup from the load-error path before propagating the load failure.
Comment on lines
+46
to
+61
| self._data = json.loads(self._path.read_text(encoding="utf-8")) or {} | ||
| except (json.JSONDecodeError, OSError) as e: | ||
| logger.warning( | ||
| "plugin state file unreadable, starting empty", | ||
| path=str(self._path), | ||
| error=str(e), | ||
| ) | ||
| self._data = {} | ||
|
|
||
| def _persist(self) -> None: | ||
| self._path.parent.mkdir(parents=True, exist_ok=True) | ||
| tmp_path = self._path.with_suffix(f"{self._path.suffix}.tmp") | ||
| tmp_path.write_text( | ||
| json.dumps(self._data, indent=2, sort_keys=True), encoding="utf-8" | ||
| ) | ||
| os.replace(tmp_path, self._path) |
Contributor
There was a problem hiding this comment.
issue (bug_risk): Concurrent enable/disable requests write through the same fixed temporary path without locking, so one request can overwrite another's in-memory state or cause os.replace to race on a temporary file; the persisted truth table is not concurrency-safe.
Triggers: When two HTTP enable/disable operations update plugin state concurrently.
Suggested fix: Serialize read-modify-write operations with a process-level lock and use a unique temporary filename per atomic write.
| if not self._path.exists(): | ||
| return | ||
| try: | ||
| self._data = json.loads(self._path.read_text(encoding="utf-8")) or {} |
Contributor
There was a problem hiding this comment.
issue (bug_risk): A syntactically valid JSON file containing a non-object, such as [] or null with the fallback not applying to the wrong shape, leaves _data with a non-dictionary value and later is_enabled() raises AttributeError on .get.
Triggers: When the state file is truncated or replaced with valid JSON of the wrong shape.
Suggested fix: Validate that the loaded value is a dictionary of plugin-state dictionaries and reset invalid shapes to {}.
Suggested change
| self._data = json.loads(self._path.read_text(encoding="utf-8")) or {} | |
| data = json.loads(self._path.read_text(encoding="utf-8")) | |
| self._data = ( | |
| data | |
| if isinstance(data, dict) | |
| and all(isinstance(value, dict) for value in data.values()) | |
| else {} | |
| ) |
…onisée - xcore/__version__.py était figé à 2.3.3 (désynchronisé de pyproject.toml depuis plusieurs releases) — aligné sur 2.6.0 - pyproject.toml : retrait de uvicorn, pydantic-settings, rich (zéro usage dans xcore/, déjà couverts transitivement par fastapi[standard] pour les deux premiers) ; pydantic[email] → pydantic (EmailStr jamais utilisé). poetry.lock régénéré en conséquence, aucun changement de comportement. - roadmap/executed_roadmap.md (FR) affichait V3 60% / V4 15% alors que son propre tableau détaillé (identique à la version EN) ne justifie que 25% / 5% — recompté et aligné sur ROADMAP_PROGRESS.md
github-actions
Bot
added
documentation
Contributor
|
|
Contributor
Résout les conflits (main a avancé à 2.5.3 pendant ce travail — fix shutdown()/AutoDispatchMixin/PermissionEngine LRU/TenantAware wrappers/ version sync/RedisCacheBackend.clear()/sanitisation des logs) : - xcore/__version__.py, pyproject.toml : garde 2.6.0 (postérieur à 2.5.3) - xcore/kernel/permissions/engine.py : combine le cache LRU borné (OrderedDict, main) avec le toggle audit_cache_hits (cette branche) - xcore/kernel/runtime/loader.py : auto-mergé proprement (fix shutdown() de main + PluginStateStore de cette branche touchaient des sections disjointes) - CHANGELOG.md / doc/changelog.md : entrée 2.6.0 en tête, 2.5.3 conservée en dessous - xcore/kernel/observability/logging.py : reformatage black (fichier touché par le fix sécurité clear-text logging, non conforme au push)
traoreera
force-pushed
the
feature/plugin-supervision-and-gc
branch
from
September 28, 2026 14:35
77ec7f0 to
54924ad
Compare
Contributor
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Résumé
PluginStateStore) —PluginLoader.load_all()saute désormais les plugins désactivés au boot, etPluginSupervisor.enable()/disable(reason=)togglent l'état en live tout en le persistant (survit à un redémarrage du process).PluginResourceTracker) : jobs scheduler, health checks et abonnements events/hooks sont désinscrits de force, indépendamment de la qualité des hookson_stop/on_unloaddu plugin.PluginRegistry.unregister()(qui existait mais n'était jamais appelé) est enfin invoqué. Nouveauctx.spawn_task()pour les tâches de fond suivies et annulées au unload. Les routes HTTP d'un plugin sont maintenant démontées au unload (pas seulement au reload)./plugins/ipc/*:GET /registry(table de vérité complète),POST /{name}/enable,POST /{name}/disable,GET /audit(journal des appels IPC),GET /events(activité EventBus/HookManager).reports/technical_debt_remediation_verification_2026-08-11.mdcontre le code actuel :propagate_services()cassait le reload/réactivation d'un pluginTrustedBase(collision sur un service protégé reçu en injection, pas exporté par le plugin)SandboxProcessManager.call()ne recyclait le subprocess que surIPCProcessDead, jamais surIPCTimeoutError— recyclage différé au prochain health check au lieu d'être immédiatPermissionEngine(audit_cache_hits=False)— option pour sauter l'audit log sur cache hit (défaut inchangé, comportement historique préservé)tests/conftest.py: préfixexcore_test_uniforme + fixture autouse de nettoyage des dossiers temporaires orphelins en cas de crash dur d'un testTest plan
poetry run pytest tests/— suite complète verte (1540 passed)make lint-check— clean sur les fichiers touchéstests/integration/test_plugin_enable_disable.py)Summary by Sourcery
Strengthen plugin lifecycle control and operational visibility with persistent activation state, forced cleanup, HTTP supervision, and targeted reliability fixes.
New Features:
Bug Fixes:
Enhancements:
Build:
Documentation:
Tests:
Chores: