diff --git a/README.md b/README.md index 15dcdc4..c94231c 100644 --- a/README.md +++ b/README.md @@ -56,6 +56,7 @@ Students and researchers face significant barriers when working with different s - **Reproducibility issues** – Sharing and reproducing experiments is frequently cumbersome and error-prone. - **Scaling friction** – Moving from a local prototype to a cloud environment or dedicated compute cluster can be slow and difficult. + ## The Solution Fluidize provides a standardized wrapper that turns complex scientific software into modular components. This makes it possible to: @@ -66,12 +67,14 @@ Fluidize provides a standardized wrapper that turns complex scientific software All of this works with **minimal or no changes** to the existing codebase, allowing our framework to scale effortlessly to any repository. + ## Architecture At Fluidize, we believe strong organization leads to better reproducibility and scalability. We treat each simulation pipeline as an individual project. Within projects, each pipeline is treated as a DAG (directed acyclic graph), where nodes represent individual pieces of scientific software (e.g. inputs, solvers, visualization tools, etc.) and edges represent data flow between nodes. + ### Nodes Nodes are the foundational building blocks of simulation pipelines. Each node represents a computational unit with: @@ -90,6 +93,7 @@ Nodes are the foundational building blocks of simulation pipelines. Each node re - No source code modification required - Automated node generation support (Public launch soon) + ### Projects Projects store a simple data layer for managing individual modules within a pipeline. @@ -99,10 +103,12 @@ Projects store a simple data layer for managing individual modules within a pipe | `graph.json` | Node (scientific software) and edge (data flow) definitions | | `metadata.yaml` | Project description and configuration | + ### Runs Pipelines can be executed both locally and on the cloud. Local execution is handled by Docker engine. Cloud execution is routed through our API, and uses the Kubernetes engine with Argo Workflow Manager. + ## Documentation Comprehensive documentation is available at [https://Fluidize-Inc.github.io/fluidize-python/](https://Fluidize-Inc.github.io/fluidize-python/) @@ -112,12 +118,14 @@ Comprehensive documentation is available at [https://Fluidize-Inc.github.io/flui - [Project Orchestration](https://Fluidize-Inc.github.io/fluidize-python/projects) - [API Reference](https://Fluidize-Inc.github.io/fluidize-python/api) + ## Contributing We would love to collaborate with you! Please see our [Contributing Guide](CONTRIBUTING.md) for details. Also - we would love to help streamline your pipeline! Please reach out to us at [founders@fluidize.ai](mailto:founders@fluidize.ai). + ## Vision and Roadmap This is just the beginning of what we believe will be a really exciting new era for how we conduct research and make discoveries in science. @@ -127,6 +135,7 @@ By standardizing tools, we hope to significantly increase the effectiveness of A - **Auto-Fluidize**: Automatically convert any scientific software to run anywhere with our framework. - **Fluidize AI Playground**: Explore and build simulation pipelines with natural language. + ## License This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. diff --git a/docs/getting-started/client.md b/docs/core-modules/client.md similarity index 81% rename from docs/getting-started/client.md rename to docs/core-modules/client.md index 197a6b7..2c9bf16 100644 --- a/docs/getting-started/client.md +++ b/docs/core-modules/client.md @@ -6,22 +6,24 @@ The Fluidize Client is the primary interface to create and edit projects. There - **API Mode**: Runs on Fluidize API to manage projects and workflows in the cloud. -## Client API - -### FluidizeClient ::: fluidize.client.FluidizeClient options: show_source: false + heading_level: 3 + extra: + show_root_heading: true members: - mode - adapters - projects - runs -### FluidizeConfig ::: fluidize.config.FluidizeConfig options: show_source: false + heading_level: 3 + extra: + show_root_heading: true members: - is_local_mode - is_api_mode diff --git a/docs/core-modules/execute.md b/docs/core-modules/execute.md deleted file mode 100644 index 726fe6e..0000000 --- a/docs/core-modules/execute.md +++ /dev/null @@ -1,25 +0,0 @@ -# Execute Module - -## Execution Management - -### ExecutionManager -::: fluidize.core.modules.execute.ExecutionManager - options: - show_source: false - -## Execution Utilities - -### PathConverter -::: fluidize.core.modules.execute.PathConverter - options: - show_source: false - -### EnvironmentBuilder -::: fluidize.core.modules.execute.EnvironmentBuilder - options: - show_source: false - -### VolumeBuilder -::: fluidize.core.modules.execute.VolumeBuilder - options: - show_source: false diff --git a/docs/core-modules/graph.md b/docs/core-modules/graph.md index 101975a..a7d26d0 100644 --- a/docs/core-modules/graph.md +++ b/docs/core-modules/graph.md @@ -1,8 +1,12 @@ # Graph Module +## Graph Manager ::: fluidize.managers.graph.GraphManager options: show_source: false + heading_level: 3 + extra: + show_root_heading: true members: - get - add_node @@ -11,29 +15,35 @@ - add_edge - delete_edge +## Graph Processor ::: fluidize.core.modules.graph.GraphProcessor options: show_source: false + heading_level: 3 + extra: + show_root_heading: true ## Graph Types -### GraphData -::: fluidize.core.types.GraphData +::: fluidize.core.types.graph.GraphData options: - show_source: false - show_root_heading: true heading_level: 3 + extra: + show_attributes: true + show_root_heading: true + -### GraphNode -::: fluidize.core.types.GraphNode +::: fluidize.core.types.graph.GraphNode options: - show_source: false - show_root_heading: true heading_level: 3 + extra: + show_attributes: true + show_root_heading: true + -### GraphEdge -::: fluidize.core.types.GraphEdge +::: fluidize.core.types.graph.GraphEdge options: - show_source: false - show_root_heading: true heading_level: 3 + extra: + show_attributes: true + show_root_heading: true diff --git a/docs/core-modules/projects.md b/docs/core-modules/projects.md index 5c97442..e51384c 100644 --- a/docs/core-modules/projects.md +++ b/docs/core-modules/projects.md @@ -4,12 +4,14 @@ ::: fluidize.managers.registry.RegistryManager options: show_source: false - show_root_heading: true heading_level: 3 + extra: + show_root_heading: true ## Project ::: fluidize.managers.project.ProjectManager options: show_source: false - show_root_heading: true heading_level: 3 + extra: + show_root_heading: true diff --git a/docs/core-modules/run.md b/docs/core-modules/run.md index 55675e1..c21122f 100644 --- a/docs/core-modules/run.md +++ b/docs/core-modules/run.md @@ -2,10 +2,12 @@ ## Run Management -### RunsManager ::: fluidize.managers.runs.RunsManager options: show_source: false + heading_level: 3 + extra: + show_root_heading: true members: - run_flow - list @@ -13,12 +15,16 @@ ## Run Execution -### RunJob ::: fluidize.core.modules.run.RunJob options: show_source: false + heading_level: 3 + extra: + show_root_heading: true -### ProjectRunner -::: fluidize.core.modules.run.ProjectRunner +::: fluidize.core.modules.run.project.ProjectRunner options: show_source: false + heading_level: 3 + extra: + show_root_heading: true diff --git a/docs/getting-started/projects-nodes.md b/docs/getting-started/projects-nodes.md deleted file mode 100644 index 8d24def..0000000 --- a/docs/getting-started/projects-nodes.md +++ /dev/null @@ -1,5 +0,0 @@ -# Projects and Nodes - -*Content coming soon...* - -For API details, see the [Projects module documentation](../core-modules/projects.md). diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md new file mode 100644 index 0000000..bad5562 --- /dev/null +++ b/docs/getting-started/quickstart.md @@ -0,0 +1 @@ +# Getting Started diff --git a/docs/index.md b/docs/index.md index 6fcb70a..93c2428 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,8 +1,94 @@ -# fluidize-python +# Fluidize -[![Release](https://img.shields.io/github/v/release/Fluidize-Inc/fluidize-python)](https://img.shields.io/github/v/release/Fluidize-Inc/fluidize-python) -[![Build status](https://img.shields.io/github/actions/workflow/status/Fluidize-Inc/fluidize-python/main.yml?branch=main)](https://github.com/Fluidize-Inc/fluidize-python/actions/workflows/main.yml?query=branch%3Amain) -[![Commit activity](https://img.shields.io/github/commit-activity/m/Fluidize-Inc/fluidize-python)](https://img.shields.io/github/commit-activity/m/Fluidize-Inc/fluidize-python) -[![License](https://img.shields.io/github/license/Fluidize-Inc/fluidize-python)](https://img.shields.io/github/license/Fluidize-Inc/fluidize-python) +[![Python](https://img.shields.io/badge/python-3.9%2B-blue?style=for-the-badge&logo=python&logoColor=white)](https://python.org) +[![PyPI](https://img.shields.io/pypi/v/fluidize?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/fluidize/) +[![License](https://img.shields.io/github/license/Fluidize-Inc/fluidize-python?style=for-the-badge)](LICENSE) +[![Documentation](https://img.shields.io/badge/docs-available-brightgreen?style=for-the-badge&logo=gitbook&logoColor=white)](https://Fluidize-Inc.github.io/fluidize-python/) -Python package for automatic generation of scientific computing software pipelines. +## About + + **fluidize-python** is a library for building modular, reproducible scientific computing pipelines. It provides a unified interface to a wide range of physical simulation tools, eliminating the need to navigate the inconsistent, incomplete instructions that often vary from tool to tool. + +This library marks our first step toward AI-orchestrated scientific computing. By standardizing tools and practices within our framework, AI agents can automatically build, configure, and execute computational pipelines across domains and simulation platforms. + +Our goal is to improve today’s simulation tools so AI can assist researchers and scientists in accelerating the pace of innovation and scientific discovery. + +## Installation + +### Prerequesites: + +- Python 3.9+ +- Docker Desktop (for local execution). Download and install Docker Desktop from https://docs.docker.com/desktop/. + + After installation, verify with: + ```bash + docker --version + ``` + + + +### From PyPI +```bash +pip install fluidize +``` + +### From Source +```bash +git clone https://github.com/Fluidize-Inc/fluidize-python.git +cd fluidize-python +make install +``` + +## Run Examples + +Example projects are located in this folder: [examples/](https://github.com/Fluidize-Inc/fluidize-python/tree/main/examples). There you can find an [Jupyter Notebook](https://github.com/Fluidize-Inc/fluidize-python/blob/main/examples/demo.ipynb) of a simple simulation + +## Architecture + +At Fluidize, we believe strong organization leads to better reproducibility and scalability. + +We treat each simulation pipeline as an individual project. Within projects, each pipeline is treated as a DAG (directed acyclic graph), where nodes represent individual pieces of scientific software (e.g. inputs, solvers, visualization tools, etc.) and edges represent data flow between nodes. + + +### Nodes +Nodes are the foundational building blocks of simulation pipelines. Each node represents a computational unit with: + +| File | Purpose | +|------|---------| +| `properties.yaml` | Container configuration, working directory, and output paths | +| `metadata.yaml` | Node description, version, authors, and repository URL | +| `Dockerfile` | Environment setup and dependency installation | +| `parameters.json` | Tunable parameters for experiments | +| `main.sh` | Execution script for the source code | +| `source/` | Original scientific computing code | + +**Key Features:**
+- Predictable input/output paths
+- Modular and extensible design
+- No source code modification required
+- Automated node generation support (Public launch soon) + + +### Projects + +Projects store a simple data layer for managing individual modules within a pipeline. + +| File | Purpose | +|------|---------| +| `graph.json` | Node (scientific software) and edge (data flow) definitions | +| `metadata.yaml` | Project description and configuration | + + +### Runs + +Pipelines can be executed both locally and on the cloud. Local execution is handled by Docker engine. Cloud execution is routed through our API, and uses the Kubernetes engine with Argo Workflow Manager. + +## Contributing + +We would love to collaborate with you! Please see our [Contributing Guide](https://github.com/Fluidize-Inc/fluidize-python/blob/main/CONTRIBUTING.md) for details. + +Also - we would love to help streamline your pipeline! Please reach out to us at [founders@fluidize.ai](mailto:founders@fluidize.ai). + +## License + +This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. diff --git a/examples/example-projects/MUJOCO/Mujoco-Simulation/parameters.json b/examples/example-projects/MUJOCO/Mujoco-Simulation/parameters.json index 7940dbc..57b41a6 100644 --- a/examples/example-projects/MUJOCO/Mujoco-Simulation/parameters.json +++ b/examples/example-projects/MUJOCO/Mujoco-Simulation/parameters.json @@ -8,9 +8,6 @@ "name": "motor_strength", "latex": null, "location": [ - "source/pinata_simulation.py", - "source/pinata_simulation.py", - "source/pinata_simulation.py", "source/pinata_simulation.py" ], "options": null, diff --git a/fluidize/adapters/local/graph.py b/fluidize/adapters/local/graph.py index 0f5aedc..70ea95f 100644 --- a/fluidize/adapters/local/graph.py +++ b/fluidize/adapters/local/graph.py @@ -7,14 +7,11 @@ from typing import Optional -from fluidize.core.modules.graph.parameters import parse_parameters_from_json from fluidize.core.modules.graph.processor import GraphProcessor from fluidize.core.types.graph import GraphData, GraphEdge, GraphNode -from fluidize.core.types.node import nodeMetadata_simulation, nodeProperties_simulation +from fluidize.core.types.node import nodeMetadata_simulation, nodeParameters_simulation, nodeProperties_simulation from fluidize.core.types.parameters import Parameter from fluidize.core.types.project import ProjectSummary -from fluidize.core.utils.dataloader.data_loader import DataLoader -from fluidize.core.utils.dataloader.data_writer import DataWriter from fluidize.core.utils.pathfinder.path_finder import PathFinder @@ -171,9 +168,9 @@ def get_parameters(self, project: ProjectSummary, node_id: str) -> list[Paramete Returns: A list of Parameter objects for the node """ - parameters_path = PathFinder.get_node_parameters_path(project, node_id) - data = DataLoader.load_json(parameters_path) - return parse_parameters_from_json(data) + node_path = PathFinder.get_node_path(project, node_id) + parameters_model = nodeParameters_simulation.from_file(node_path) + return parameters_model.parameters def upsert_parameter(self, project: ProjectSummary, node_id: str, parameter: Parameter) -> Parameter: """ @@ -187,12 +184,11 @@ def upsert_parameter(self, project: ProjectSummary, node_id: str, parameter: Par Returns: The upserted parameter """ - parameters_path = PathFinder.get_node_parameters_path(project, node_id) - data = DataLoader.load_json(parameters_path) - params = parse_parameters_from_json(data) + node_path = PathFinder.get_node_path(project, node_id) + parameters_model = nodeParameters_simulation.from_file(node_path) # Check if parameter with same name exists - for p in params: + for p in parameters_model.parameters: if p.name == parameter.name: # Update the existing parameter with new values p.value = parameter.value @@ -211,13 +207,10 @@ def upsert_parameter(self, project: ProjectSummary, node_id: str, parameter: Par break else: # Parameter doesn't exist, add it - params.append(parameter) + parameters_model.parameters.append(parameter) - # Write updated parameters back - DataWriter.write_json( - filepath=parameters_path, - data={"parameters": [p.model_dump() for p in params]}, - ) + # Save updated parameters back + parameters_model.save() return parameter def set_parameters(self, project: ProjectSummary, node_id: str, parameters: list[Parameter]) -> list[Parameter]: @@ -232,10 +225,10 @@ def set_parameters(self, project: ProjectSummary, node_id: str, parameters: list Returns: The list of parameters that were set """ - parameters_path = PathFinder.get_node_parameters_path(project, node_id) - data = {"parameters": [p.model_dump() for p in parameters]} - - DataWriter.write_json(filepath=parameters_path, data=data) + node_path = PathFinder.get_node_path(project, node_id) + parameters_model = nodeParameters_simulation.from_file(node_path) + parameters_model.parameters = parameters + parameters_model.save() return parameters def show_parameters(self, project: ProjectSummary, node_id: str) -> str: diff --git a/fluidize/client.py b/fluidize/client.py index 940483c..3d81fd1 100644 --- a/fluidize/client.py +++ b/fluidize/client.py @@ -34,8 +34,6 @@ class FluidizeClient: def __init__(self, mode: Literal["local", "api", "auto"] = "auto", base_path: Optional[Path] = None): """ - Initialize the Fluidize client. - Args: mode: Operation mode - "local", "api", or "auto" for environment detection base_path: Optional custom base path for local mode. If None, uses ~/.fluidize @@ -55,14 +53,22 @@ def __init__(self, mode: Literal["local", "api", "auto"] = "auto", base_path: Op self.projects = RegistryManager(self._adapter) def _initialize_adapter(self) -> Any: - """Initialize the appropriate adapter based on the mode.""" + """Initialize the appropriate adapter based on the mode. + + Returns: + Any: The initialized adapter + """ if self.config.is_api_mode(): return self._initialize_api_adapter() else: return self._initialize_local_adapter() def _initialize_api_adapter(self) -> FluidizeSDK: - """Initialize the API adapter using FluidizeSDK.""" + """Initialize the API adapter using FluidizeSDK. + + Returns: + FluidizeSDK: The initialized API adapter + """ if not self.config.api_key: msg = "API mode requires an API key. Set the FLUIDIZE_API_KEY environment variable." raise ValueError(msg) @@ -72,18 +78,35 @@ def _initialize_api_adapter(self) -> FluidizeSDK: ) def _initialize_local_adapter(self) -> LocalAdapter: - """Initialize the local adapter.""" + """Initialize the local adapter. + + Returns: + LocalAdapter: The initialized local adapter + """ return LocalAdapter(self.config) @property def mode(self) -> str: - """Get the current operation mode.""" + """Get the current operation mode. + + Returns: + str: The current operation mode + """ return self.config.mode @property def adapter(self) -> Any: - """Access the underlying adapter for advanced operations.""" + """Access the underlying adapter for advanced operations. + + Returns: + Any: The underlying adapter + """ return self._adapter def __repr__(self) -> str: + """Return a string representation of the client. + + Returns: + str: A string representation of the client + """ return f"FluidizeClient(mode='{self.mode}')" diff --git a/fluidize/config.py b/fluidize/config.py index 68e2f43..680be97 100644 --- a/fluidize/config.py +++ b/fluidize/config.py @@ -91,7 +91,11 @@ def check_docker_available(self) -> bool: return result.returncode == 0 def warn_if_docker_unavailable(self) -> None: - """Issue a warning if Docker is not available for local runs.""" + """Issue a warning if Docker is not available for local runs. + + Returns: + None + """ if not self.check_docker_available(): warnings.warn( "Docker is not available. Local simulation runs will not be possible. " diff --git a/fluidize/core/modules/run/project/project_runner.py b/fluidize/core/modules/run/project/project_runner.py index b0164e1..7f9c76a 100644 --- a/fluidize/core/modules/run/project/project_runner.py +++ b/fluidize/core/modules/run/project/project_runner.py @@ -15,29 +15,50 @@ class ProjectRunner: """ def __init__(self, project: ProjectSummary): - """Initialize with project and get handler""" + """ + Args: + project: ProjectSummary + """ self.project = project self.handler = get_handler("project_runner", project) def prepare_run_environment(self, metadata: RunFlowPayload) -> int: """ - Create a new run folder for the project - Returns the run number + Create a new run folder for the project. + + Args: + metadata: RunFlowPayload + + Returns: + int: Run number """ return cast(int, self.handler.prepare_run_environment(metadata)) async def execute_node(self, node_id: str, prev_node_id: Optional[str] = None, **kwargs: Any) -> dict[str, Any]: """ - Execute a single node within the project run - Returns the execution result + Execute a single node within the project run. + + Args: + node_id: Node ID + prev_node_id: Previous node ID + **kwargs: Additional keyword arguments + + Returns: + dict[str, Any]: Execution result """ return await asyncio.to_thread(self.handler.execute_node, node_id, prev_node_id=prev_node_id, **kwargs) async def execute_flow(self, nodes_to_run: list[str], prev_nodes: list[str], **kwargs: Any) -> list[dict[str, Any]]: """ - Execute a flow of nodes in the correct order - nodes_to_run: List of node IDs - Returns execution results for all nodes + Execute a flow of nodes in order. + + Args: + nodes_to_run: List of node IDs + prev_nodes: List of previous node IDs + **kwargs: Additional keyword arguments + + Returns: + list[dict[str, Any]]: Execution results for all nodes """ # Make sure that nodes_to_run and prev_nodes are same size lists if len(nodes_to_run) != len(prev_nodes): diff --git a/fluidize/core/types/file_models/json_file_model_base.py b/fluidize/core/types/file_models/json_file_model_base.py new file mode 100644 index 0000000..b0e8ff2 --- /dev/null +++ b/fluidize/core/types/file_models/json_file_model_base.py @@ -0,0 +1,106 @@ +from __future__ import annotations + +from typing import Any, TypeVar, Union + +from pydantic import BaseModel, ConfigDict, PrivateAttr, ValidationError +from upath import UPath + +T = TypeVar("T", bound="JSONFileModelBase") + + +class JSONFileModelBase(BaseModel): + _filepath: Union[UPath, None] = PrivateAttr(default=None) + + @property + def filepath(self) -> UPath: + """Return the exact path to the model file. Raises if not set.""" + if not self._filepath: + raise ValueError() + return self._filepath + + @property + def directory(self) -> UPath: + """Return the folder containing the model file. Raises if filepath not set.""" + fp = self.filepath + return fp.parent + + @classmethod + def from_file(cls: type[T], directory: Union[str, UPath]) -> T: + from fluidize.core.utils.dataloader.data_loader import DataLoader + + filename = getattr(cls, "_filename", None) + if not filename: + raise TypeError() + + path = UPath(directory) / filename + data = DataLoader.load_json(path) + + if not data: + raise FileNotFoundError() + + try: + instance = cls.model_validate(data) + except ValidationError: + raise + except Exception as e: + raise ValueError() from e + else: + instance._filepath = path + return instance + + @classmethod + def from_dict_and_path(cls: type[T], data: dict, path: UPath) -> T: + """Creates a model instance from a dictionary and a path, without reading the file again.""" + if not data: + raise ValueError() + + try: + instance = cls.model_validate(data) + except ValidationError: + raise + except Exception as e: + raise ValueError() from e + else: + instance._filepath = path + return instance + + def model_dump_wrapped(self) -> dict[str, Any]: + config = getattr(self, "Key", None) + key = getattr(config, "key", None) + + if not key: + return self.model_dump() + + return {key: self.model_dump(mode="json")} + + def save(self, directory: UPath | None = None) -> None: + from fluidize.core.utils.dataloader.data_loader import DataLoader + from fluidize.core.utils.dataloader.data_writer import DataWriter + + if directory: + filename = getattr(self.__class__, "_filename", None) + if not filename: + raise TypeError() + self._filepath = UPath(directory) / filename + + if not self._filepath: + raise ValueError() + + # Load existing data to preserve other keys, if the file already exists. + # Pass a new UPath object to avoid issues with object caching if it's the same file. + existing_data = DataLoader.load_json(UPath(self._filepath)) + + new_data = self.model_dump_wrapped() + existing_data.update(new_data) + + DataWriter.write_json(self._filepath, existing_data) + + def edit(self, **kwargs: Any) -> None: + for key, value in kwargs.items(): + if hasattr(self, key): + setattr(self, key, value) + else: + raise AttributeError() + self.save() + + model_config = ConfigDict(arbitrary_types_allowed=True) diff --git a/fluidize/core/types/file_models/parameters_model.py b/fluidize/core/types/file_models/parameters_model.py new file mode 100644 index 0000000..2c2abd4 --- /dev/null +++ b/fluidize/core/types/file_models/parameters_model.py @@ -0,0 +1,56 @@ +from typing import Any, ClassVar + +from pydantic import Field, model_validator + +from fluidize.core.constants import FileConstants +from fluidize.core.types.parameters import Parameter + +from .json_file_model_base import JSONFileModelBase + + +class ParametersModel(JSONFileModelBase): + _filename: ClassVar[str] = FileConstants.PARAMETERS_SUFFIX + """ + A base model for parameters objects stored in JSON structure. + + This model provides two main functionalities: + 1. A validator to automatically unpack nested data based on a 'key' + from the subclass's Config. + 2. A method to wrap the model's data back into the nested structure + for serialization. + """ + + parameters: list[Parameter] = Field(default_factory=list) + + @model_validator(mode="before") + @classmethod + def _unpack_and_validate(cls, data: Any) -> Any: + """ + Unpacks and validates the data against the key + specified in the subclass's Config. + """ + if not isinstance(data, dict): + return data + + config = getattr(cls, "Key", None) + key = getattr(config, "key", None) + + # If there's no key in the config or the key is not in the data, + # assume the data is already in the correct, unpacked structure. + if not key or key not in data: + return data + + unpacked_data = data[key] + if not isinstance(unpacked_data, list): + # If parameters is not a list, treat it as empty + unpacked_data = [] + + # Return data in the format expected by the model + return {"parameters": unpacked_data} + + def model_dump_wrapped(self) -> dict[str, Any]: + """Override to avoid double wrapping of parameters key.""" + return {"parameters": [p.model_dump() for p in self.parameters]} + + class Key: + key = "parameters" diff --git a/fluidize/core/types/graph.py b/fluidize/core/types/graph.py index 46b337c..97fdb20 100644 --- a/fluidize/core/types/graph.py +++ b/fluidize/core/types/graph.py @@ -10,31 +10,60 @@ class Position(BaseModel): - x: float - y: float + """Position of a node in layout space.""" + + x: float #: X coordinate in layout space. + y: float #: Y coordinate in layout space. class graphNodeData(BaseModel): - label: str - simulation_id: Optional[str] = None + """Extra metadata for a node.""" + + label: str #: Node label. + simulation_id: Optional[str] = None #: Simulation ID. -# Default Node Type in Graph +# Default Node Type in GraphGraph class GraphNode(BaseModel): - id: str - position: Position - data: graphNodeData - type: str + """A node in the graph. + + Attributes: + id: Unique node ID. + position: Node position. + data: Extra metadata. + type: Renderer/type key. + """ + + id: str #: Node ID. + position: Position #: Node position. + data: graphNodeData #: Node data. + type: str #: Node type. # Edge Type in Graph class GraphEdge(BaseModel): - id: str - source: str - target: str - type: str + """An edge in the graph. + + Attributes: + id: Unique edge ID. + source: Source node ID. + target: Target node ID. + type: Renderer/type key. + """ + + id: str #: Edge ID. + source: str #: Source node ID. + target: str #: Target node ID. + type: str #: Edge type. class GraphData(BaseModel): - nodes: list[GraphNode] - edges: list[GraphEdge] + """A graph representation of a project in the `graph.json` file. + + Attributes: + nodes: List of nodes. + edges: List of edges. + """ + + nodes: list[GraphNode] #: List of nodes. + edges: list[GraphEdge] #: List of edges. diff --git a/fluidize/core/types/node.py b/fluidize/core/types/node.py index 9e49ce4..15a2d23 100644 --- a/fluidize/core/types/node.py +++ b/fluidize/core/types/node.py @@ -11,6 +11,7 @@ from pydantic import BaseModel, ConfigDict, computed_field from .file_models.metadata_model import MetadataModel +from .file_models.parameters_model import ParametersModel from .file_models.properties_model import PropertiesModel from .runs import RunStatus @@ -95,3 +96,15 @@ class nodeMetadata_simulation(MetadataModel): class Key: key = "simulation" metadata_version = "1.0" + + +class nodeParameters_simulation(ParametersModel): + """ + Parameters configuration for a simulation node. + + Handles loading and saving of parameters.json files with the structure: + {"parameters": [list of Parameter objects]} + """ + + class Key: + key = "parameters" diff --git a/fluidize/managers/graph.py b/fluidize/managers/graph.py index 1c4292d..3842d94 100644 --- a/fluidize/managers/graph.py +++ b/fluidize/managers/graph.py @@ -2,9 +2,12 @@ Project-scoped graph manager for user-friendly graph operations. """ -from typing import Any, Optional +from typing import TYPE_CHECKING, Any, Optional from fluidize.core.types.graph import GraphData, GraphEdge, GraphNode + +if TYPE_CHECKING: + from .node import NodeManager from fluidize.core.types.node import nodeMetadata_simulation, nodeProperties_simulation from fluidize.core.types.parameters import Parameter from fluidize.core.types.project import ProjectSummary @@ -20,10 +23,8 @@ class GraphManager: def __init__(self, adapter: Any, project: ProjectSummary) -> None: """ - Initialize project-scoped graph manager. - Args: - adapter: adapter adapter (FluidizeSDK or Localadapter) + adapter: adapter (FluidizeSDK or LocalAdapter) project: The project this graph manager is bound to """ self.adapter = adapter @@ -42,7 +43,21 @@ def get(self) -> GraphData: """ return self.adapter.graph.get_graph(self.project) # type: ignore[no-any-return] - def add_node(self, node: GraphNode, sim_global: bool = True) -> GraphNode: + def get_node(self, node_id: str) -> "NodeManager": + """ + Get a NodeManager for a specific node in the project. + + Args: + node_id: ID of the node to get a manager for + + Returns: + NodeManager instance for the specified node + """ + from .node import NodeManager + + return NodeManager(self.adapter, self.project, node_id) + + def add_node(self, node: GraphNode, sim_global: bool = True) -> "NodeManager": """ Add a new node to this project's graph. @@ -51,9 +66,10 @@ def add_node(self, node: GraphNode, sim_global: bool = True) -> GraphNode: sim_global: Whether to use global simulations (placeholder for future) Returns: - The inserted node + The added node """ - return self.adapter.graph.insert_node(self.project, node, sim_global) # type: ignore[no-any-return] + inserted_node = self.adapter.graph.insert_node(self.project, node, sim_global) + return self.get_node(inserted_node.id) def add_node_from_scratch( self, @@ -61,7 +77,7 @@ def add_node_from_scratch( node_properties: nodeProperties_simulation, node_metadata: nodeMetadata_simulation, repo_link: Optional[str] = None, - ) -> GraphNode: + ) -> "NodeManager": """ Add a new node to this project's graph from scratch, creating all necessary files and directories. @@ -72,11 +88,12 @@ def add_node_from_scratch( repo_link: Optional repository URL to clone into the source directory Returns: - The inserted node + The added node """ - return self.adapter.graph.insert_node_from_scratch( # type: ignore[no-any-return] + inserted_node = self.adapter.graph.insert_node_from_scratch( self.project, node, node_properties, node_metadata, repo_link ) + return self.get_node(inserted_node.id) def update_node_position(self, node: GraphNode) -> GraphNode: """ diff --git a/fluidize/managers/node.py b/fluidize/managers/node.py new file mode 100644 index 0000000..cef596e --- /dev/null +++ b/fluidize/managers/node.py @@ -0,0 +1,446 @@ +""" +Node-scoped manager for user-friendly node operations. +""" + +from typing import Any, Optional + +from upath import UPath + +from fluidize.core.constants import FileConstants +from fluidize.core.types.graph import GraphNode +from fluidize.core.types.node import nodeMetadata_simulation, nodeParameters_simulation, nodeProperties_simulation +from fluidize.core.types.parameters import Parameter +from fluidize.core.types.project import ProjectSummary +from fluidize.core.utils.pathfinder.path_finder import PathFinder + + +class NodeManager: + """ + Node manager for a specific node within a project. + + Provides node-specific operations like editing parameters, metadata, + and properties without requiring project and node context on each method call. + """ + + def __init__(self, adapter: Any, project: ProjectSummary, node_id: str) -> None: + """ + Initialize node-scoped manager. + + Args: + adapter: adapter adapter (FluidizeSDK or Localadapter) + project: The project this node belongs to + node_id: The ID of the node this manager is bound to + """ + self.adapter = adapter + self.project = project + self.node_id = node_id + + @property + def id(self) -> str: + """ + Get the node ID. + + Returns: + The ID of the node this manager is bound to + """ + return self.node_id + + @property + def data(self) -> Any: + """ + Get the node's data. + + Returns: + The data of the graph node + """ + return self.get_node().data + + def get_node(self) -> GraphNode: + """ + Get the complete graph node data. + + Returns: + GraphNode containing the node data + + Raises: + ValueError: If the node is not found in the project graph + """ + graph = self.adapter.graph.get_graph(self.project) + for node in graph.nodes: + if node.id == self.node_id: + return node # type: ignore[no-any-return] + msg = f"Node with ID '{self.node_id}' not found in project '{self.project.id}'" + raise ValueError(msg) + + def exists(self) -> bool: + """ + Check if this node exists in the project graph. + + Returns: + True if the node exists, False otherwise + """ + try: + self.get_node() + except ValueError: + return False + else: + return True + + def delete(self) -> None: + """ + Delete this node from the project graph and filesystem. + """ + self.adapter.graph.delete_node(self.project, self.node_id) + + def update_position(self, x: float, y: float) -> GraphNode: + """ + Update the node's position in the graph. + + Args: + x: New x coordinate + y: New y coordinate + + Returns: + The updated graph node + """ + node = self.get_node() + node.position.x = x + node.position.y = y + return self.adapter.graph.update_node_position(self.project, node) # type: ignore[no-any-return] + + def get_metadata(self) -> nodeMetadata_simulation: + """ + Get the node's metadata from metadata.yaml. + + Returns: + The node's metadata + + Raises: + FileNotFoundError: If metadata file doesn't exist + ValueError: If metadata file is invalid + """ + node_path = PathFinder.get_node_path(self.project, self.node_id) + return nodeMetadata_simulation.from_file(node_path) + + def update_metadata(self, **kwargs: Any) -> nodeMetadata_simulation: + """ + Update specific fields in the node's metadata. + + Args: + **kwargs: Fields to update (e.g., name="New Name", description="New desc") + + Returns: + The updated metadata + + Raises: + AttributeError: If trying to update a field that doesn't exist + """ + metadata = self.get_metadata() + metadata.edit(**kwargs) + return metadata + + def save_metadata(self, metadata: nodeMetadata_simulation) -> None: + """ + Save metadata object to the node's metadata.yaml file. + + Args: + metadata: The metadata object to save + """ + node_path = PathFinder.get_node_path(self.project, self.node_id) + metadata.save(node_path) + + def get_properties(self) -> nodeProperties_simulation: + """ + Get the node's properties from properties.yaml. + + Returns: + The node's properties + + Raises: + FileNotFoundError: If properties file doesn't exist + ValueError: If properties file is invalid + """ + node_path = PathFinder.get_node_path(self.project, self.node_id) + return nodeProperties_simulation.from_file(node_path) + + def update_properties(self, **kwargs: Any) -> nodeProperties_simulation: + """ + Update specific fields in the node's properties. + + Args: + **kwargs: Fields to update (e.g., container_image="new:tag", should_run=False) + + Returns: + The updated properties + + Raises: + AttributeError: If trying to update a field that doesn't exist + """ + properties = self.get_properties() + properties.edit(**kwargs) + return properties + + def save_properties(self, properties: nodeProperties_simulation) -> None: + """ + Save properties object to the node's properties.yaml file. + + Args: + properties: The properties object to save + """ + node_path = PathFinder.get_node_path(self.project, self.node_id) + properties.save(node_path) + + def get_parameters_model(self) -> nodeParameters_simulation: + """ + Get the node's parameters model from parameters.json. + + Returns: + The node's parameters model + + Raises: + FileNotFoundError: If parameters file doesn't exist + ValueError: If parameters file is invalid + """ + node_path = PathFinder.get_node_path(self.project, self.node_id) + return nodeParameters_simulation.from_file(node_path) + + def get_parameters(self) -> list[Parameter]: + """ + Get the node's parameters list from parameters.json. + + Returns: + List of Parameter objects for the node + """ + return self.get_parameters_model().parameters + + def get_parameter(self, name: str) -> Optional[Parameter]: + """ + Get a specific parameter by name. + + Args: + name: Name of the parameter to retrieve + + Returns: + The parameter if found, None otherwise + """ + parameters = self.get_parameters() + for param in parameters: + if param.name == name: + return param + return None + + def update_parameter(self, parameter: Parameter) -> Parameter: + """ + Update or add a parameter. + + Args: + parameter: The parameter to update/add + + Returns: + The updated parameter + """ + parameters_model = self.get_parameters_model() + + # Check if parameter with same name exists + for p in parameters_model.parameters: + if p.name == parameter.name: + # Update existing parameter + p.value = parameter.value + p.description = parameter.description + p.type = parameter.type + p.label = parameter.label + p.latex = parameter.latex + p.options = parameter.options + p.scope = parameter.scope + # Handle location extension + if parameter.location: + if p.location: + p.location.extend(parameter.location) + else: + p.location = parameter.location + break + else: + # Parameter doesn't exist, add it + parameters_model.parameters.append(parameter) + + parameters_model.save() + return parameter + + def set_parameters(self, parameters: list[Parameter]) -> list[Parameter]: + """ + Replace all parameters with the provided list. + + Args: + parameters: List of parameters to set + W + Returns: + The list of parameters that were set + """ + parameters_model = self.get_parameters_model() + parameters_model.parameters = parameters + parameters_model.save() + return parameters + + def remove_parameter(self, name: str) -> bool: + """ + Remove a parameter by name. + + Args: + name: Name of the parameter to remove + + Returns: + True if parameter was removed, False if it didn't exist + """ + parameters_model = self.get_parameters_model() + original_count = len(parameters_model.parameters) + parameters_model.parameters = [p for p in parameters_model.parameters if p.name != name] + + if len(parameters_model.parameters) < original_count: + parameters_model.save() + return True + return False + + def show_parameters(self) -> str: + """ + Get a formatted string display of all parameters. + + Returns: + A formatted string displaying the parameters + """ + parameters = self.get_parameters() + + if not parameters: + return f"No parameters found for node '{self.node_id}'" + + output = f"Parameters for node '{self.node_id}':\n\n" + + for i, param in enumerate(parameters, 1): + output += f"Parameter {i}:\n" + output += f" Name: {param.name}\n" + output += f" Value: {param.value}\n" + output += f" Description: {param.description}\n" + output += f" Type: {param.type}\n" + output += f" Label: {param.label}\n" + if param.latex: + output += f" LaTeX: {param.latex}\n" + if param.location: + output += f" Location: {param.location}\n" + if param.options: + output += f" Options: {[opt.label for opt in param.options]}\n" + if param.scope: + output += f" Scope: {param.scope}\n" + output += "\n" + + return output + + def get_node_directory(self) -> UPath: + """ + Get the filesystem path to this node's directory. + + Returns: + Path to the node's directory + """ + return PathFinder.get_node_path(self.project, self.node_id) + + def get_metadata_path(self) -> UPath: + """ + Get the filesystem path to this node's metadata.yaml file. + + Returns: + Path to the metadata file + """ + return self.get_node_directory() / FileConstants.METADATA_SUFFIX + + def get_properties_path(self) -> UPath: + """ + Get the filesystem path to this node's properties.yaml file. + + Returns: + Path to the properties file + """ + return PathFinder.get_properties_path(self.project, self.node_id) + + def get_parameters_path(self) -> UPath: + """ + Get the filesystem path to this node's parameters.json file. + + Returns: + Path to the parameters file + """ + return PathFinder.get_node_parameters_path(self.project, self.node_id) + + def validate(self) -> dict[str, Any]: + """ + Validate the node's files and structure. + + Returns: + Dictionary containing validation results with keys: + - 'valid': bool indicating if node is valid + - 'graph_node_exists': bool + - 'metadata_exists': bool + - 'properties_exists': bool + - 'parameters_exists': bool + - 'errors': list of error messages + """ + result: dict[str, Any] = { + "valid": True, + "graph_node_exists": False, + "metadata_exists": False, + "properties_exists": False, + "parameters_exists": False, + "errors": [], + } + + # Check if node exists in graph + try: + self.get_node() + result["graph_node_exists"] = True + except ValueError as e: + result["errors"].append(str(e)) + + # Check metadata file + try: + self.get_metadata() + result["metadata_exists"] = True + except Exception as e: + result["errors"].append(f"Metadata error: {e}") + + # Check properties file + try: + self.get_properties() + result["properties_exists"] = True + except Exception as e: + result["errors"].append(f"Properties error: {e}") + + # Check parameters file + try: + self.get_parameters() + result["parameters_exists"] = True + except Exception as e: + result["errors"].append(f"Parameters error: {e}") + + result["valid"] = len(result["errors"]) == 0 + return result + + def to_dict(self) -> dict[str, Any]: + """ + Convert the complete node information to a dictionary. + + Returns: + Dictionary containing node graph data, metadata, properties, and parameters + """ + try: + return { + "graph_node": self.get_node().model_dump(), + "metadata": self.get_metadata().model_dump(), + "properties": self.get_properties().model_dump(), + "parameters": [p.model_dump() for p in self.get_parameters()], + "paths": { + "node_directory": str(self.get_node_directory()), + "metadata_file": str(self.get_metadata_path()), + "properties_file": str(self.get_properties_path()), + "parameters_file": str(self.get_parameters_path()), + }, + } + except Exception as e: + return {"error": str(e), "node_id": self.node_id, "project": self.project.id} diff --git a/fluidize/managers/project.py b/fluidize/managers/project.py index 4b90996..ea27bc3 100644 --- a/fluidize/managers/project.py +++ b/fluidize/managers/project.py @@ -12,17 +12,15 @@ class ProjectManager: """ - Project entity that wraps project data and provides access to scoped managers. + Project manager that wraps project data and provides access to scoped managers. Provides convenient access to graph and runs operations for this specific project. """ def __init__(self, adapter: Any, project_summary: ProjectSummary) -> None: """ - Initialize project wrapper. - Args: - adapter: adapter adapter (FluidizeSDK or Localadapter) + adapter: adapter (FluidizeSDK or LocalAdapter) project_summary: The underlying project data """ self._adapter = adapter @@ -57,42 +55,74 @@ def runs(self) -> RunsManager: # Delegate all ProjectSummary attributes @property def id(self) -> str: - """Get project ID.""" + """Get project ID. + + Returns: + The project ID + """ return self._project_summary.id @property def label(self) -> Optional[str]: - """Get project label.""" + """Get project label. + + Returns: + The project label + """ return self._project_summary.label @property def description(self) -> Optional[str]: - """Get project description.""" + """Get project description. + + Returns: + The project description + """ return self._project_summary.description @property def location(self) -> Optional[str]: - """Get project location.""" + """Get project location. + + Returns: + The project location + """ return self._project_summary.location @property def status(self) -> Optional[str]: - """Get project status.""" + """Get project status. + + Returns: + The project status + """ return self._project_summary.status @property def metadata_version(self) -> str: - """Get project metadata version.""" + """Get project metadata version. + + Returns: + The project metadata version + """ return self._project_summary.metadata_version @property def created_at(self) -> Optional[str]: - """Get project creation timestamp.""" + """Get project creation timestamp. + + Returns: + The project creation timestamp + """ return getattr(self._project_summary, "created_at", None) @property def updated_at(self) -> Optional[str]: - """Get project update timestamp.""" + """Get project update timestamp. + + Returns: + The project update timestamp + """ return getattr(self._project_summary, "updated_at", None) def to_dict(self) -> dict[str, Any]: @@ -113,8 +143,5 @@ def to_dict(self) -> dict[str, Any]: "updated_at": self.updated_at, } - def __repr__(self) -> str: - return f"Project(id='{self.id}', label='{self.label}')" - def __str__(self) -> str: return f"Project {self.id}: {self.label or 'No label'}" diff --git a/fluidize/managers/registry.py b/fluidize/managers/registry.py index 9946de2..c4b6094 100644 --- a/fluidize/managers/registry.py +++ b/fluidize/managers/registry.py @@ -5,7 +5,7 @@ class RegistryManager: """ - Manager for project CRUD operations. + Registry manager for project CRUD operations. Provides methods to create, retrieve, update, and list projects. All methods return Project entities that give access to project-scoped operations. @@ -13,10 +13,8 @@ class RegistryManager: def __init__(self, adapter: Any) -> None: """ - Initialize the Projects manager. - Args: - adapter: adapter adapter (FluidizeSDK or Localadapter) + adapter: adapter (FluidizeSDK or LocalAdapter) """ self.adapter = adapter diff --git a/fluidize/managers/runs.py b/fluidize/managers/runs.py index 47cbe4a..4427f51 100644 --- a/fluidize/managers/runs.py +++ b/fluidize/managers/runs.py @@ -20,10 +20,8 @@ class RunsManager: def __init__(self, adapter: Any, project: ProjectSummary) -> None: """ - Initialize project-scoped runs manager. - Args: - adapter: adapter adapter (FluidizeSDK or Localadapter) + adapter: adapter (FluidizeSDK or LocalAdapter) project: The project this runs manager is bound to """ self.adapter = adapter diff --git a/mkdocs.yml b/mkdocs.yml index dc8bb80..ba159d6 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -10,14 +10,13 @@ copyright: Maintained by Fluidize-Inc. nav: - Home: index.md - Getting Started: - - Fluidize Client: getting-started/client.md - - Projects and Nodes: getting-started/projects-nodes.md + - Quickstart: getting-started/quickstart.md - Examples: getting-started/examples.md - Core Modules: + - Client: core-modules/client.md - Projects: core-modules/projects.md - Graph: core-modules/graph.md - Run: core-modules/run.md - - Execute: core-modules/execute.md plugins: - search - mkdocstrings: @@ -26,17 +25,22 @@ plugins: paths: ["fluidize"] options: merge_init_into_class: true - show_signature: false + show_signature: true + show_signature_annotations: true + members_order: source docstring_style: google filters: - "!^_" show_source: false - show_root_heading: true show_root_full_path: false + extra: + show_attributes: true + show_root_heading: true theme: name: material - feature: - tabs: true + features: + - tabs + - navigation.indexes palette: - media: "(prefers-color-scheme: light)" scheme: default diff --git a/tests/unit/backends/local/test_graph.py b/tests/unit/backends/local/test_graph.py index dfb197b..a61bbab 100644 --- a/tests/unit/backends/local/test_graph.py +++ b/tests/unit/backends/local/test_graph.py @@ -310,31 +310,33 @@ def test_individual_operations(self, sample_project, operation, method_name, arg else: processor_method.assert_called_once() - @patch("fluidize.adapters.local.graph.DataLoader") + @patch("fluidize.adapters.local.graph.nodeParameters_simulation") @patch("fluidize.adapters.local.graph.PathFinder") - def test_get_parameters_success(self, mock_pathfinder, mock_dataloader, sample_project): + def test_get_parameters_success(self, mock_pathfinder, mock_node_params, sample_project): """Test successful parameter retrieval.""" # Mock setup mock_parameters_path = Mock() - mock_pathfinder.get_node_parameters_path.return_value = mock_parameters_path - mock_dataloader.load_json.return_value = { - "parameters": [ - { - "name": "test_param", - "value": "test_value", - "type": "text", - "label": "Test Parameter", - "description": "A test parameter", - } - ] - } + mock_pathfinder.get_node_path.return_value = mock_parameters_path + + # Mock the parameters model instance + mock_params_instance = Mock() + mock_params_instance.parameters = [ + Parameter( + name="test_param", + value="test_value", + type="text", + label="Test Parameter", + description="A test parameter", + ) + ] + mock_node_params.from_file.return_value = mock_params_instance handler = GraphHandler() result = handler.get_parameters(sample_project, "test-node-id") # Verify calls - mock_pathfinder.get_node_parameters_path.assert_called_once_with(sample_project, "test-node-id") - mock_dataloader.load_json.assert_called_once_with(mock_parameters_path) + mock_pathfinder.get_node_path.assert_called_once_with(sample_project, "test-node-id") + mock_node_params.from_file.assert_called_once_with(mock_parameters_path) # Verify result assert len(result) == 1 @@ -342,16 +344,19 @@ def test_get_parameters_success(self, mock_pathfinder, mock_dataloader, sample_p assert result[0].name == "test_param" assert result[0].value == "test_value" - @patch("fluidize.adapters.local.graph.DataWriter") - @patch("fluidize.adapters.local.graph.DataLoader") + @patch("fluidize.adapters.local.graph.nodeParameters_simulation") @patch("fluidize.adapters.local.graph.PathFinder") - def test_upsert_parameter_new_parameter(self, mock_pathfinder, mock_dataloader, mock_datawriter, sample_project): + def test_upsert_parameter_new_parameter(self, mock_pathfinder, mock_node_params, sample_project): """Test upserting a new parameter.""" # Mock setup mock_parameters_path = Mock() - mock_pathfinder.get_node_parameters_path.return_value = mock_parameters_path - mock_dataloader.load_json.return_value = {"parameters": []} - mock_datawriter.write_json.return_value = True + mock_pathfinder.get_node_path.return_value = mock_parameters_path + + # Mock the parameters model instance with empty parameters + mock_params_instance = Mock() + mock_params_instance.parameters = [] + mock_node_params.from_file.return_value = mock_params_instance + mock_params_instance.save.return_value = None new_parameter = Parameter( name="new_param", value="new_value", type="text", label="New Parameter", description="A new parameter" @@ -361,41 +366,38 @@ def test_upsert_parameter_new_parameter(self, mock_pathfinder, mock_dataloader, result = handler.upsert_parameter(sample_project, "test-node-id", new_parameter) # Verify calls - mock_pathfinder.get_node_parameters_path.assert_called_once_with(sample_project, "test-node-id") - mock_dataloader.load_json.assert_called_once_with(mock_parameters_path) - mock_datawriter.write_json.assert_called_once() + mock_pathfinder.get_node_path.assert_called_once_with(sample_project, "test-node-id") + mock_node_params.from_file.assert_called_once_with(mock_parameters_path) + mock_params_instance.save.assert_called_once() - # Verify the written data contains the new parameter - written_data = mock_datawriter.write_json.call_args[1]["data"] - assert len(written_data["parameters"]) == 1 - assert written_data["parameters"][0]["name"] == "new_param" + # Verify the parameter was added to the instance + assert len(mock_params_instance.parameters) == 1 + assert mock_params_instance.parameters[0].name == "new_param" # Verify result assert result == new_parameter - @patch("fluidize.adapters.local.graph.DataWriter") - @patch("fluidize.adapters.local.graph.DataLoader") + @patch("fluidize.adapters.local.graph.nodeParameters_simulation") @patch("fluidize.adapters.local.graph.PathFinder") - def test_upsert_parameter_existing_parameter( - self, mock_pathfinder, mock_dataloader, mock_datawriter, sample_project - ): + def test_upsert_parameter_existing_parameter(self, mock_pathfinder, mock_node_params, sample_project): """Test upserting an existing parameter extends locations.""" # Mock setup with existing parameter mock_parameters_path = Mock() - mock_pathfinder.get_node_parameters_path.return_value = mock_parameters_path - mock_dataloader.load_json.return_value = { - "parameters": [ - { - "name": "existing_param", - "value": "existing_value", - "type": "text", - "label": "Existing Parameter", - "description": "An existing parameter", - "location": ["file1.py"], - } - ] - } - mock_datawriter.write_json.return_value = True + mock_pathfinder.get_node_path.return_value = mock_parameters_path + + # Mock the parameters model instance with existing parameter + existing_param = Parameter( + name="existing_param", + value="existing_value", + type="text", + label="Existing Parameter", + description="An existing parameter", + location=["file1.py"], + ) + mock_params_instance = Mock() + mock_params_instance.parameters = [existing_param] + mock_node_params.from_file.return_value = mock_params_instance + mock_params_instance.save.return_value = None update_parameter = Parameter( name="existing_param", @@ -409,24 +411,32 @@ def test_upsert_parameter_existing_parameter( handler = GraphHandler() result = handler.upsert_parameter(sample_project, "test-node-id", update_parameter) - # Verify the written data extends the location - written_data = mock_datawriter.write_json.call_args[1]["data"] - assert len(written_data["parameters"]) == 1 - param_data = written_data["parameters"][0] - assert param_data["name"] == "existing_param" - assert param_data["location"] == ["file1.py", "file2.py"] + # Verify calls + mock_pathfinder.get_node_path.assert_called_once_with(sample_project, "test-node-id") + mock_node_params.from_file.assert_called_once_with(mock_parameters_path) + mock_params_instance.save.assert_called_once() + + # Verify the parameter location was extended + updated_param = mock_params_instance.parameters[0] + assert updated_param.name == "existing_param" + assert updated_param.location == ["file1.py", "file2.py"] # Verify result assert result == update_parameter - @patch("fluidize.adapters.local.graph.DataWriter") + @patch("fluidize.adapters.local.graph.nodeParameters_simulation") @patch("fluidize.adapters.local.graph.PathFinder") - def test_set_parameters_success(self, mock_pathfinder, mock_datawriter, sample_project): + def test_set_parameters_success(self, mock_pathfinder, mock_node_params, sample_project): """Test setting parameters replaces all existing parameters.""" # Mock setup mock_parameters_path = Mock() - mock_pathfinder.get_node_parameters_path.return_value = mock_parameters_path - mock_datawriter.write_json.return_value = True + mock_pathfinder.get_node_path.return_value = mock_parameters_path + + # Mock the parameters model instance + mock_params_instance = Mock() + mock_params_instance.parameters = [] + mock_node_params.from_file.return_value = mock_params_instance + mock_params_instance.save.return_value = None parameters = [ Parameter(name="param1", value="value1", type="text", label="Parameter 1", description="First parameter"), @@ -439,42 +449,49 @@ def test_set_parameters_success(self, mock_pathfinder, mock_datawriter, sample_p result = handler.set_parameters(sample_project, "test-node-id", parameters) # Verify calls - mock_pathfinder.get_node_parameters_path.assert_called_once_with(sample_project, "test-node-id") - mock_datawriter.write_json.assert_called_once() + mock_pathfinder.get_node_path.assert_called_once_with(sample_project, "test-node-id") + mock_node_params.from_file.assert_called_once_with(mock_parameters_path) + mock_params_instance.save.assert_called_once() - # Verify the written data - written_data = mock_datawriter.write_json.call_args[1]["data"] - assert len(written_data["parameters"]) == 2 - assert written_data["parameters"][0]["name"] == "param1" - assert written_data["parameters"][1]["name"] == "param2" + # Verify the parameters were set correctly + assert mock_params_instance.parameters == parameters + assert len(mock_params_instance.parameters) == 2 + assert mock_params_instance.parameters[0].name == "param1" + assert mock_params_instance.parameters[1].name == "param2" # Verify result assert result == parameters - @patch("fluidize.adapters.local.graph.DataLoader") + @patch("fluidize.adapters.local.graph.nodeParameters_simulation") @patch("fluidize.adapters.local.graph.PathFinder") - def test_show_parameters_success(self, mock_pathfinder, mock_dataloader, sample_project): + def test_show_parameters_success(self, mock_pathfinder, mock_node_params, sample_project): """Test showing parameters in nice format.""" # Mock setup mock_parameters_path = Mock() - mock_pathfinder.get_node_parameters_path.return_value = mock_parameters_path - mock_dataloader.load_json.return_value = { - "parameters": [ - { - "name": "motor_strength", - "value": "20.0", - "type": "text", - "label": "Motor Strength", - "description": "Control signal strength for bat motor", - "scope": "simulation", - "location": ["source/pinata_simulation.py"], - } - ] - } + mock_pathfinder.get_node_path.return_value = mock_parameters_path + + # Mock the parameters model instance with a parameter + mock_params_instance = Mock() + mock_params_instance.parameters = [ + Parameter( + name="motor_strength", + value="20.0", + type="text", + label="Motor Strength", + description="Control signal strength for bat motor", + scope="simulation", + location=["source/pinata_simulation.py"], + ) + ] + mock_node_params.from_file.return_value = mock_params_instance handler = GraphHandler() result = handler.show_parameters(sample_project, "test-node-id") + # Verify calls + mock_pathfinder.get_node_path.assert_called_once_with(sample_project, "test-node-id") + mock_node_params.from_file.assert_called_once_with(mock_parameters_path) + # Verify the formatted output contains expected content assert "Parameters for node 'test-node-id':" in result assert "Name: motor_strength" in result @@ -485,17 +502,25 @@ def test_show_parameters_success(self, mock_pathfinder, mock_dataloader, sample_ assert "Scope: simulation" in result assert "Location: source/pinata_simulation.py" in result - @patch("fluidize.adapters.local.graph.DataLoader") + @patch("fluidize.adapters.local.graph.nodeParameters_simulation") @patch("fluidize.adapters.local.graph.PathFinder") - def test_show_parameters_no_parameters(self, mock_pathfinder, mock_dataloader, sample_project): + def test_show_parameters_no_parameters(self, mock_pathfinder, mock_node_params, sample_project): """Test showing parameters when none exist.""" # Mock setup for empty parameters mock_parameters_path = Mock() - mock_pathfinder.get_node_parameters_path.return_value = mock_parameters_path - mock_dataloader.load_json.return_value = {"parameters": []} + mock_pathfinder.get_node_path.return_value = mock_parameters_path + + # Mock the parameters model instance with empty parameters + mock_params_instance = Mock() + mock_params_instance.parameters = [] + mock_node_params.from_file.return_value = mock_params_instance handler = GraphHandler() result = handler.show_parameters(sample_project, "empty-node-id") + # Verify calls + mock_pathfinder.get_node_path.assert_called_once_with(sample_project, "empty-node-id") + mock_node_params.from_file.assert_called_once_with(mock_parameters_path) + # Verify the no parameters message assert result == "No parameters found for node 'empty-node-id'" diff --git a/tests/unit/core/types/file_models/__init__.py b/tests/unit/core/types/file_models/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/unit/core/types/file_models/test_json_file_model_base.py b/tests/unit/core/types/file_models/test_json_file_model_base.py new file mode 100644 index 0000000..f3609a8 --- /dev/null +++ b/tests/unit/core/types/file_models/test_json_file_model_base.py @@ -0,0 +1,300 @@ +"""Unit tests for JSONFileModelBase class.""" + +import json +import tempfile +from typing import ClassVar +from unittest.mock import patch + +import pytest +from pydantic import ValidationError +from upath import UPath + +from fluidize.core.types.file_models.json_file_model_base import JSONFileModelBase + + +class MockJSONFileModel(JSONFileModelBase): + """Test implementation of JSONFileModelBase.""" + + _filename: ClassVar[str] = "test.json" + test_field: str = "default_value" + + # Configure to forbid extra fields for validation tests + model_config: ClassVar = {"extra": "forbid"} + + class Key: + key = "test_key" + + +class MockJSONFileModelNoKey(JSONFileModelBase): + """Test implementation without Key configuration.""" + + _filename: ClassVar[str] = "test_no_key.json" + test_field: str = "default_value" + + +class MockJSONFileModelNoFilename(JSONFileModelBase): + """Test implementation without _filename.""" + + test_field: str = "default_value" + + +class TestJSONFileModelBase: + """Test suite for JSONFileModelBase class.""" + + def test_filepath_property_with_path(self): + """Test filepath property when path is set.""" + model = MockJSONFileModel(test_field="test") + test_path = UPath("/test/path/test.json") + model._filepath = test_path + + assert model.filepath == test_path + + def test_filepath_property_without_path(self): + """Test filepath property when path is not set.""" + model = MockJSONFileModel(test_field="test") + + with pytest.raises(ValueError): + _ = model.filepath + + def test_directory_property(self): + """Test directory property returns parent of filepath.""" + model = MockJSONFileModel(test_field="test") + test_path = UPath("/test/path/test.json") + model._filepath = test_path + + assert model.directory == test_path.parent + + def test_directory_property_without_path(self): + """Test directory property when filepath is not set.""" + model = MockJSONFileModel(test_field="test") + + with pytest.raises(ValueError): + _ = model.directory + + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_from_file_success(self, mock_data_loader): + """Test successful file loading.""" + mock_data_loader.load_json.return_value = {"test_field": "loaded_value"} + directory = UPath("/test/directory") + + result = MockJSONFileModel.from_file(directory) + + assert result.test_field == "loaded_value" + assert result._filepath == directory / "test.json" + mock_data_loader.load_json.assert_called_once_with(directory / "test.json") + + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_from_file_no_filename(self, mock_data_loader): + """Test from_file with class that has no _filename.""" + directory = UPath("/test/directory") + + with pytest.raises(TypeError): + MockJSONFileModelNoFilename.from_file(directory) + + mock_data_loader.load_json.assert_not_called() + + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_from_file_empty_data(self, mock_data_loader): + """Test from_file with empty data.""" + mock_data_loader.load_json.return_value = None + directory = UPath("/test/directory") + + with pytest.raises(FileNotFoundError): + MockJSONFileModel.from_file(directory) + + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_from_file_validation_error(self, mock_data_loader): + """Test from_file with validation error.""" + mock_data_loader.load_json.return_value = {"invalid_field": "value"} + directory = UPath("/test/directory") + + with pytest.raises(ValidationError): + MockJSONFileModel.from_file(directory) + + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_from_file_other_exception(self, mock_data_loader): + """Test from_file with other exception during model validation.""" + mock_data_loader.load_json.return_value = {"test_field": "value"} + directory = UPath("/test/directory") + + with ( + patch.object(MockJSONFileModel, "model_validate", side_effect=RuntimeError("Test error")), + pytest.raises(ValueError), + ): + MockJSONFileModel.from_file(directory) + + def test_from_dict_and_path_success(self): + """Test successful creation from dict and path.""" + data = {"test_field": "dict_value"} + path = UPath("/test/path/test.json") + + result = MockJSONFileModel.from_dict_and_path(data, path) + + assert result.test_field == "dict_value" + assert result._filepath == path + + def test_from_dict_and_path_empty_data(self): + """Test from_dict_and_path with empty data.""" + data = {} + path = UPath("/test/path/test.json") + + with pytest.raises(ValueError): + MockJSONFileModel.from_dict_and_path(data, path) + + def test_from_dict_and_path_validation_error(self): + """Test from_dict_and_path with validation error.""" + data = {"invalid_field": "value"} + path = UPath("/test/path/test.json") + + with pytest.raises(ValidationError): + MockJSONFileModel.from_dict_and_path(data, path) + + def test_from_dict_and_path_other_exception(self): + """Test from_dict_and_path with other exception during validation.""" + data = {"test_field": "value"} + path = UPath("/test/path/test.json") + + with ( + patch.object(MockJSONFileModel, "model_validate", side_effect=RuntimeError("Test error")), + pytest.raises(ValueError), + ): + MockJSONFileModel.from_dict_and_path(data, path) + + def test_model_dump_wrapped_with_key(self): + """Test model_dump_wrapped with Key configuration.""" + model = MockJSONFileModel(test_field="test_value") + + result = model.model_dump_wrapped() + + expected = {"test_key": {"test_field": "test_value"}} + assert result == expected + + def test_model_dump_wrapped_without_key(self): + """Test model_dump_wrapped without Key configuration.""" + model = MockJSONFileModelNoKey(test_field="test_value") + + result = model.model_dump_wrapped() + + expected = {"test_field": "test_value"} + assert result == expected + + def test_model_dump_wrapped_no_key_attribute(self): + """Test model_dump_wrapped when Key class has no key attribute.""" + + # Create a model class with a Key that has no key attribute + class MockModelNoKeyAttr(JSONFileModelBase): + _filename: ClassVar[str] = "test.json" + test_field: str = "default_value" + + class Key: + pass # No key attribute + + model = MockModelNoKeyAttr(test_field="test_value") + result = model.model_dump_wrapped() + + expected = {"test_field": "test_value"} + assert result == expected + + @patch("fluidize.core.utils.dataloader.data_writer.DataWriter") + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_save_with_directory(self, mock_data_loader, mock_data_writer): + """Test save with directory parameter.""" + mock_data_loader.load_json.return_value = {"existing": "data"} + model = MockJSONFileModel(test_field="test_value") + directory = UPath("/test/directory") + + model.save(directory) + + expected_path = directory / "test.json" + assert model._filepath == expected_path + mock_data_loader.load_json.assert_called_once() + mock_data_writer.write_json.assert_called_once() + + @patch("fluidize.core.utils.dataloader.data_writer.DataWriter") + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_save_without_directory(self, mock_data_loader, mock_data_writer): + """Test save without directory parameter using existing filepath.""" + mock_data_loader.load_json.return_value = {"existing": "data"} + model = MockJSONFileModel(test_field="test_value") + model._filepath = UPath("/existing/path/test.json") + + model.save() + + mock_data_loader.load_json.assert_called_once_with(UPath("/existing/path/test.json")) + mock_data_writer.write_json.assert_called_once() + + def test_save_no_filename_attribute(self): + """Test save with class that has no _filename attribute.""" + model = MockJSONFileModelNoFilename(test_field="test_value") + directory = UPath("/test/directory") + + with pytest.raises(TypeError): + model.save(directory) + + def test_save_no_filepath(self): + """Test save without filepath and without directory parameter.""" + model = MockJSONFileModel(test_field="test_value") + + with pytest.raises(ValueError): + model.save() + + @patch("fluidize.core.utils.dataloader.data_writer.DataWriter") + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_save_data_merge(self, mock_data_loader, mock_data_writer): + """Test that save merges new data with existing data.""" + existing_data = {"existing_key": "existing_value", "test_key": {"old_field": "old_value"}} + mock_data_loader.load_json.return_value = existing_data + + model = MockJSONFileModel(test_field="new_value") + model._filepath = UPath("/test/path/test.json") + + model.save() + + # Check that the data was merged correctly + call_args = mock_data_writer.write_json.call_args[0] + written_data = call_args[1] + + assert "existing_key" in written_data + assert written_data["existing_key"] == "existing_value" + assert written_data["test_key"]["test_field"] == "new_value" + + @patch("fluidize.core.utils.dataloader.data_writer.DataWriter") + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_edit_valid_attributes(self, mock_data_loader, mock_data_writer): + """Test edit with valid attributes.""" + mock_data_loader.load_json.return_value = {"test_key": {"test_field": "original_value"}} + model = MockJSONFileModel(test_field="original_value") + model._filepath = UPath("/test/path/test.json") + + model.edit(test_field="new_value") + + assert model.test_field == "new_value" + mock_data_writer.write_json.assert_called_once() + + def test_edit_invalid_attribute(self): + """Test edit with invalid attribute.""" + model = MockJSONFileModel(test_field="original_value") + model._filepath = UPath("/test/path/test.json") + + with pytest.raises(AttributeError): + model.edit(nonexistent_field="value") + + def test_integration_file_operations(self): + """Integration test for file operations.""" + with tempfile.TemporaryDirectory() as temp_dir: + directory = UPath(temp_dir) + test_file = directory / "test.json" + + # Create initial file content (only fields the model expects) + initial_data = {"test_field": "initial_value"} + with open(test_file, "w") as f: + json.dump(initial_data, f) + + # Load from file + model = MockJSONFileModel.from_file(directory) + assert model.test_field == "initial_value" + assert model.filepath == test_file + + # Test that methods exist + assert hasattr(model, "edit") + assert hasattr(model, "save") diff --git a/tests/unit/core/types/file_models/test_parameters_model.py b/tests/unit/core/types/file_models/test_parameters_model.py new file mode 100644 index 0000000..1228c2e --- /dev/null +++ b/tests/unit/core/types/file_models/test_parameters_model.py @@ -0,0 +1,268 @@ +"""Unit tests for ParametersModel class.""" + +from unittest.mock import patch + +import pytest +from pydantic import ValidationError +from upath import UPath + +from fluidize.core.types.file_models.parameters_model import ParametersModel +from fluidize.core.types.parameters import Parameter + + +class TestParametersModel: + """Test suite for ParametersModel class.""" + + def test_parameters_model_initialization_empty(self): + """Test ParametersModel initialization with empty parameters.""" + model = ParametersModel() + + assert model.parameters == [] + assert model._filename == "parameters.json" + + def test_parameters_model_initialization_with_params(self): + """Test ParametersModel initialization with parameters.""" + param1 = Parameter(value="val1", description="desc1", type="text", label="label1", name="param1") + param2 = Parameter(value="val2", description="desc2", type="number", label="label2", name="param2") + + model = ParametersModel(parameters=[param1, param2]) + + assert len(model.parameters) == 2 + assert model.parameters[0] == param1 + assert model.parameters[1] == param2 + + def test_unpack_and_validate_non_dict_data(self): + """Test _unpack_and_validate with non-dictionary data.""" + result = ParametersModel._unpack_and_validate("not_a_dict") + + assert result == "not_a_dict" + + def test_unpack_and_validate_dict_without_key(self): + """Test _unpack_and_validate with dict that doesn't contain the key.""" + data = {"other_key": "value"} + + result = ParametersModel._unpack_and_validate(data) + + assert result == data + + def test_unpack_and_validate_dict_with_key_list(self): + """Test _unpack_and_validate with dict containing parameters key with list.""" + param_data = [ + {"value": "val1", "description": "desc1", "type": "text", "label": "label1", "name": "param1"}, + {"value": "val2", "description": "desc2", "type": "number", "label": "label2", "name": "param2"}, + ] + data = {"parameters": param_data} + + result = ParametersModel._unpack_and_validate(data) + + assert result == {"parameters": param_data} + + def test_unpack_and_validate_dict_with_key_non_list(self): + """Test _unpack_and_validate with dict containing parameters key with non-list value.""" + data = {"parameters": "not_a_list"} + + result = ParametersModel._unpack_and_validate(data) + + assert result == {"parameters": []} + + def test_unpack_and_validate_dict_with_key_none(self): + """Test _unpack_and_validate with dict containing parameters key with None.""" + data = {"parameters": None} + + result = ParametersModel._unpack_and_validate(data) + + assert result == {"parameters": []} + + def test_unpack_and_validate_no_key_config(self): + """Test _unpack_and_validate when class has no Key config.""" + # Test with a different class - manually test the logic + data = {"test_field": "value", "other_field": "data"} + + # This would be the behavior if no Key class exists + # The method would just return the data as-is since there's no key to unpack + result = data # This simulates what would happen without Key config + + assert result == data + + def test_unpack_and_validate_key_config_no_key_attr(self): + """Test _unpack_and_validate when Key config has no key attribute.""" + + # Create a test class with Key config but no key attribute + class TestParametersModelNoKeyAttr(ParametersModel): + class Key: + pass + + data = { + "parameters": [{"value": "val", "description": "desc", "type": "text", "label": "label", "name": "param"}] + } + + result = TestParametersModelNoKeyAttr._unpack_and_validate(data) + + assert result == data + + def test_model_dump_wrapped(self): + """Test model_dump_wrapped returns correctly formatted data.""" + param1 = Parameter(value="val1", description="desc1", type="text", label="label1", name="param1") + param2 = Parameter(value="val2", description="desc2", type="number", label="label2", name="param2") + + model = ParametersModel(parameters=[param1, param2]) + + result = model.model_dump_wrapped() + + assert "parameters" in result + assert len(result["parameters"]) == 2 + assert result["parameters"][0]["name"] == "param1" + assert result["parameters"][1]["name"] == "param2" + + def test_model_dump_wrapped_empty_parameters(self): + """Test model_dump_wrapped with empty parameters.""" + model = ParametersModel() + + result = model.model_dump_wrapped() + + assert result == {"parameters": []} + + def test_key_class_configuration(self): + """Test that Key class is properly configured.""" + assert hasattr(ParametersModel, "Key") + assert hasattr(ParametersModel.Key, "key") + assert ParametersModel.Key.key == "parameters" + + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_from_file_with_wrapped_data(self, mock_data_loader): + """Test from_file with wrapped data structure.""" + param_data = [ + {"value": "val1", "description": "desc1", "type": "text", "label": "label1", "name": "param1"}, + {"value": "val2", "description": "desc2", "type": "number", "label": "label2", "name": "param2"}, + ] + wrapped_data = {"parameters": param_data} + mock_data_loader.load_json.return_value = wrapped_data + + directory = UPath("/test/directory") + result = ParametersModel.from_file(directory) + + assert len(result.parameters) == 2 + assert result.parameters[0].name == "param1" + assert result.parameters[1].name == "param2" + + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_from_file_with_unwrapped_data(self, mock_data_loader): + """Test from_file with unwrapped data structure.""" + param_data = [ + {"value": "val1", "description": "desc1", "type": "text", "label": "label1", "name": "param1"}, + ] + unwrapped_data = {"parameters": param_data} + mock_data_loader.load_json.return_value = unwrapped_data + + directory = UPath("/test/directory") + result = ParametersModel.from_file(directory) + + assert len(result.parameters) == 1 + assert result.parameters[0].name == "param1" + + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_from_file_invalid_parameter_data(self, mock_data_loader): + """Test from_file with invalid parameter data.""" + invalid_data = {"parameters": [{"invalid": "data"}]} + mock_data_loader.load_json.return_value = invalid_data + + directory = UPath("/test/directory") + + with pytest.raises(ValidationError): + ParametersModel.from_file(directory) + + def test_from_dict_and_path_with_valid_data(self): + """Test from_dict_and_path with valid data.""" + param_data = [ + {"value": "val1", "description": "desc1", "type": "text", "label": "label1", "name": "param1"}, + ] + data = {"parameters": param_data} + path = UPath("/test/path/parameters.json") + + result = ParametersModel.from_dict_and_path(data, path) + + assert len(result.parameters) == 1 + assert result.parameters[0].name == "param1" + assert result._filepath == path + + def test_model_validation_integration(self): + """Integration test for model validation with various data formats.""" + # Test with complete parameter data + complete_param = { + "value": "test_value", + "description": "Test description", + "type": "text", + "label": "Test Label", + "name": "test_param", + "latex": "\\alpha", + "location": ["section1", "subsection2"], + "options": [{"value": "opt1", "label": "Option 1"}], + "scope": "global", + } + + data = {"parameters": [complete_param]} + model = ParametersModel.model_validate(data) + + assert len(model.parameters) == 1 + param = model.parameters[0] + assert param.name == "test_param" + assert param.latex == "\\alpha" + assert param.location == ["section1", "subsection2"] + assert len(param.options) == 1 + assert param.scope == "global" + + @patch("fluidize.core.utils.dataloader.data_writer.DataWriter") + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_save_integration(self, mock_data_loader, mock_data_writer): + """Integration test for save functionality.""" + existing_data = {"other_key": "other_value"} + mock_data_loader.load_json.return_value = existing_data + + param1 = Parameter(value="val1", description="desc1", type="text", label="label1", name="param1") + model = ParametersModel(parameters=[param1]) + model._filepath = UPath("/test/path/parameters.json") + + model.save() + + # Verify that data was merged correctly + call_args = mock_data_writer.write_json.call_args[0] + written_data = call_args[1] + + assert "other_key" in written_data + assert "parameters" in written_data + assert len(written_data["parameters"]) == 1 + assert written_data["parameters"][0]["name"] == "param1" + + @patch("fluidize.core.utils.dataloader.data_writer.DataWriter") + @patch("fluidize.core.utils.dataloader.data_loader.DataLoader") + def test_edit_functionality(self, mock_data_loader, mock_data_writer): + """Test edit functionality inherited from base class.""" + mock_data_loader.load_json.return_value = {"parameters": []} + param1 = Parameter(value="val1", description="desc1", type="text", label="label1", name="param1") + model = ParametersModel(parameters=[param1]) + model._filepath = UPath("/test/path/parameters.json") + + # Edit the parameters list + new_param = Parameter(value="val2", description="desc2", type="text", label="label2", name="param2") + model.edit(parameters=[new_param]) + + assert len(model.parameters) == 1 + assert model.parameters[0].name == "param2" + mock_data_writer.write_json.assert_called_once() + + def test_parameters_field_default_factory(self): + """Test that parameters field uses default_factory correctly.""" + model1 = ParametersModel() + model2 = ParametersModel() + + # Ensure each instance gets its own list + assert model1.parameters is not model2.parameters + assert model1.parameters == [] + assert model2.parameters == [] + + # Modify one and ensure the other is unaffected + param = Parameter(value="val", description="desc", type="text", label="label", name="param") + model1.parameters.append(param) + + assert len(model1.parameters) == 1 + assert len(model2.parameters) == 0 diff --git a/tests/unit/managers/test_node.py b/tests/unit/managers/test_node.py new file mode 100644 index 0000000..48d1240 --- /dev/null +++ b/tests/unit/managers/test_node.py @@ -0,0 +1,486 @@ +"""Unit tests for NodeManager - node-scoped operations.""" + +from unittest.mock import Mock, patch + +import pytest + +from fluidize.core.types.graph import GraphData, GraphNode, Position, graphNodeData +from fluidize.core.types.parameters import Parameter +from fluidize.managers.node import NodeManager +from tests.fixtures.sample_projects import SampleProjects + + +class TestNodeManager: + """Test suite for NodeManager class.""" + + @pytest.fixture + def mock_adapter(self): + """Create a mock adapter with graph handler.""" + adapter = Mock() + adapter.graph = Mock() + return adapter + + @pytest.fixture + def sample_project(self): + """Sample project for testing.""" + return SampleProjects.standard_project() + + @pytest.fixture + def sample_node(self): + """Sample graph node for testing.""" + return GraphNode( + id="test-node-001", + position=Position(x=100.0, y=200.0), + data=graphNodeData(label="Test Node", simulation_id="test-sim-001"), + type="simulation", + ) + + @pytest.fixture + def node_manager(self, mock_adapter, sample_project): + """Create a NodeManager instance for testing.""" + return NodeManager(mock_adapter, sample_project, "test-node-001") + + def test_init(self, mock_adapter, sample_project): + """Test NodeManager initialization.""" + node_manager = NodeManager(mock_adapter, sample_project, "test-node-001") + + assert node_manager.adapter is mock_adapter + assert node_manager.project is sample_project + assert node_manager.node_id == "test-node-001" + + def test_get_node_success(self, node_manager, mock_adapter, sample_node): + """Test successful node retrieval.""" + graph_data = GraphData(nodes=[sample_node], edges=[]) + mock_adapter.graph.get_graph.return_value = graph_data + + result = node_manager.get_node() + + assert result == sample_node + mock_adapter.graph.get_graph.assert_called_once_with(node_manager.project) + + def test_get_node_not_found(self, node_manager, mock_adapter): + """Test node not found error.""" + graph_data = GraphData(nodes=[], edges=[]) + mock_adapter.graph.get_graph.return_value = graph_data + + with pytest.raises(ValueError, match="Node with ID 'test-node-001' not found"): + node_manager.get_node() + + def test_exists_true(self, node_manager, mock_adapter, sample_node): + """Test exists returns True when node exists.""" + graph_data = GraphData(nodes=[sample_node], edges=[]) + mock_adapter.graph.get_graph.return_value = graph_data + + assert node_manager.exists() is True + + def test_exists_false(self, node_manager, mock_adapter): + """Test exists returns False when node doesn't exist.""" + graph_data = GraphData(nodes=[], edges=[]) + mock_adapter.graph.get_graph.return_value = graph_data + + assert node_manager.exists() is False + + def test_delete(self, node_manager, mock_adapter): + """Test node deletion.""" + node_manager.delete() + + mock_adapter.graph.delete_node.assert_called_once_with(node_manager.project, "test-node-001") + + def test_update_position(self, node_manager, mock_adapter, sample_node): + """Test node position update.""" + graph_data = GraphData(nodes=[sample_node], edges=[]) + mock_adapter.graph.get_graph.return_value = graph_data + mock_adapter.graph.update_node_position.return_value = sample_node + + result = node_manager.update_position(300.0, 400.0) + + assert result == sample_node + assert sample_node.position.x == 300.0 + assert sample_node.position.y == 400.0 + mock_adapter.graph.update_node_position.assert_called_once() + + @patch("fluidize.managers.node.nodeMetadata_simulation") + def test_get_metadata(self, mock_metadata_class, node_manager): + """Test getting node metadata.""" + mock_metadata = Mock() + mock_metadata_class.from_file.return_value = mock_metadata + + result = node_manager.get_metadata() + + assert result == mock_metadata + mock_metadata_class.from_file.assert_called_once() + + @patch("fluidize.managers.node.nodeProperties_simulation") + def test_get_properties(self, mock_properties_class, node_manager): + """Test getting node properties.""" + mock_properties = Mock() + mock_properties_class.from_file.return_value = mock_properties + + result = node_manager.get_properties() + + assert result == mock_properties + mock_properties_class.from_file.assert_called_once() + + @patch("fluidize.managers.node.nodeParameters_simulation") + def test_get_parameters_model(self, mock_parameters_class, node_manager): + """Test getting node parameters model.""" + mock_parameters = Mock() + mock_parameters.parameters = [] + mock_parameters_class.from_file.return_value = mock_parameters + + result = node_manager.get_parameters_model() + + assert result == mock_parameters + mock_parameters_class.from_file.assert_called_once() + + @patch("fluidize.managers.node.nodeParameters_simulation") + def test_get_parameters(self, mock_parameters_class, node_manager): + """Test getting node parameters list.""" + mock_parameter = Parameter( + value="test_value", description="Test parameter", type="text", label="Test", name="test_param" + ) + mock_parameters = Mock() + mock_parameters.parameters = [mock_parameter] + mock_parameters_class.from_file.return_value = mock_parameters + + result = node_manager.get_parameters() + + assert result == [mock_parameter] + + def test_get_parameter_found(self, node_manager): + """Test getting a specific parameter by name.""" + mock_parameter = Parameter( + value="test_value", description="Test parameter", type="text", label="Test", name="test_param" + ) + + with patch.object(node_manager, "get_parameters", return_value=[mock_parameter]): + result = node_manager.get_parameter("test_param") + assert result == mock_parameter + + def test_get_parameter_not_found(self, node_manager): + """Test getting a parameter that doesn't exist.""" + with patch.object(node_manager, "get_parameters", return_value=[]): + result = node_manager.get_parameter("nonexistent") + assert result is None + + def test_validate_all_valid(self, node_manager, sample_node): + """Test validation when all components are valid.""" + + with ( + patch.object(node_manager, "get_node", return_value=sample_node), + patch.object(node_manager, "get_metadata"), + patch.object(node_manager, "get_properties"), + patch.object(node_manager, "get_parameters", return_value=[]), + ): + result = node_manager.validate() + + assert result["valid"] is True + assert result["graph_node_exists"] is True + assert result["metadata_exists"] is True + assert result["properties_exists"] is True + assert result["parameters_exists"] is True + assert len(result["errors"]) == 0 + + def test_validate_with_errors(self, node_manager): + """Test validation when there are errors.""" + with ( + patch.object(node_manager, "get_node", side_effect=ValueError("Node not found")), + patch.object(node_manager, "get_metadata", side_effect=FileNotFoundError("Metadata not found")), + patch.object(node_manager, "get_properties"), + patch.object(node_manager, "get_parameters", return_value=[]), + ): + result = node_manager.validate() + + assert result["valid"] is False + assert result["graph_node_exists"] is False + assert result["metadata_exists"] is False + assert len(result["errors"]) == 2 + assert "Node not found" in result["errors"][0] + assert "Metadata error: Metadata not found" in result["errors"][1] + + def test_id_property(self, node_manager): + """Test id property returns node_id.""" + assert node_manager.id == "test-node-001" + + def test_data_property(self, node_manager, mock_adapter, sample_node): + """Test data property returns node data.""" + graph_data = GraphData(nodes=[sample_node], edges=[]) + mock_adapter.graph.get_graph.return_value = graph_data + + result = node_manager.data + + assert result == sample_node.data + + @patch("fluidize.managers.node.nodeMetadata_simulation") + def test_update_metadata(self, mock_metadata_class, node_manager): + """Test updating node metadata.""" + mock_metadata = Mock() + mock_metadata_class.from_file.return_value = mock_metadata + + result = node_manager.update_metadata(name="New Name", description="New description") + + assert result == mock_metadata + mock_metadata.edit.assert_called_once_with(name="New Name", description="New description") + + @patch("fluidize.managers.node.nodeMetadata_simulation") + @patch("fluidize.managers.node.PathFinder") + def test_save_metadata(self, mock_path_finder, mock_metadata_class, node_manager): + """Test saving metadata to file.""" + mock_node_path = Mock() + mock_path_finder.get_node_path.return_value = mock_node_path + mock_metadata = Mock() + + node_manager.save_metadata(mock_metadata) + + mock_path_finder.get_node_path.assert_called_once_with(node_manager.project, "test-node-001") + mock_metadata.save.assert_called_once_with(mock_node_path) + + @patch("fluidize.managers.node.nodeProperties_simulation") + def test_update_properties(self, mock_properties_class, node_manager): + """Test updating node properties.""" + mock_properties = Mock() + mock_properties_class.from_file.return_value = mock_properties + + result = node_manager.update_properties(container_image="new:tag", should_run=False) + + assert result == mock_properties + mock_properties.edit.assert_called_once_with(container_image="new:tag", should_run=False) + + @patch("fluidize.managers.node.nodeProperties_simulation") + @patch("fluidize.managers.node.PathFinder") + def test_save_properties(self, mock_path_finder, mock_properties_class, node_manager): + """Test saving properties to file.""" + mock_node_path = Mock() + mock_path_finder.get_node_path.return_value = mock_node_path + mock_properties = Mock() + + node_manager.save_properties(mock_properties) + + mock_path_finder.get_node_path.assert_called_once_with(node_manager.project, "test-node-001") + mock_properties.save.assert_called_once_with(mock_node_path) + + @patch("fluidize.managers.node.nodeParameters_simulation") + def test_update_parameter_existing(self, mock_parameters_class, node_manager): + """Test updating an existing parameter.""" + existing_param = Parameter( + value="old_value", description="Old desc", type="text", label="Old", name="test_param" + ) + mock_parameters = Mock() + mock_parameters.parameters = [existing_param] + mock_parameters_class.from_file.return_value = mock_parameters + + new_param = Parameter(value="new_value", description="New desc", type="text", label="New", name="test_param") + + result = node_manager.update_parameter(new_param) + + assert result == new_param + assert existing_param.value == "new_value" + assert existing_param.description == "New desc" + mock_parameters.save.assert_called_once() + + @patch("fluidize.managers.node.nodeParameters_simulation") + def test_update_parameter_new(self, mock_parameters_class, node_manager): + """Test adding a new parameter.""" + mock_parameters = Mock() + mock_parameters.parameters = [] + mock_parameters_class.from_file.return_value = mock_parameters + + new_param = Parameter(value="new_value", description="New desc", type="text", label="New", name="new_param") + + result = node_manager.update_parameter(new_param) + + assert result == new_param + assert new_param in mock_parameters.parameters + mock_parameters.save.assert_called_once() + + @patch("fluidize.managers.node.nodeParameters_simulation") + def test_update_parameter_with_location(self, mock_parameters_class, node_manager): + """Test updating parameter with location extension.""" + existing_param = Parameter( + value="old_value", description="Old desc", type="text", label="Old", name="test_param", location=["old"] + ) + mock_parameters = Mock() + mock_parameters.parameters = [existing_param] + mock_parameters_class.from_file.return_value = mock_parameters + + new_param = Parameter( + value="new_value", + description="New desc", + type="text", + label="New", + name="test_param", + location=["new", "location"], + ) + + node_manager.update_parameter(new_param) + + assert existing_param.location == ["old", "new", "location"] + + @patch("fluidize.managers.node.nodeParameters_simulation") + def test_set_parameters(self, mock_parameters_class, node_manager): + """Test replacing all parameters.""" + mock_parameters = Mock() + mock_parameters.parameters = [] + mock_parameters_class.from_file.return_value = mock_parameters + + new_params = [ + Parameter(value="val1", description="desc1", type="text", label="label1", name="param1"), + Parameter(value="val2", description="desc2", type="text", label="label2", name="param2"), + ] + + result = node_manager.set_parameters(new_params) + + assert result == new_params + assert mock_parameters.parameters == new_params + mock_parameters.save.assert_called_once() + + @patch("fluidize.managers.node.nodeParameters_simulation") + def test_remove_parameter_success(self, mock_parameters_class, node_manager): + """Test successfully removing a parameter.""" + param1 = Parameter(value="val1", description="desc1", type="text", label="label1", name="param1") + param2 = Parameter(value="val2", description="desc2", type="text", label="label2", name="param2") + + mock_parameters = Mock() + mock_parameters.parameters = [param1, param2] + mock_parameters_class.from_file.return_value = mock_parameters + + result = node_manager.remove_parameter("param1") + + assert result is True + assert mock_parameters.parameters == [param2] + mock_parameters.save.assert_called_once() + + @patch("fluidize.managers.node.nodeParameters_simulation") + def test_remove_parameter_not_found(self, mock_parameters_class, node_manager): + """Test removing a parameter that doesn't exist.""" + param1 = Parameter(value="val1", description="desc1", type="text", label="label1", name="param1") + + mock_parameters = Mock() + mock_parameters.parameters = [param1] + mock_parameters_class.from_file.return_value = mock_parameters + + result = node_manager.remove_parameter("nonexistent") + + assert result is False + assert mock_parameters.parameters == [param1] + mock_parameters.save.assert_not_called() + + def test_show_parameters_empty(self, node_manager): + """Test showing parameters when no parameters exist.""" + with patch.object(node_manager, "get_parameters", return_value=[]): + result = node_manager.show_parameters() + assert result == "No parameters found for node 'test-node-001'" + + def test_show_parameters_with_data(self, node_manager): + """Test showing parameters with data.""" + param1 = Parameter( + value="value1", + description="Description 1", + type="text", + label="Label 1", + name="param1", + latex="\\alpha", + location=["section1"], + scope="global", + ) + param2 = Parameter(value="value2", description="Description 2", type="number", label="Label 2", name="param2") + + with patch.object(node_manager, "get_parameters", return_value=[param1, param2]): + result = node_manager.show_parameters() + + assert "Parameters for node 'test-node-001':" in result + assert "Parameter 1:" in result + assert "Name: param1" in result + assert "Value: value1" in result + assert "LaTeX: \\alpha" in result + assert "Location: ['section1']" in result + assert "Scope: global" in result + assert "Parameter 2:" in result + assert "Name: param2" in result + + @patch("fluidize.managers.node.PathFinder") + def test_get_node_directory(self, mock_path_finder, node_manager): + """Test getting node directory path.""" + mock_path = Mock() + mock_path_finder.get_node_path.return_value = mock_path + + result = node_manager.get_node_directory() + + assert result == mock_path + mock_path_finder.get_node_path.assert_called_once_with(node_manager.project, "test-node-001") + + @patch("fluidize.managers.node.PathFinder") + def test_get_metadata_path(self, mock_path_finder, node_manager): + """Test getting metadata file path.""" + mock_node_path = Mock() + mock_path_finder.get_node_path.return_value = mock_node_path + + # Mock the __truediv__ method to handle the / operator + mock_node_path.__truediv__ = Mock(return_value="mocked_metadata_path") + + node_manager.get_metadata_path() + + mock_path_finder.get_node_path.assert_called_once_with(node_manager.project, "test-node-001") + mock_node_path.__truediv__.assert_called_once() + + @patch("fluidize.managers.node.PathFinder") + def test_get_properties_path(self, mock_path_finder, node_manager): + """Test getting properties file path.""" + mock_path = Mock() + mock_path_finder.get_properties_path.return_value = mock_path + + result = node_manager.get_properties_path() + + assert result == mock_path + mock_path_finder.get_properties_path.assert_called_once_with(node_manager.project, "test-node-001") + + @patch("fluidize.managers.node.PathFinder") + def test_get_parameters_path(self, mock_path_finder, node_manager): + """Test getting parameters file path.""" + mock_path = Mock() + mock_path_finder.get_node_parameters_path.return_value = mock_path + + result = node_manager.get_parameters_path() + + assert result == mock_path + mock_path_finder.get_node_parameters_path.assert_called_once_with(node_manager.project, "test-node-001") + + def test_to_dict_success(self, node_manager, sample_node): + """Test converting node to dictionary successfully.""" + mock_metadata = Mock() + mock_metadata.model_dump.return_value = {"name": "Test Node"} + mock_properties = Mock() + mock_properties.model_dump.return_value = {"container_image": "test:latest"} + mock_parameter = Mock() + mock_parameter.model_dump.return_value = {"name": "param1", "value": "value1"} + + with ( + patch.object(node_manager, "get_node", return_value=sample_node), + patch.object(node_manager, "get_metadata", return_value=mock_metadata), + patch.object(node_manager, "get_properties", return_value=mock_properties), + patch.object(node_manager, "get_parameters", return_value=[mock_parameter]), + patch.object(node_manager, "get_node_directory", return_value="/path/to/node"), + patch.object(node_manager, "get_metadata_path", return_value="/path/to/metadata.yaml"), + patch.object(node_manager, "get_properties_path", return_value="/path/to/properties.yaml"), + patch.object(node_manager, "get_parameters_path", return_value="/path/to/parameters.json"), + ): + result = node_manager.to_dict() + + assert "graph_node" in result + assert "metadata" in result + assert "properties" in result + assert "parameters" in result + assert "paths" in result + assert result["metadata"] == {"name": "Test Node"} + assert result["properties"] == {"container_image": "test:latest"} + assert len(result["parameters"]) == 1 + + def test_to_dict_error(self, node_manager): + """Test to_dict when an error occurs.""" + with patch.object(node_manager, "get_node", side_effect=Exception("Test error")): + result = node_manager.to_dict() + + assert "error" in result + assert result["error"] == "Test error" + assert result["node_id"] == "test-node-001" + assert result["project"] == node_manager.project.id diff --git a/tests/unit/managers/test_project.py b/tests/unit/managers/test_project.py index a047284..aa79276 100644 --- a/tests/unit/managers/test_project.py +++ b/tests/unit/managers/test_project.py @@ -186,23 +186,6 @@ def test_to_dict_with_timestamps(self, mock_adapter): assert result["created_at"] == "2024-01-01T00:00:00Z" assert result["updated_at"] == "2024-01-01T12:00:00Z" - def test_repr(self, project_wrapper, sample_project_summary): - """Test __repr__ method.""" - result = repr(project_wrapper) - expected = f"Project(id='{sample_project_summary.id}', label='{sample_project_summary.label}')" - assert result == expected - - def test_repr_with_none_label(self, mock_adapter): - """Test __repr__ method when label is None.""" - minimal_summary = SampleProjects.minimal_project() - project = ProjectManager(mock_adapter, minimal_summary) - - result = repr(project) - # Handle case where minimal project might have label=None or no label attribute - label_value = getattr(minimal_summary, "label", None) - expected = f"Project(id='{minimal_summary.id}', label='{label_value}')" - assert result == expected - def test_str_with_label(self, project_wrapper, sample_project_summary): """Test __str__ method with label.""" result = str(project_wrapper) @@ -287,10 +270,7 @@ def test_wrapper_with_different_project_types(self, mock_adapter, project_fixtur graph = project.graph assert isinstance(graph, GraphManager) - # String representations should work - repr_result = repr(project) + # String representation should work str_result = str(project) - assert isinstance(repr_result, str) assert isinstance(str_result, str) - assert project_summary.id in repr_result assert project_summary.id in str_result diff --git a/tests/unit/managers/test_project_graph.py b/tests/unit/managers/test_project_graph.py index b419de6..f411022 100644 --- a/tests/unit/managers/test_project_graph.py +++ b/tests/unit/managers/test_project_graph.py @@ -9,6 +9,7 @@ from fluidize.core.types.parameters import Parameter from fluidize.core.types.runs import RunStatus from fluidize.managers.graph import GraphManager +from fluidize.managers.node import NodeManager from tests.fixtures.sample_graphs import SampleGraphs from tests.fixtures.sample_projects import SampleProjects @@ -94,7 +95,8 @@ def test_add_node_success(self, project_graph, mock_adapter): result = project_graph.add_node(node) - assert result == node + assert isinstance(result, NodeManager) + assert result.node_id == node.id mock_adapter.graph.insert_node.assert_called_once_with( project_graph.project, node, @@ -108,7 +110,8 @@ def test_add_node_with_sim_global_false(self, project_graph, mock_adapter): result = project_graph.add_node(node, sim_global=False) - assert result == node + assert isinstance(result, NodeManager) + assert result.node_id == node.id mock_adapter.graph.insert_node.assert_called_once_with(project_graph.project, node, False) def test_add_node_from_scratch_success(self, project_graph, mock_adapter): @@ -142,7 +145,8 @@ def test_add_node_from_scratch_success(self, project_graph, mock_adapter): result = project_graph.add_node_from_scratch(node, node_properties, node_metadata) - assert result == node + assert isinstance(result, NodeManager) + assert result.node_id == node.id mock_adapter.graph.insert_node_from_scratch.assert_called_once_with( project_graph.project, node, @@ -175,7 +179,8 @@ def test_add_node_from_scratch_with_repo_link(self, project_graph, mock_adapter) result = project_graph.add_node_from_scratch(node, node_properties, node_metadata, repo_link) - assert result == node + assert isinstance(result, NodeManager) + assert result.node_id == node.id mock_adapter.graph.insert_node_from_scratch.assert_called_once_with( project_graph.project, node, node_properties, node_metadata, repo_link ) @@ -344,7 +349,8 @@ def test_add_node_from_scratch_delegates_to_adapter(self, project_graph, mock_ad result = project_graph.add_node_from_scratch(node, node_properties, node_metadata) - assert result == node + assert isinstance(result, NodeManager) + assert result.node_id == node.id mock_adapter.graph.insert_node_from_scratch.assert_called_once_with( project_graph.project, node, node_properties, node_metadata, None ) diff --git a/utils/interactive-testing.ipynb b/utils/interactive-testing.ipynb deleted file mode 100644 index 51c23f1..0000000 --- a/utils/interactive-testing.ipynb +++ /dev/null @@ -1,464 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Fluidize-Python Interactive Demo\n", - "\n", - "This notebook demonstrates the fluidize-python library for managing scientific computing projects.\n", - "\n", - "## Setup\n", - "\n", - "First, let's import the client and see where our projects will be stored:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\ud83d\udcc1 Projects will be stored in: /Users/henrybae/.fluidize/projects\n", - "\ud83d\udcc1 Base directory: /Users/henrybae/.fluidize\n", - "\ud83d\ude80 Client ready in 'local' mode!\n" - ] - } - ], - "source": [ - "# Import the fluidize client - handlers auto-register!\n", - "from fluidize.client import FluidizeClient\n", - "\n", - "# Create client and config\n", - "client = FluidizeClient(mode=\"local\")\n", - "\n", - "print(f\"\ud83d\udcc1 Projects will be stored in: {client.config.local_projects_path}\")\n", - "print(f\"\ud83d\udcc1 Base directory: {client.config.local_base_path}\")\n", - "print(f\"\ud83d\ude80 Client ready in '{client.mode}' mode!\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 1. Creating Projects\n", - "\n", - "Let's create some projects with different configurations:" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u2705 Created project 1:\n", - " ID: MUJOCO\n", - " Label: MUJOCO DEMO\n", - " Status: active\n" - ] - } - ], - "source": [ - "# Create a comprehensive project\n", - "project1 = client.projects.create(\n", - " project_id=\"MUJOCO\",\n", - " label=\"MUJOCO DEMO\",\n", - " description=\"A MuJoCo simulation project\",\n", - " status=\"active\",\n", - ")\n", - "\n", - "print(\"\u2705 Created project 1:\")\n", - "print(f\" ID: {project1.id}\")\n", - "print(f\" Label: {project1.label}\")\n", - "print(f\" Status: {project1.status}\")" - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\ud83d\udccb Found 2 projects:\n", - "\n", - " 1. data-pipeline-2024\n", - " Label: Data Processing Pipeline\n", - " Status: active\n", - " Description: A comprehensive data processing pipeline for custo...\n", - "\n", - " 2. MUJOCO\n", - " Label: MUJOCO DEMO\n", - " Status: active\n", - " Description: A MuJoCo simulation project\n", - "\n" - ] - } - ], - "source": [ - "# Get all projects\n", - "projects = client.projects.list()\n", - "\n", - "print(f\"\ud83d\udccb Found {len(projects)} projects:\")\n", - "print()\n", - "\n", - "for i, project in enumerate(projects, 1):\n", - " print(f\"{i:2}. {project.id}\")\n", - " print(f\" Label: {project.label}\")\n", - " print(f\" Status: {project.status}\")\n", - " if project.description:\n", - " print(f\" Description: {project.description[:50]}{'...' if len(project.description) > 50 else ''}\")\n", - " print()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 2. Creating Nodes" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\ud83d\udce6 Imported all node creation types successfully!\n" - ] - } - ], - "source": [ - "# Import required types for creating nodes from scratch\n", - "import datetime\n", - "\n", - "from fluidize.core.types.graph import GraphNode, Position, graphNodeData\n", - "from fluidize.core.types.node import nodeMetadata_simulation, nodeProperties_simulation\n", - "\n", - "print(\"\ud83d\udce6 Imported all node creation types successfully!\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Creating Nodes in Two Ways\n", - "\n", - "Fluidize supports two approaches for creating nodes in your project graph:\n", - "\n", - "1. **`add_node()`** - For nodes that use existing simulation templates\n", - "2. **`add_node_from_scratch()`** - Complete node creation with all files and directories\n", - "\n", - "Let's demonstrate the comprehensive `add_node_from_scratch` approach that creates:\n", - "- Graph node entry in `graph.json`\n", - "- Complete `properties.yaml` with container configuration\n", - "- Rich `metadata.yaml` with authors, tags, and references\n", - "- Source directory (optionally cloned from a repository)" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\ud83c\udfaf Working with project: MUJOCO DEMO\n", - "\ud83d\udcca Current graph state: 1 nodes, 0 edges\n" - ] - } - ], - "source": [ - "# Get our MUJOCO project for node creation\n", - "project = client.projects.get(\"MUJOCO\")\n", - "\n", - "print(f\"\ud83c\udfaf Working with project: {project.label}\")\n", - "print(f\"\ud83d\udcca Current graph state: {len(project.graph.get().nodes)} nodes, {len(project.graph.get().edges)} edges\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Complete Node Creation Example\n", - "\n", - "Let's create a comprehensive MuJoCo simulation node with all metadata:" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u2705 Created GraphNode:\n", - " ID: Mujoco-Simulation\n", - " Label: MuJoCo Humanoid Simulation\n", - " Position: (150.0, 100.0)\n" - ] - } - ], - "source": [ - "# 1. Create the GraphNode (defines position and basic info in the graph)\n", - "mujoco_graph_node = GraphNode(\n", - " id=\"Mujoco-Simulation\",\n", - " position=Position(x=150.0, y=100.0),\n", - " data=graphNodeData(label=\"MuJoCo Humanoid Simulation\"),\n", - " type=\"physics-simulation\",\n", - ")\n", - "\n", - "print(\"\u2705 Created GraphNode:\")\n", - "print(f\" ID: {mujoco_graph_node.id}\")\n", - "print(f\" Label: {mujoco_graph_node.data.label}\")\n", - "print(f\" Position: ({mujoco_graph_node.position.x}, {mujoco_graph_node.position.y})\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u2705 Created nodeProperties:\n", - " Container: nvidia/cuda:11.8-devel-ubuntu20.04\n", - " Mount path: source\n", - " Output folder: source/outputs\n", - " Should run: True\n" - ] - } - ], - "source": [ - "# 2. Create nodeProperties (container and execution configuration)\n", - "mujoco_properties = nodeProperties_simulation(\n", - " # Required fields\n", - " container_image=\"\", # Docker image with CUDA support\n", - " simulation_mount_path=\"source\", # Mount path inside container\n", - " source_output_folder=\"source/outputs\", # Where outputs are stored\n", - ")\n", - "\n", - "mujoco_metadata = nodeMetadata_simulation(\n", - " name=\"MuJoCo Humanoid Locomotion Demo\",\n", - " id=\"mujoco-humanoid-demo\",\n", - " version=\"0.1\",\n", - " description=\"A demo simulation of a humanoid in MuJoCo\",\n", - " date=datetime.date.today(),\n", - " authors=[],\n", - " tags=[],\n", - ")\n", - "\n", - "print(\"\u2705 Created nodeProperties:\")\n", - "print(f\" Container: {mujoco_properties.container_image}\")\n", - "print(f\" Mount path: {mujoco_properties.simulation_mount_path}\")\n", - "print(f\" Output folder: {mujoco_properties.source_output_folder}\")\n", - "print(f\" Should run: {mujoco_properties.should_run}\")" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\ud83d\ude80 Creating node from scratch...\n", - "Warning: Required field 'authors' is missing or empty in nodeMetadata\n", - "Info: Optional field 'image_name' not provided in nodeProperties\n", - "Info: Optional field 'last_run' not provided in nodeProperties\n", - "Info: Optional field 'code_url' not provided in nodeMetadata\n", - "Info: Optional field 'paper_url' not provided in nodeMetadata\n", - "Info: Optional field 'tags' not provided in nodeMetadata\n", - "\u2705 Node created successfully!\n", - " Node ID: Mujoco-Simulation\n", - " Label: MuJoCo Humanoid Simulation\n", - " Type: physics-simulation\n", - " Simulation ID: None\n" - ] - } - ], - "source": [ - "# 4. Create the complete node from scratch!\n", - "print(\"\ud83d\ude80 Creating node from scratch...\")\n", - "\n", - "created_node = project.graph.add_node_from_scratch(\n", - " node=mujoco_graph_node,\n", - " node_properties=mujoco_properties,\n", - " node_metadata=mujoco_metadata,\n", - ")\n", - "\n", - "print(\"\u2705 Node created successfully!\")\n", - "print(f\" Node ID: {created_node.id}\")\n", - "print(f\" Label: {created_node.data.label}\")\n", - "print(f\" Type: {created_node.type}\")\n", - "print(f\" Simulation ID: {created_node.data.simulation_id}\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 3. Retrieving Specific Projects\n", - "\n", - "Get detailed information about a specific project:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\ud83d\udcca Project Details:\n", - " ID: MUJOCO\n", - " Label: MUJOCO DEMO\n", - " Description: A MuJoCo simulation project\n", - " Status: active\n", - " Location: \n", - " Metadata Version: 1.0\n" - ] - } - ], - "source": [ - "# Get project details\n", - "project = client.projects.get(\"MUJOCO\")\n", - "\n", - "print(\"\ud83d\udcca Project Details:\")\n", - "print(f\" ID: {project.id}\")\n", - "print(f\" Label: {project.label}\")\n", - "print(f\" Description: {project.description}\")\n", - "print(f\" Status: {project.status}\")\n", - "print(f\" Location: {project.location}\")\n", - "print(f\" Metadata Version: {project.metadata_version}\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "scrolled": true - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "No start node provided, using first node: Mujoco-Simulation\n", - "BFS traversal starting from node 'Mujoco-Simulation':\n", - " - Adding node to traversal: Mujoco-Simulation, previous node: None\n", - "Nodes to run: ['Mujoco-Simulation']\n", - "Created project run folder: /Users/henrybae/.fluidize/projects/MUJOCO/runs/run_10\n", - "Created run environment with number: 10\n" - ] - }, - { - "data": { - "text/plain": [ - "{'flow_status': 'running', 'run_number': 10}" - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "No parameters.json found for node Mujoco-Simulation\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Executing node Mujoco-Simulation in run 10\n", - "\n", - "=== Starting run for node: Mujoco-Simulation ===\n", - "1. Preparing environment...\n", - "\ud83d\udd0d [Environment] DEBUG: Attempting to load parameters from /Users/henrybae/.fluidize/projects/MUJOCO/runs/run_10/Mujoco-Simulation\n", - "\ud83d\udd0d [Environment] DEBUG: Loaded param_data = {'metadata': {'description': 'Parameter tuning for pinata motor strength', 'version': '1.0'}, 'parameters': [{'name': 'motor_strength', 'value': '20.0', 'type': 'text', 'label': 'Motor Strength', 'description': 'Control signal strength for bat motor (higher = faster swing, more collision force)', 'scope': 'simulation', 'location': ['source/pinata_simulation.py']}]}\n", - "\ud83d\udd0d [Environment] DEBUG: all_params = [{'name': 'motor_strength', 'value': '20.0', 'type': 'text', 'label': 'Motor Strength', 'description': 'Control signal strength for bat motor (higher = faster swing, more collision force)', 'scope': 'simulation', 'location': ['source/pinata_simulation.py']}]\n", - "\ud83d\udd0d [Environment] DEBUG: Found 1 simulation params, 0 properties params\n", - "\ud83d\udd0d [Environment] DEBUG: simulation_params = [{'name': 'motor_strength', 'value': '20.0', 'type': 'text', 'label': 'Motor Strength', 'description': 'Control signal strength for bat motor (higher = faster swing, more collision force)', 'scope': 'simulation', 'location': ['source/pinata_simulation.py']}]\n", - "\ud83d\udd0d [Environment] DEBUG: properties_params = []\n", - "\ud83d\udd0d [Environment] DEBUG: context = {'motor_strength': '20.0'}\n", - "\ud83d\udd0d [Environment] DEBUG: param_locations = {'motor_strength': ['source/pinata_simulation.py']}\n", - "\ud83d\udd0d [Environment] DEBUG: node_run_folder = /Users/henrybae/.fluidize/projects/MUJOCO/runs/run_10/Mujoco-Simulation\n", - "\ud83d\udd0d [Environment] DEBUG: checking location_path = /Users/henrybae/.fluidize/projects/MUJOCO/runs/run_10/Mujoco-Simulation/source/pinata_simulation.py, exists = True\n", - "\ud83d\udd27 [Environment] Processing 1 targeted files (vs exhaustive search)\n", - "\ud83d\udd0d [Environment] DEBUG: files_to_process = [PosixUPath('/Users/henrybae/.fluidize/projects/MUJOCO/runs/run_10/Mujoco-Simulation/source/pinata_simulation.py')]\n", - "\ud83d\udcdd [Environment] Updated parameters in: source/pinata_simulation.py\n", - "2. Executing simulation...\n", - "3. Handling files...\n", - "=== Run completed for node: Mujoco-Simulation with result: True ===\n", - "\n" - ] - } - ], - "source": [ - "from fluidize.core.types.runs import RunFlowPayload\n", - "\n", - "payload = RunFlowPayload(\n", - " name=\"simulation-run-1\", description=\"Running simulation flow\", tags=[\"simulation\", \"analysis\"]\n", - ")\n", - "\n", - "\n", - "project.runs.run_flow(payload)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 4. Updating Projects\n", - "\n", - "Modify existing projects:" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "fluidize-python", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.18" - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} diff --git a/utils/start_jupyter.sh b/utils/start_jupyter.sh deleted file mode 100755 index c85ab0a..0000000 --- a/utils/start_jupyter.sh +++ /dev/null @@ -1,53 +0,0 @@ -#!/bin/bash -# Fluidize-Python Jupyter Notebook Launcher - -set -e # Exit on error - -echo "πŸš€ Starting Fluidize-Python Jupyter Notebook" -echo "=============================================" - -# Change to project root directory (parent of utils) -cd "$(dirname "$0")/.." - -# Ensure uv environment is set up -echo "πŸ“¦ Setting up uv environment..." -if ! command -v uv &> /dev/null; then - echo "❌ uv not found. Please install uv first: https://github.com/astral-sh/uv" - exit 1 -fi - -# Check if environment exists and is up to date -if [ ! -d ".venv" ] || [ "pyproject.toml" -nt ".venv/pyvenv.cfg" ]; then - echo "πŸ“¦ Setting up/updating uv environment..." - uv sync - echo "πŸ“¦ Installing package in development mode..." - uv run pip install -e . -else - echo "πŸ“¦ Using existing uv environment (up to date)" -fi - -# Check if jupyter is installed in the uv environment -echo "πŸ“š Ensuring Jupyter is available..." -if ! uv run jupyter --version &> /dev/null; then - echo "πŸ“š Adding Jupyter to uv environment..." - uv add --dev jupyter -else - echo "πŸ“š Jupyter already available" -fi - -# Show environment info -echo "🐍 Python: $(which python)" -echo "πŸ“ Projects directory: ~/.fluidize/projects/" -echo "πŸ““ Notebook: utils/fluidize_demo.ipynb" -echo "πŸ“‚ Current directory: $(pwd)" -echo "" - -# Start Jupyter notebook from project root -echo "🌟 Starting Jupyter Notebook..." -echo " The notebook will open in your browser automatically" -echo " Press Ctrl+C to stop the server" -echo "" - -# Start Jupyter from the project root so imports work correctly -# The notebook will be available at utils/fluidize_demo.ipynb -uv run jupyter notebook --notebook-dir=. utils/fluidize_demo.ipynb