hofiutils

hofiutils is a lightweight Python library designed to archive, index, and analyze laser-gas simulation datasets. It provides a physics-first API that handles data searching and binary file I/O behind the scenes, exposing raw simulation results as coordinate-aware xarray datasets.

Features

• Lazy Data Loading: Scans metadata instantly using a global index file without loading heavy binary data into memory until requested.
• Fluent API: Chain filtering criteria together seamlessly (e.g., archive.filter(...).get_filenames()).
• Physics-Aware Wrappers: Automatically couples structural arrays (x, y) with physical quantities (density, tele) via xarray.
• Incremental Aggregation: Easily register and merge new batches of simulation data into an existing archive.

---

Installation

To install hofiutils in editable mode (recommended for development and collaboration), clone this repository and install it via pip:

git clone https://github.com/your-username/hofiutils.git
cd hofiutils
pip install -e .

Dependencies

The installation automatically handles the following core scientific Python packages:

• numpy
• pandas
• h5py
• xarray
• netCDF4

---

Quickstart

1. Initialize the Archive

Point the SimulationArchive to the directory containing your compiled .h5 files and the summary.csv index.

from hofiutils import SimulationArchive

# Initialize the master archive
archive = SimulationArchive("./data/archive")

2. Search and Filter Simulations

You can filter the data by passing simulation parameters directly as keyword arguments. The .filter() method returns a new, subsetted archive instance.

# Filter for specific physical parameters
laser_scan = archive.filter(wavelength=800, spot_size=10.0, dopant=0.1)

# Inspect what files match the filter
matching_files = laser_scan.get_filenames()
print(f"Found {len(matching_files)} matching simulations.")

3. Loading a Snapshot (Xarray Integration)

Extract data for a specific simulation file at a given delay time. The data is returned as an xarray.Dataset.

# Grab a snapshot at 10 fs delay from the first matching file
filename = matching_files[0]
snapshot = archive.get_snapshot(filename, delay=1.00e-14)

# Data variables and coordinates are bundled together perfectly
print(snapshot)

# Instant coordinate-aware plotting
snapshot.density.plot(x="x", y="y")

---

Repository Structure

hofiutils/
├── pyproject.toml         # Package metadata and dependency configuration
├── README.md              # Library documentation
├── data/                  # Simulation database (Git-ignored)
│   └── archive/           # Contains compiled .h5 files and summary.csv
├── examples/              # Examples for quick reference and demo
│   └── *.ipynb/           
├── scripts/               # Management utilities
│   └── migrate.py         # Converts raw .npz extracts to structured .h5 files
└── src/                   # Source code
    └── hofiutils/
        ├── __init__.py    # Exposes top-level API
        └── archive.py     # SimulationArchive class definition

---

Maintenance: Registering New Data

The migrate.py script can be used to generate a new set of .h5 files and a corresponding new_batch_summary.csv.

To integrate the new batch into your master archive, drop the new .h5 files into your data archive directory and run the registration method:

archive = SimulationArchive("./data/archive")

# Incorporate the new batch summary into the master index
archive.register_new("./path/to/new_batch_summary.csv")