Feat: database improvements#301
Merged
Merged
Conversation
Change Calculation._to_database properties from a flat dict keyed by
`quantity`/`quantity_selection` to a nested `{quantity: {selection: model}}`
mapping. The outer key is the quantity name (with leading underscores
stripped); the inner dict is keyed by selection with the default source
keyed `default`.
merge_to_database now returns the nested shape and _collect_to_database
merges per quantity. Updates the dispatcher/handler tests and the
_DatabaseData docstring accordingly.
Stoichiometry data is now folded field-wise into Structure_DB, and dispersion (eigenvalue range) into Band_DB. A new PhononBand_DB holds the phonon dispersion frequency range for phonon_band (previously empty). The Dispersion and Stoichiometry dispatchers no longer expose a `_to_database`, so they are no longer collected as standalone top-level properties; their handler `to_database` remains and is called by the parent handlers. This mirrors how cell data is already folded into Structure_DB.
Replace the generic merge for the structure database entry with a custom merge that stores the final structure always, the initial structure (first ionic step of the default trajectory) only when it differs from the final one, and any other source only when it differs from both. Geometries are compared by lattice vectors and positions, so fixed-cell relaxations are still detected as distinct. StructureHandler.to_database now accepts a steps argument so a single geometry can be described; each emitted model is built from one step.
Add first_peak_position and first_peak_height to PairCorrelation_DB. The handler scans the existing sample points of the total g(r) for the first strict local maximum above an absolute threshold (the ideal-gas baseline of 1.0), so small bumps before the first shell are ignored. Both fields are None when no such peak exists.
Adapt tests/regression/test_manual.py to the new _to_database contract. It
resolves each legacy reference key to the nested (quantity, selection) lookup,
compares only the fields the reference actually set (so fields added by the
port are not flagged as changed), and treats the intended changes as
documented exceptions:
- stoichiometry and dispersion are folded into parents -> asserted absent
from the top level;
- the structure entry is split, so only the final geometry is compared
(initial_* is now a separate model covered by the unit tests);
- kpoint stays top-level (phonon exposes it under the phonon selection).
Reference pickles are located via PY4VASP_REGRESSION_DIR (default work/examples)
and the cases skip when they are absent. Result on the port: 70 passed.
…elds
Structure_DB now describes a single geometry with unprefixed fields
(cell_volume, lattice_vector_*, angle_*, ...). The initial/final distinction
is expressed solely by the two models in the structure entry
({"final": ..., "initial": ...}) rather than by prefixed fields packed into
one object. StructureHandler.to_database builds one geometry for the selected
step (default: the final step).
Updates the structure unit test to the unprefixed single-geometry model and
the regression comparison to match legacy final_*/initial_* fields against the
corresponding split models.
Reformat the files touched by this branch with black (slice-colon spacing and line wrapping). No behavioral changes; isort reports no import changes.
CurrentDensityHandler.to_database() previously returned an empty {}. It now
returns a CurrentDensity_DB carrying the grid shape and the min/max/mean of the
current-density magnitude |j| aggregated over all grid points and perturbations.
get_formula_and_compound previously returned all None whenever the element names were missing, discarding the ion counts that were still available. It now keeps num_ion_types (and the primitive reduction) in that case; only the name-derived formula, compound, and unique-type list stay None. A structure without ion-type names therefore still records its ion counts (folded into Structure_DB).
write._write_dataset now skips any field whose schema target is unset (check.is_none) even when the raw data supplies a value: a source that defines no path for a field simply does not store it. Previously the unset VaspData(None) target was passed to h5py as a key, crashing with "A name should be string or bytes, not VaspData" when writing the ionic() dielectric demo through the "ion" source (which has no q_point path). Re-enables the dielectric_function/ionic/ion regression case.
…to-database-refactor-077296
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rework the database payload produced by
Calculation._to_database(): reorganize itinto a nested structure, fold sub-quantities into their parents, and fill in several
per-quantity summaries that were previously empty or missing.
Motivation
The database payload is consumed by the external writer (
vaspdb). Previously everyquantity — including sub-components that only make sense as part of a larger quantity
(
stoichiometry,dispersion) — was a sibling top-level key, and several quantitiesemitted no content at all. This PR gives the payload one entry per user-facing quantity,
richer content, and a more consistent shape.
What changed
Payload shape (breaking for consumers of
properties)propertiesis now a dict of dicts:{quantity: {selection: model}}. The outerkey is the quantity name (leading underscores stripped; group members as
group_quantity, e.g.phonon_mode); the inner dict is keyed by selection with thedefault source keyed
default.Sub-quantities folded into parents
stoichiometryis folded intoStructure_DB;dispersionintoBand_DBand a newPhononBand_DB.stoichiometry/dispersionno longer appear as standalone top-levelentries. (
cellwas already folded intoStructure_DB.)Structure: separate initial/final geometries
finalgeometry (always) andan
initialgeometry (only when it differs from final), plus any other source thatdiffers from both. Geometries are compared by lattice vectors and positions.
Structure_DBgeometry fields lost theirinitial_/final_prefixes — each model nowdescribes a single geometry (
cell_volume,lattice_vector_1,angle_alpha, …).New / richer per-quantity content
first_peak_positionandfirst_peak_heightof the total g(r)(first local maximum above the ideal-gas baseline, so sub-threshold shoulders are
ignored).
CurrentDensity_DBwith the grid shape and min/max/mean of thecurrent-density magnitude (was an empty stub).
PhononBand_DBwith the phonon frequency range (was an empty stub).num_ion_typesand the primitive reduction) are kepteven when element names are absent (formula/compound stay
None).Robustness
write._write_datasetnow skips a field whose schema source defines no path for it,instead of crashing when the raw data supplies a value (fixes writing the
ionic()dielectric demo through the
ionsource).Notes for
vaspdbpropertiesconsumers must handle the nested{quantity: {selection: model}}shape,the removal of standalone
stoichiometry/dispersionkeys, and theStructure_DBfield rename (no
initial_/final_prefixes; separateinitial/finalmodels).