Urban drainage#329
Draft
Leynse wants to merge 77 commits into
Draft
Conversation
…ric function - Apply first to netcdf storage volume - Work in progress still
…(can always add a snapwave one) and with real*4 (can always make a double version as well) - In sfincs_domain.f90, 'call read_netcdf_storage_volume' is now replaced by 'call read_netcdf_quadtree_to_sfincs' - Succesfully tested for testbed run 'storage_volume_qt_sbg_thd2'
- Tested by supplying dummy stoarge volume nc file for manning, now nicely stops
- If netinfiltrationfile exists, then netinftype should be used - Do precheck of netinfiltrationfile before reading in all data
- Current implementation will check and retrieve multiple variables from the same file (e.g. cnb) individually - Added some documentation what part of the code is reading/allocating/generic needed conversions - Kept naming in netcdf files for now consistent with the equivalent netcdf files
…pe (orignal vs netcdf) vs grid type (regular vs quadtree) - Idea for now is if quadtree model, only netcdf input is support, and for regular model only the original binary files
…f90 to sfincs_infiltration.f90
* - Added version for reading in netcdf quadtree manningfile - For now: input argument is still manningfile, internally checked whether it is a netcdf file or binary - For now: expected variable in the netcdf file is 'manning' - For now: tested only with dummy of stoarge nc file - Full testing pending on having ncinput example * - Add non-stopping warning if a manningfile is prescribed while there is also a subgridfile, because then it will be unused! * - Bugfix in general read_netcdf_quadtree_to_sfincs routine * - Bugfix that the netcdf manningfile is actually read in * - Now define 'rghfield' in sfincs_data.f90 rather than sfincs_domain.f90, so we can use that variable for output to netcdf file * - Add 'manning' as netcdf output in case not a subgrid model, and if manningfile supplied (either binary for regular model, or as netcdf for quadtree - Tested and working for new quadtree netcdf manningfile input
…ithub.com/Deltares/SFINCS into 272-add-netcdf-quadtree-option-input-for-all-infiltration-options-using-generic-reader
…r, adding option to mimic drainage
…into redo_infiltration
…into redo_infiltration
…into redo_infiltration
Add sfincs_src_structures module and refactor discharge handling: introduce a cell-wise discharge accumulator qsrc (allocated in sfincs_data and domain init) that both river sources and src-point structures write into. Extract structure logic (pumps, culverts, check valves, controlled gates) from sfincs_discharges into new sfincs_src_structures with initialize/update routines; rename/clean up discharge initialization to initialize_discharges and separate river time-series storage (qsrc_ts). Update continuity and update_discharges to apply qsrc per-cell (OpenACC/OpenMP parallel loops), zero qsrc each step, and use atomic updates when accumulating. Update NetCDF input/output (sfincs_ncinput/F90, sfincs_ncoutput.F90) and runtime output (sfincs_output.f90) to handle river variables and qsrc_ts. Wire new file into project files (Makefile.am, sfincs_lib.vfproj) and initialize/update both modules from sfincs_lib. Misc: OpenACC directives and minor formatting/variable renames adjusted to match the refactor.
Introduce a new logical flag store_river_discharge and wire it into input parsing (.claude added to .gitignore). Guard NetCDF history-file creation, river dimension/variable definition, and river output on this flag so river sources are only recorded when requested. Refactor river-source time interpolation in sfincs_discharges: locate the bracketing interval once, compute an interpolation weight (wt) and use it inside a single parallel loop (added it_prev, it_next, wt) to compute qtsrc per source before atomic accumulation; reduce device/data handling and improve parallel efficiency. Minor bookkeeping adjustments to variable lists for OpenACC/OpenMP loops.
Add the toml-f third-party TOML parser (sources and licenses) and wire it into the build (Makefile.am and Visual Studio .vfproj). Implement TOML-based parsing for src_structure inputs in sfincs_src_structures.f90: new types, constants, read_toml_src_structures routine, validation helpers and a dispatcher in initialize_src_structures() that probes for TOML and falls back to the legacy reader. Also include minor non-functional whitespace/formatting cleanups in sfincs_discharges.f90.
Move all runtime flat arrays and state for source/sink structures out of sfincs_data and into a dedicated sfincs_src_structures module. Rename drainage_* symbols to struc_* (e.g. qdrain -> qstruc, nmindrn_in/out -> struc_nm_in/out, drainage_params -> individual struc_* param arrays) and update readers and runtime code to use the new names. Add helpers allocate_struc_flat_arrays, finalize_src_structures_state and marshal_src_structures_to_flat_arrays to initialize, post-process (cell lookup, distances) and convert TOML-derived structures into the flat arrays. Update sfincs_openacc OpenACC present lists and update_src_structures to reference the new arrays, and remove deallocation/ownership of structure state from sfincs_data. Log messages and comments updated to reflect the new ownership and data flow.
Introduce a new sfincs_rule_expression module that implements a small boolean rule mini-language (parser, bytecode, evaluator and tests) for gate-like source structures. Register the module in the project (vfproj, Makefile.am) and wire it into the codebase: add uses of sfincs_rule_expression and sfincs_src_structures where needed, expose rule bytecode arrays to OpenACC, and ensure rule storage is finalized for accelerator use. Replace several checks and I/O that previously used ndrn with nr_src_structures (and qdrain -> qstruc updates) across sfincs_input.f90, sfincs_lib.f90, sfincs_ncoutput.F90, sfincs_openacc.f90, and sfincs_output.f90. Also adjust a default cd_val and set nr_src_structures=0 in the bathtub configuration. These changes enable per-structure rule-driven gating and proper NetCDF/OpenACC handling of source-structure data.
Refactor src-structure handling to unify TOML and legacy drn paths and add dynamic gate state features. Introduces marshal_src_structures_to_flat_arrays, convert_legacy_to_toml, initialize_gate_status and write_src_structures_log_summary; the legacy fixed-column file is transcribed to a TOML sibling and then parsed to keep a single parser. Adds new flat arrays and fields: struc_nm_obs_1/2, struc_t_state, struc_qmax, struc_flow_coef, struc_opening_duration, struc_closing_duration, and struc_rule_open_src/struc_rule_close_src; removes the old id field and consolidates name handling. Gate handling was reworked into a state machine (closed/open/opening/closing) driven by rules evaluated against obs cells, with timed transitions using opening/closing durations and initialisation of gate status from zs. Culvert/check-valve/pump logic now uses flow_coef and qmax clamps; a number of OpenACC/OpenMP directives and private lists updated to include the new arrays. Misc: initialization/allocation/deallocation updated, truncation warnings renamed for name length, and various comments/documentation adjusted.
Large cleanup and refactor: normalize timer names to lowercase across modules; move river-discharge state and input paths into sfincs_discharges (srcfile, disfile, netsrcdisfile, nr_discharge_points) and update the netCDF discharge reader signature. Remove legacy spatial drainage-mimic (qdrain_rate / initialize_drainage_mimic) and related inputs, and add logical flags (discharges, drainage_structures) to sfincs_data. Ensure precipitation and infiltration now directly accumulate into qsrc (m3/s) within their modules (meteo/infiltration), and make discharge update atomic and early-return when no points. Update input handling for bathtub mode to force input paths to 'none'. Adjust OpenACC kernels/present lists and netCDF output/logging to use the new timer names and new discharge/structure counters. Misc: small API/glue changes (sfincs_src_structures exposes drnfile; sfincs_input imports discharge/drn names; various callers updated). Overall aims: simplify I/O ownership, remove obsolete drainage mimic, and make source-sink accumulation more explicit and consistent.
Rework the sfincs_input module: reorganized and documented the read_sfincs_input driver, clarified behavior of the flat keyword/value parser, and added detailed comments for the per-type read_* helpers. Modernized keyword handling and defaults, preserved backward-compatible legacy keywords (with new wavemaker_<...> overrides), and replaced several integer switch keywords with logical flags/clearer variable names. Removed obsolete switch declarations from sfincs_data.f90 and improved ordering/clarity of domain, forcing, infiltration and output option reads. Overall this is a refactor to improve readability, maintainability and keyword semantics without changing external file formats.
Refactor sfincs_input to introduce a generic get_keyword interface (with type-specific module procedures) and get_keyword_real_array. Replace numerous legacy read_* calls with get_keyword, add support for optional legacy alias arrays and per-alias deprecation warnings, and centralize value parsing via find_value/read_line. Add use of sfincs_log, explicit implicit none, and module visibility (private/public + interface). Update wavemaker and forcing keyword handling to try modern names first and fall back to short legacy names. Remove old removed-keyword error checks for bucketfile/bucket_loss_frac and tidy comments/formatting.
Introduce a new input keyword 'store_maximum_waterlevel' (default .true.) to control storing the maximum water level on the dtmaxout interval. Adjust initialization so the flag is no longer unconditionally reset earlier; it is now disabled only when dtmaxout <= 0 alongside store_maximum_velocity and store_maximum_flux. Also minor comment cleanup and placeholder notes in the output section.
Introduce a second urban-drainage zone type (injection_well) and extend piped_drainage semantics. Docs (input_urban_drainage.rst) expanded to describe two types, examples, new/changed TOML keys (type, injection_rate, maximum_capacity, h_threshold semantics), and updated outputs. Code changes: sfincs_urban_drainage now tracks zone type IDs, per-zone injection rate/maximum capacity, per-zone cumulative injection, and per-cell qmax/backflow handling for each type; update_urban_drainage implements separate logic for piped_drainage (outfall deposit + backflow) and injection_well (area-weighted pumping, cumulative-cap stop). API/variable renames and additions: urban_drainage_q_outfall -> urban_drainage_q_total, urbdrain_cuminj added to NetCDF output with long_name 'urban drainage zone cumulative injection volume', and nc variable long_name updated for total discharge. OpenACC/OpenMP present lists and kernels updated to include new arrays. Log summary and initialization updated to snap outfalls only for piped zones, derive design_precip from max_outfall_rate for piped zones, and error on invalid zone configs. Overall: adds injection-well functionality and necessary I/O, data structures, and runtime behavior while preserving piped drainage behavior.
Replace separate x/y keys (src_1_x, src_1_y, src_2_x, src_2_y, obs_1_x, ...) with compact coordinate-pair array keys (src_1, src_2, obs_1, obs_2). Update per-type required-key checks to expect the new keys and add check_required_coord_pair and read_coord_pair helpers to validate/read 2-element arrays and emit clear errors for missing/invalid entries. Replace legacy get_value calls and key-presence checks with the new readers, and update TOML writer output to serialize src_1/src_2 as [x, y]. Overall improves input format clarity and validation for structure coordinates.
Replace separate outfall_x/outfall_y keys with a single 2-element outfall array (docs and parser). The urban_drainage parser now reads and validates toml arrays for outfall, sets internal outfall_x/y, and updates related warnings. Introduce fmt_real helper to produce consistent, minimal-width real formatting (works around ifx leading-zero quirk) and refactor many log lines to use trim(fmt_real(...)) with aligned labels. Add dtmapout keyword with a backward-compatible alias for dtout. Update NetCDF metadata for urban drainage discharge (clarified long_name) and remove the cumulative injection NetCDF variable/usages.
Change the input keyword in source/src/sfincs_input.f90 from 'regular_output_on_mesh' to 'output_on_quadtree_mesh' and pass the old name as an alias to get_keyword for backward compatibility. Update the inline comment to clarify that this controls writing quadtree output to the quadtree mesh (only relevant for regular meshed grids).
Rename and reorganize source-structure API and implementation for clarity and correct sign semantics. q_src_struc was renamed to src_struc_q_now and all endpoint/observation identifiers were changed from src_1/src_2/obs_1/obs_2 (and nm_in/nm_out/nm_obs_*) to explicit endpoint/obs names (x_s1/y_s1, x_s2/y_s2, x_o1/y_o1, x_o2/y_o2 and src_struc_nm_s1/_s2/_o1/_o2). Adjusted comments to document sign convention (qq > 0 = flow from endpoint 1 -> endpoint 2) and renamed reduction_depth to pump_reduction_depth. Reworked structure type codes (culvert_simple = 2, culvert = 3, gate = 4) and moved/implemented the gate (culvert-style) solver accordingly. Updated all call sites, OpenACC/OpenMP present/private lists, NetCDF and qdrain output writes, and atomic qsrc updates to use the new names and endpoint mapping. This clarifies intent, fixes naming inconsistencies, and readies the module for further fixes to culvert/gate behaviour.
Add support for <=, >= and = comparisons (new cmp_* and tok_* constants), update tokenizer to recognize '=' and two-character '<=' / '>=' tokens, and extend evaluator to handle the new comparators. Strip all whitespace (space, tab, LF, CR) from rule source prior to parsing so inputs like 'z1 < 0.5' parse correctly. Simplify grammar to use symbolic '&' and '|' only (textual 'and'/'or' handling removed). Remove the in-module test_rule_expression debugging subroutine and refine comparator-related error messages.
Adjust formatting of structure log output to align labels and values for improved readability. Updated many WRITE format descriptors in source/src/sfincs_src_structures.f90 to include a 1x separator and lengthened label strings (e.g. ' name :') so columns line up. Also removed an extra blank line in source/src/sfincs_input.f90.
…nto urban_drainage
Add support for an externally-supplied delta bed-level array (dzbext) and optional time-varying bed level output. Changes include: - Introduce logical use_dzbext and allocatable target dzbext in sfincs_data, and deallocate dzbext on shutdown. - Expose dzbext through the BMI getters (shape/type/rank) and lazily allocate/initialize dzbext when the BMI flag is enabled. - Replace the old update_zbuv binding with a BMI-facing bmi_update_bed_level (bound as update_bed_level) that calls a new update_bed_level routine. - Implement update_bed_level to apply dzbext to cell-centre or subgrid bed arrays and rebuild uv-derived quantities (zbuvmx), with separate logic for subgrid and non-subgrid modes. - Add support in NetCDF output to define zb either with or without a time dimension when store_dynamic_bed_level is enabled, and adjust writes so static writes are skipped when the variable is time-varying; per-step writer and map outputs now use subgrid_z_zmin for subgrid runs when writing dynamic bed level. Also includes minor whitespace/cleanup changes. These changes enable external drivers (via BMI) to modify bed level increments and optionally record time-varying bed topography in output files.
Leynse
commented
May 6, 2026
Leynse
left a comment
Leynse
left a comment
Collaborator
Author
There was a problem hiding this comment.
- To be split into separate PRs as discussed, like this cannot be processed
- See also input file discussion comments
| **drnfile = sfincs.drn** | ||
| **Common input keys** | ||
|
|
||
| Every ``[[src_structure]]`` block carries a small set of keys that are shared across all four types. Per-type required and optional keys are documented in the sub-subsections further below. |
Collaborator
Author
There was a problem hiding this comment.
Let's discuss whether this can become [src_structure.XXX]
Collaborator
Author
There was a problem hiding this comment.
[src_structure.pump], [src_structure.culvert_simple], etc.
| src_2_y = 25.0 | ||
| flow_coef = 0.345 | ||
|
|
||
| Culvert (detailed) |
Collaborator
Author
There was a problem hiding this comment.
Discuss whether all of this should go into manual
| - ``2`` — opening (transient, time-based) | ||
| - ``3`` — closing (transient, time-based) | ||
|
|
||
| At every time step, SFINCS checks the current state of the structure. If the structure is closed, it evaluates the ``rules_open`` expression; when that rule becomes true, the structure starts opening and ``fraction_open`` increases linearly from 0 to 1 over ``opening_duration`` seconds. If the structure is open, it evaluates the ``rules_close`` expression; when that rule becomes true, the structure starts closing and ``fraction_open`` decreases linearly from 1 to 0 over ``closing_duration`` seconds. While a structure is opening or closing, SFINCS only looks at the clock — the rules are not re-checked — so the structure cannot rapidly toggle on and off. Set ``opening_duration`` or ``closing_duration`` to ``0.0`` for an instantaneous transition. A structure without rules simply stays fully open for the entire simulation. |
Collaborator
Author
There was a problem hiding this comment.
Discuss whether all of this should go into manual
|
|
||
| # sfincs.drn | ||
|
|
||
| [[src_structure]] |
Collaborator
Author
There was a problem hiding this comment.
I would remove type, as input and have that directly as:
[src_structure.pump]
| Overview | ||
| -------- | ||
|
|
||
| Urban drainage is a simple bulk sink/source model for two kinds of lumped drainage infrastructure: buried pipe networks that discharge to a receiving water body (**piped drainage**) and pumps that remove water from the model and store it underground (**injection wells**). Each **drainage zone** is a polygon in the horizontal plane and has exactly one ``type``: |
Collaborator
Author
There was a problem hiding this comment.
Discuss whether we want all this in manual
| @@ -0,0 +1,233 @@ | |||
| module sfincs_polygons | |||
Collaborator
Author
There was a problem hiding this comment.
Why not put in the 3rd party open section?
| @@ -0,0 +1,298 @@ | |||
| module sfincs_log | |||
Collaborator
Author
There was a problem hiding this comment.
What is the difference with original sfincs_log.f90, which one is used?
Draft
…nto urban_drainage_dike_breaching
Add a new multi-stage Dockerfile (build_scripts/Dockerfile.cpu.ifx) to build SFINCS with the Intel ifx compiler using the oneAPI HPC kit and produce a slim runtime image (static Intel runtime, bundled netcdf-fortran built with ifx). Update sfincs_rule_expression.f90 to recognize and evaluate a new atom 'z1-z2' (atom_z1_minus_z2): added token kind handling, parsing (longest-match for identifiers with '-'), evaluation branch, and updated error/message text to include the new atom.
Add support for interruptible open/close transitions for source structures and export per-structure gate open fraction to NetCDF. Introduces a logical/byte flag src_struc_interruptible (alloc, default false, parsed from TOML) and wires it through allocation, initialization, OpenACC data lists, logging, and parsing. State-machine logic for opening/closing was updated so interruptible structures can reverse mid-ramp by evaluating the opposite rule and reseeding src_struc_t_state to continue the ramp without a jump. Also add a NetCDF variable 'drainage_fraction_open' (his_file%drain_fraction_open_varid) and write src_struc_fraction_open per output timestep. Minor bookkeeping: OpenACC update and put_var calls updated to include the new arrays.
…nto urban_drainage_dike_breaching
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.