Features in cdm-dev to consider

[JohnLCaron]

Dsl to replace NcML:

https://github.com/JohnLCaron/devcdm/tree/main/dataset/src/main/kotlin/dev/ucdm/dataset/cdmdsl

Here are examples (one is pasted below), with Ncml on top, Dsl on bottom, for comparison:

https://github.com/JohnLCaron/devcdm/tree/main/dataset/src/test/data/cdml

The dataset module also has a rewrite of the coordinate system builders and an implementation of nccopy and ncdump.

Other modules have a gRpc client/server, start of a grib port to kotlin, a new grid API in Java.

We could consider porting some of this functionality to netchdf or another library built on top. Have to be cautious about getting too ambitious.

<?xml version="1.0" encoding="UTF-8"?>
<nc:netcdf xmlns:nc="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"
  location="nc/[example.nc](http://example.nc/)">
   
    <nc:variable name="ReletiveHumidity" type="int" shape="time lat lon" orgName="rh">
       <nc:attribute name="long_name" type="string" value="relative humidity"/>
       <nc:attribute name="units" type="string" value="percent"/>
    </nc:variable>
    <nc:variable name="lat" type="float" shape="lat">
       <nc:attribute name="units" type="string" value="degrees_north"/>
       <nc:values separator=" ">41.0 40.0 39.0</nc:values>
    </nc:variable>
    <nc:variable name="lon" type="float" shape="lon">
       <nc:attribute name="units" type="string" value="degrees_east"/>
       <nc:values separator=" ">-109.0 -107.0 -105.0 -103.0</nc:values>
    </nc:variable>
    <nc:variable name="time" type="int" shape="time">
       <nc:attribute name="units" type="string" value="hours"/>
       <nc:values separator=" ">6 18 24 36</nc:values>
    </nc:variable>
</nc:netcdf>

Cdml("nc/[example.nc](http://example.nc/)") {
   getVariable("rh").rename("ReletiveHumidity") {
      addAttribute("long_name").setValue("relatively humidity")
      addAttribute("units", "percent")
   }
  addVariable("lat", ArrayType.FLOAT, "lat").setValues(41.0 40.0 39.0) {
     addAttribute("units", "degrees_north")
  }
  addVariable("lon", ArrayType.FLOAT, "lon").setValues(-109.0 -107.0 -105.0 -103.0) {
     addAttribute("units", "degrees_east")
  }
  addVariable("time", ArrayType.INT, "time").setValues(6 18 24 36) {
     addAttribute("units", "hours since 2020/11/11T00:00:01")
  }

}

[JohnLCaron]

From sean:

Cool, the Cdml DSL is a lot easier on the eyes, that's for sure! I agree with being cautious about getting too ambitious. My opinion is that sticking to the basics with netchdf is good - a small-ish codebase tackling independent implementation details seems so valuable from a documentation / preservation point of view that it's worth being super cautious about what goes into it. One approach could be incubating things like a gRpc client/server, a GRID api, etc. in the main repo, but breaking those out into their own repo once they look settled. What I would want to avoid is the codebase becoming as large as cdm-core, know what I mean?

[JohnLCaron]

May be useful to wrap a netchdf file with Cdml. But much of the utility of Ncml was to then write that back out.

The value of gRPC is to allow access to the library from other languages.

[lesserwhirls]

I think the ability to augment metadata is worth having when dealing with coordinate system detection / feature type wrangling, but not sure how much of that netchdf currently does.

I think gRPC is worth it, but I'd like to settle on a definitive proto for basic level access. The latest in the netCDF-Java project is https://github.com/Unidata/netcdf-java/blob/maint-5.x/cdm/gcdm/src/main/proto/ucar/gcdm/gcdm_netcdf.proto (which probably isn't much different than the one you wrote), but I don't think that proto should live there - I'd like to see it managed in it's own repo, perhaps with a few implementations along side of it (java, python) and docs on how to implement it in other languages.