Add an option to load/convert the data as an Array or Dictionary.

[mitchphillipson]

Most users transitioning from GAMS are familiar accessing data via arrays. This package should have the ability to convert or load the data directly to an array.

Ideally the array type should have a named index, similar to NamedArrays. The dictionary should be a DefaultDict so that indices that do not appear default to zero.

If the package is updated to satisfy the Tables.jl interface then users can load the data into NamedTuples, but this is an immutable structure so the data could not be manipulated.

[MartinBonde]

Have a look at the Tables interface pull request https://github.com/jd-foster/GDXInterface.jl/pull/11

Default zeros would be dangerous in some cases, if not combined with any sort of domain checking. E.g. x["typo"] -> 0.
We could add a custom default dict type with domain checking, but its not obvious how to treat wildcard domains or domain sets not in the gdx file.

I tend towards keeping read_gdx table-oriented and add explicit conversion helpers instead, e.g. to_dict(gdx, :x; field=:level, default=0.0) and to_array(gdx, :x; field=:level).

What do you think?

[mitchphillipson]

Explicit conversions would be great.

I agree that domain checking is very important. Both wildcards and domain sets not in the GDX should mimic how GAMS treats them. I've prepared a small example:

File 1: Create a GDX

set A /a, b/;

parameter x(A) /a 1, b 2/;

execute_unload "test.gdx", x;

File 2: Load and Use

parameter x, y, z(*);

$gdxin test.gdx
$loaddc x

display x;

* Domain Set: Elements can be added
x("c") = 10;

* Domain Set: You cannot access elements that do not exist, throws an error
*y = x("d"); 

display x;

* Wildcard: You can set any values
z("a") = x("a");
z("b") = x("b");

* Wildcard: You can access elements that do not exist, defaults to 0
y = z("c");

display y;

display z;

The first file creates and exports an indexed parameter, but does not export the domain set. The second file creates three parameters and loads x from the first file.

If you inspect the parameter x (either in the GAMS viewer or the Python gams.transfer package), you can see x has the domain A. For x, you can set values for any element, but you can only get values for elements already contained in x. Notice, we set the value for x("c") but cannot get the value for x("d").

The wildcard domain on z means you can get and set values for any element. We assign z("a") and z("b") then set y be the value from z("c"), which was not declared or is 0.

To summarize:

• Wildcards - Treat as a default dict. You can get/set any value with a default value of 0 (matching GAMS)
• Domain sets that weren't loaded - You can only get values that already exist in the domain, but can set any value which then gets added to the domain.