New dial_dataclass that allows for heteroscedastic noise and multiple outputs

[konstpieper]

Adds the following features:

• multiple dataset_y values plus validation of length
• labels for dataset_y columns
• add standardized way to encode output statistics through 'statistics_y'
• better validation of input length

Fixes the following bugs:

• erroneous conversions between variance and standard deviation
• correct inverse transformation of standard deviations
• errors in examples: 1d_sinusoidal_growth_client, manual_client

[Lance-Drane]

Everything else apart from the Distribution classes looks fine to me, and tests + the scripts seemed to run fine for me locally. If you think my suggested fix looks correct then I can probably apply the changes and immediately rebase the changes myself.

[Lance-Drane]

OK, synchronized develop into this branch and also added my suggestion to fix the dataclass types. I also caught an INTERSECT-SDK bug and fixed it there, so it will need to be at least 0.9.3 (you can run uv sync --all-groups --all-extras --all-packages to automatically update your venv with the correct dependency once you pull the latest changes)

I'm getting a few errors from the various scripts, but the unit tests run fine; if we can figure out why these aren't working, we should go ahead and merge. I think all of these might be related to an inconsistent dim_x and dim_y handling in default values.

1d_sable_client

Step 0
Values at testing points [-0.051  0.051]: Mean: [-0.10043607  0.09424892], Stddev: [0.02755723 0.02869024]
Step 0
Values at testing points [-0.051  0.051]: Mean: [-0.10043607  0.09424892], Stddev: [0.02755723 0.02869024]
Running simulation at (0.28): 0.719
WARNING:intersect-sdk:Exception from user's callback function:
cannot reshape array of size 2 into shape (1000,)

1d_sinusoidal_growth_client

WARNING:intersect-sdk:Exception from user's callback function:
cannot reshape array of size 2 into shape (100,)

2d_rosenbrock_client

(this error happens several times for some reason)

WARNING:intersect-sdk:Exception from user's callback function:
Argument must be an image or collection in this Axes
WARNING:intersect-sdk:Exception from user's callback function:
Argument must be an image or collection in this Axes
WARNING:intersect-sdk:shutting down and handling signal 15
WARNING:intersect-sdk:shutting down and handling signal 15
INFO:intersect-sdk-common:Unsubscribed from tmp-.tmp-.tmp-.-.tmp-7ed26860-0098-4249-98a8-b20f29318393.response
WARNING:intersect-sdk:Exception from user's callback function:
Argument must be an image or collection in this Axes
WARNING:intersect-sdk:Exception from user's callback function:
Argument must be an image or collection in this Axes
WARNING:intersect-sdk:Exception from user's callback function:
Argument must be an image or collection in this Axes
WARNING:intersect-sdk:Exception from user's callback function:
Argument must be an image or collection in this Axes
WARNING:intersect-sdk:shutting down and handling signal 15
WARNING:intersect-sdk:shutting down and handling signal 15
WARNING:intersect-sdk:shutting down and handling signal 15

[konstpieper]

I'm getting a few errors from the various scripts, but the unit tests run fine; if we can figure out why these aren't working, we should go ahead and merge. I think all of these might be related to an inconsistent dim_x and dim_y handling in default values.

I can not reproduce any of the errors above with your recent changes. The "user's callback function" seems to indicate that they are happening on the client side. For 1d_sable_client it is probably a ".reshape((self.meshgrid_size,)" in handle_surrogate_values, for 1d_sinusoidal_growth_client it is probably the same place. That would indicate that one of these
means = payload['values']
transformed_stddevs = payload['transformed_stddevs']
does not have the correct shape. But as I said, with current server and client I can not reproduce.

[Lance-Drane]

LGTM, I'm not getting the errors any more