Skip to content

Add JSON file import support to task arguments dialog#1892

Open
Mbeaulne wants to merge 1 commit into03-04-bulk_argument_uploadfrom
03-04-bulk_argument_upload_-_json
Open

Add JSON file import support to task arguments dialog#1892
Mbeaulne wants to merge 1 commit into03-04-bulk_argument_uploadfrom
03-04-bulk_argument_upload_-_json

Conversation

@Mbeaulne
Copy link
Collaborator

@Mbeaulne Mbeaulne commented Mar 4, 2026

Description

Added JSON file import support to the task arguments dialog alongside existing CSV import functionality. Users can now import both CSV and JSON files to populate pipeline input values. The JSON import supports single objects for direct value mapping and arrays of objects for bulk input mode. Non-string values (objects, arrays, numbers) are automatically JSON.stringified for proper handling.

Enhanced the bulk value parser to be structure-aware, preventing incorrect splitting on commas inside JSON objects, arrays, or quoted strings during bulk input processing.

Related Issue and Pull requests

Type of Change

  • Bug fix
  • New feature
  • Improvement
  • Cleanup/Refactor
  • Breaking change
  • Documentation update

Checklist

  • I have tested this does not break current pipelines / runs functionality
  • I have tested the changes on staging

Screenshots (if applicable)

Test Instructions

JSON used for testing

[
  {
    "experiment_key": "12345",
    "predictions": "1234",
    "config": { "dslName": "experiment_a", "lr": 0.01, "epochs": 100 },
    "tags": ["production", "v2"],
    "metadata": { "owner": "alice", "team": "ml-platform", "priority": 1 },
    "script": "import torch\nmodel = torch.load('checkpoint.pt')\nresults = model.evaluate(dataset='train')"
  },
  {
    "experiment_key": "1",
    "predictions": "4",
    "config": { "dslName": "experiment_b", "lr": 0.001, "epochs": 200 },
    "tags": ["staging", "v2"],
    "metadata": { "owner": "bob", "team": "ml-platform", "priority": 2 },
    "script": "from sklearn.ensemble import RandomForestClassifier\nclf = RandomForestClassifier(n_estimators=100, max_depth=5)\nclf.fit(X_train, y_train)"
  },
  {
    "experiment_key": "2",
    "predictions": "6",
    "config": { "dslName": "experiment_c", "lr": 0.01, "epochs": 100, "scheduler": { "type": "cosine", "warmup_steps": 500 } },
    "tags": ["production", "v3"],
    "metadata": { "owner": "alice", "team": "recommendations", "priority": 1 },
    "script": "params = {\"batch_size\": 32, \"dropout\": 0.1}\nrun_training(**params)"
  },
  {
    "experiment_key": "9",
    "predictions": "5",
    "config": { "dslName": "experiment_d", "lr": 0.02, "epochs": 50, "scheduler": { "type": "linear", "warmup_steps": 100 } },
    "tags": ["canary"],
    "metadata": { "owner": "carol", "team": "recommendations", "priority": 3 },
    "script": "data = [x for x in range(100) if x % 2 == 0]\nprint(f'filtered {len(data)} items')"
  }
]
  1. Navigate to a pipeline task submission dialog
  2. Test CSV import functionality to ensure existing behavior is preserved
  3. Test JSON import with a single object: {"input1": "value1", "input2": "value2"}
  4. Test JSON import with an array for bulk mode: [{"input1": "a"}, {"input1": "b"}]
  5. Test JSON import with complex values: {"config": {"lr": 0.01, "epochs": 100}}
  6. Verify bulk input parsing handles JSON objects correctly without splitting on internal commas
  7. Test error handling with invalid JSON files

Additional Comments

The import button now accepts both .csv and .json files and dynamically handles the appropriate parsing logic based on file extension. The UI messaging adapts to show whether CSV or JSON data was imported, including appropriate terminology for ignored columns/keys.

@github-actions
Copy link

github-actions bot commented Mar 4, 2026

🎩 To tophat this PR:

You can add the following URL parameter to your browser to tophat this PR:

`?tophat_location=03-04-bulk_argument_upload_-_json/f1cd6c2`

Copy link
Collaborator Author

Mbeaulne commented Mar 4, 2026

@Mbeaulne Mbeaulne changed the title bulk argument upload - json Add JSON file import support to task arguments dialog Mar 4, 2026
@Mbeaulne Mbeaulne force-pushed the 03-04-bulk_argument_upload_-_json branch from 482ea15 to 64c9574 Compare March 4, 2026 19:47
@Mbeaulne Mbeaulne force-pushed the 03-04-bulk_argument_upload_-_json branch from 64c9574 to f1cd6c2 Compare March 4, 2026 20:43
@Mbeaulne Mbeaulne marked this pull request as ready for review March 5, 2026 16:35
@Mbeaulne Mbeaulne requested a review from a team as a code owner March 5, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant