GUI
The RELOAD GUI helps setting parameters, selecting algorithms and data sets, and checking results. Here we report details about i) setup, and ii) results GUIs. The rest of the RELOAD interaction with the user is limited to configuration files e.g., the loaders, and to output files with results.
From top to bottom, the user performs the following actions. First, the user selects the target metric. Second, the output format is selected: the default option 'ui' provides results through UI, while "basic" only outputs text files.
Clicking the Feature Selection Strategies button opens the window above. This GUI describes the available feature selection strategies, allowing the user to choose the ones that suit the problem the most. Selecting more than one strategy leads applying each strategy sequentially according to the order specified by the user in the table. The "ranked threshold" flag specifies if the threshold should be intended as an absolute number x to select all the features that get a score > x, or an amount y which indicates that we want to select the "best" y features according to such strategy.
Training phase heavily relies on the setup of the k parameter for k-fold validation [34]. If the analysis targets sliding window algorithms, the size of the sliding window should be set (field Window Size and Sliding Policy.
First, the user defines the path of the input folder, that is the root path of all the configuration files and folders described below. Then, the user selects: i) the output folder, which will contain all the results in CSV format; ii) the configuration folder, which contains the loaders and the metrics configuration files; iii) datasets folder, that specifies where textual files (if any) of datasets are placed, and iv) the setup folder, that contains the data sets.
Then, the score folder is identified: it is used only for temporary storage of ranked anomaly checkers during computations.
The user selects the algorithms, among the implemented ones, and the data sets, among those in the setup folder and that have a loader. When a new data loader has to be defined, as it is the case pressing the “Create Loader” button in the “Data Analysis” box allows to choose a file name and opens a GUI to specify key items of loaders. Depending on the type of the loader (CSV, ARFF), RELOAD allows the user to choose relevant items to extract data from the chosen data source.
Once the data loader is defined, the user can choose the algorithms he wants to apply on such dataset(s). By clicking the “Add Algorithm” button, RELOAD opens a small window that allows choosing amongst all the available algorithms.
It is indeed possible to execute algorithms through meta-learning according to the interface below. This allows to "update" the algorithm that was chosen through the basic interface to a meta-learner that may either build only on the selected algorithm (Bagging, Boosting), or on multiple algorithms (others). Specific paramaters for each strategy can be customized through UI.
On the bottom of the GUI, the Update button is used to refresh the interface whenever a configuration is modified, while the Run button starts the experiments.
RELOAD provides a main summary GUI, while producing many files that expand on specific aspects.
In addition, RELOAD creates, for each <dataset, (meta)algorithm> couple, a tab that reports on i) the selected features, ii) train paramaters, iii) metric scores on training set, iv) metric scores on evaluation set, v) plots to see distribution of algorithms scores. Screenshots of the two GUIs can be found in the 'Sample Usage' page.