Data Model
The Data Model consists of up to 3 levels. Data, ArchivableData (optional), and the actual data object. All data objects extends the Data class. In some cases where the data is archivable such as the Experiment, the Model, and the Project classes then they also have the ArchivableData layer in between. This could be an interface but since it only has one getter/setter it’s easier to just put it as an additional class layer. The diagram below shows the basic layering of the data objects (arhived should be archived):
It is important that we can separate out the archivable data objects to meet the needs of the GUI. Specifically some GUI code relies on classes that extends Data or extends ArchivableData.
The data model objects contain all the columns from the database as attributes. I did a 1 to 1 mapping of database column to data model objects. The database columns _SHOULD _always be guaranteed _BUT _in some cases they haven’t yet been implemented (more below). On the opposite side the archivable attribute doesn’t yet exist in the database layer. But the goal is that there should always be a 1:1 mapping along with additional “helper” attributes.
With that in mind an example of a column that should be in the data model but isn’t is the data for the charts. Under the policy table in the database we have a column called progress (which I believe is supposed to be where the chart data in the experiment view comes from) but since the column is of jsonb type I currently map it to a String called progress. It hasn’t yet been confirmed what it will map to exactly so until that is decided I’ve been pushing back implementing anything. However to move forward and not be held back I created an extra attribute called scores which is of type List so that I can at least stuff some values into the charts (it’s randomly generated). This needs to be refactored but until the json is defined I don’t want to go too far and just have to redo everything multiple times.
When a data object’s attributes are predefined I created Enums. However since the design was still changing and I I’m not always familiar with all the possible values, or even sometimes where they are retrieved from. I didn’t go through the effort of creating converters in JOOQ for the Enums. Instead I just added extra methods with the keyword enum added at the end in the data model objects. So for example Run.runType also has the methods Run.getRunTypeEnum() and Run.setRunTypeEnum(RunType runType). Keep in mind that both the attributes and data model layers themselves were still in flux until very recently, especially in regards to the Run and Policy layers.
A side note, I view the policy as a special data model class even though it really isn’t. I view it this way because it’s used a lot, and in most screens where we need a policy we’ll also need the whole hierarchy of objects related to that policy. So for example we’ll need the run, the experiment, model, and project, which could be to display something on the screen, create a link, etc. That’s why you’ll see in the JOOQ code that it’s almost always a bunch of left joins.
That being said in other cases I have data on the screen that doesn’t seem to have a database column to represent it and therefore I’ve just put “todo” on the screen. Those all need to first be setup in the database layer but I don’t have enough knowledge to do at this time. I’ll be cleaning up the todo’s in the code and the github issues this weekend to help manage that. Nonetheless an example of this is the experiments list view in which the table has more todo’s than populated columns.
Getting back to the parent objects (for a policy this would be the project, model, experiment, and run data model objects) I will only include what I need for the screen and no more. With that in mind the name and id of all the parent data objects are always included as they are almost always needed but beyond that it’s completely dependent on what is needed on the screens. I’ve so far avoided doing SELECT * on the left join tables. Now we could just include everything since it’s not that much data in most cases and it’s not a high volume website but I’ve been holding off for now. Therefore be aware that data from the parent objects is very likely to be missing (outside of id and name) if it’s not needed on the screens.
The database layer is comprised of 2 layers, the Repository layer and the DAO layer. All GUI code ONLY calls the DAO class, it should NOT be aware of the Repository layer. The Repo layer is where all the direct SQL code (in JOOQ) is located. It’s meant to be single sql calls only. The DAO layer is meant to be where all the transactional code and any other multi-sql code is located. In most cases right now it’s a pass-through and so the DAO just extends the Repo classes but this is expected to change as we start to add the ability to save data to the database. Even so the GUI code should not be aware of the Repository layer to keep everything consistent and standard.
You can see an example of this hierarchy with the ProjectDAO class which already includes transactional code with the creation of new projects which requires several layers of data objects to be created transactionally. Outside of creating new projects it’s just a pass-through to the Repository layer. And of course the GUI only uses the ProjectDAO class.
Lastly, and this is expected to be removed once the database layer is more solidified, I have created a class called FakeDataUtils where I put all the fake generated data so that it’s easy to find and remove later. As of this moment the runs and policies are all fake and come from this class because I don’t yet know what the data will be like (for example the json code for the progress column). The chart data is also all generated in this class.