Add NewEventDetector

[viktorbeck98]

Detects events, that are not known from training as anomalies

[ernstleierzopf]

I tried to work out how the detectors work and how the NewEventDetector could work.

For me the basic functionality of this detector is that each template line is a new event (or path when looking at the NewMatchPathDetector).
Extending this functionality, the user should be able to set event header variables that can be distinguished.
For example with audit.log, the Type could be the an interesting variable.

This would show new events/paths for each template line for each header variable (Type).

@viktorbeck98 is this the correct functionality?

[viktorbeck98]

@ernstleierzopf with "template line" do you mean an unseen event type?

The functionality to address non-event-specific header variables is actually already covered by the NewValueDetector and was added three days ago with this commit: 3e2b3de
I think the NewValueDetector is fitting for this functionality since it's addressing the values of these header variables (by building an itemset of values), whereas the NewEventDetector should by responsible for simply detecting event types that have not been encountered before by simply building an itemset from the event_ids.

[ernstleierzopf]

"template line" as one line of the audit_templates.txt.

What is exactly a new event? Like the event_id itself?
I wanted to use the header_variables just to define the variables that should be used as an extra layer, such as Type.
That could also be done by a parameter in params.

Still do not understand what exactly the right output is - like the path in NewMatchPathDetector.

[viktorbeck98]

Yes that is correct. To give you some context: The template file holds predefined templates for logs. Currently each new line corresponds to a new event type and is addressed by counting from top to bottom which defines the event_id, starting with 0 (i know this is not user friendly but it is at least compatible with literature and will be improved in the near future).

Yes, a new event corresponds to a new event_id.
Example:

1. The NewEventDetector encounters logs with event_id 0, 1, and 2 (corresponding to the templates/event types in line 0, 1, and 2 in the template file) during training, thus constructing the set [0,1,2]. We're actually not interested in any values inside the log, only the event_id for this detector.
2. While testing the detector suddenly receives a log with event_id=3. Since 3 in [0,1,2] = False the detector raises an alert.