Logging Improvements

[CDx4f3kCAf3Y]

There are many components working together within the C code. This is then compounded by the number of streams configured.

I think it would be a very helpful for troubleshooting to add the name of the component writing a log entry and the name of the stream, if applicable, as fields before the actual log message.

The log entries currently look like this:

[<date> <time>] [<level>] <message>

With this change, they would look like this:

[<date> <time>] [<level>] [<component>] [<stream name (or just blank if no stream name?)>] <message>

[CDx4f3kCAf3Y]

Testing with commit 767ed94.

This is looking good but needs more work.

I am seeing instances where the stream name is duplicated within the same thread:

There are some (as expected) legacy (not having component or stream names) instances:

I think the ones that service the web APIs could be called WebAPI. If the web API is for a specific stream, it could add that too.

I'm not sure if all logging instances will fit nicely into the new model where TLS char buffers can be used to keep the actual log function calls unmodified. As an intermediate step before adding component and stream name arguments to every single log_* call, I'm wondering if instead some #define ... can be used to wrap the log_* functions instead?

Maybe something like this (very crude...)?

// include/core/logger.h
// add a global stream_name variable
const char stream_name = null;
// add `# define ...`s that call the new logging functions names
#define log_error(...) _log_error(component_name, stream_name, __VA_ARGS__)
#define log_debug(...) _log_debug(component_name, stream_name, __VA_ARGS__)

// src/core/logger.c
// prefix original log_* functions with '_' to ensure they don't get called directly
// and add arguments for component and stream names
...
void _log_error(const char *component_name, const char *stream_name, const char *format, ...) {
    // add logic, if needed, to handle when stream_name is null or blank
    ...
}
...
void _log_debug(const char *component_name, const char *stream_name, const char *format, ...) {
    // add logic, if needed, to handle when stream_name is null or blank
    ...
}

// src/web/api_handlers_detection_results.c (or any file calling the original log_* functions)
// at the top of each file, before core/logger.h is included, define the component_name
#define component_name "WebAPI"
#include "core/logger.h"
..
// in the rest of the file, make sure anywhere a stream name is used
// it gets stored in a local stream_name variable (the macro will use this instead of the global stream_name variable)
...
void init_detection_results(void) {
    ...
    // This logging call would result in (stream_name not locally defined) --> [timestamp] [LEVEL] [component] message
    log_info("Detection results storage initialized (using database)");
...
void store_detection_result(const char *stream_name, const detection_result_t *result) {
    if (!stream_name || !result) {
        // This logging call may result in (stream_name is not set) --> [timestamp] [LEVEL] [component] message
        //                               or in (stream_name is set) --> [timestamp] [LEVEL] [component] [stream] message
        log_error("Invalid parameters for store_detection_result: stream_name=%p, result=%p",
                 stream_name, result);
        return;
    }

    // This logging call would result in --> [timestamp] [LEVEL] [component] [stream] message
    log_debug("Storing detection results for stream '%s': %d detections", stream_name, result->count);
...

This way, the c preprocessor will replace the existing log_info() calls with the real call to _log_info() including the new arguments. Then when the file is compiled, if the function has a local stream_name variable (and it is a non-null non-blank value), it uses that and the name shows up in the logs. If the function has a local stream_name variable (but it is a null or blank value) or if the function does not have a local stream_name variable (the global variable being used instead), then the name does not show up in the logs.

What do you think?