stream: add input name and infrastructure fields to all input loggers#1403
stream: add input name and infrastructure fields to all input loggers#1403applike-ss wants to merge 2 commits into
Conversation
|
@copilot This does at least not work for sqs, but maybe also does not work for other implementations. Research what is causing it and find+implement a proper fix for it. Make sure that all implementations work as expected by adding integration tests for each stream input type and validate the fields are set and filled with content based on log output. |
…integration tests
Implemented in 50b5c00. Root cause for SQS was that queue fields were captured too early (before queue properties were initialized), so |
|
@copilot would it be enough to set the logger once in the Run func for the sqs input? |
Not reliably. Setting it once in |
Problem
Log messages from stream inputs lacked consistent context, making it hard to identify which configured input produced a log entry or what the corresponding infrastructure resource was.
Changes
All inputs —
inputfieldNewConfigurableInputnow enriches the logger with"input": namebefore passing it to each input factory. Every log message from any input created through the configurable system carries the stream input's config key (e.g.my-consumer).SQS / SNS —
queue_name,queue_url,queue_arnAlready present in
NewSqsInputWithInterfaces(pre-existing work). Fixed the unit tests to expect the threeGetName/GetUrl/GetArnmock calls that construction now triggers.Kafka —
kafka_topicNewKafkaInputresolves the fully-qualified topic name viakafka.BuildFullTopicNameand adds it as"kafka_topic"to the logger before constructing the reader and handing the logger down.Kinesis
No structural change needed:
kinesis.NewKinsumeralready enriches the logger withstream_nameandkinsumer_client_id. Theinputfield from the configurable layer propagates through naturally.Redis list —
redis_server_name,redis_keyNewRedisListInputWithInterfacesenriches the logger with the server name and list key from settings.File —
file_nameNewFileInputWithInterfacesenriches the logger with the filename from settings.InMemory / Noop
Test/no-op transports with no infrastructure resource; left unchanged.