Skip to content

Problem: Incorrect parse when arbitrary keys present. #5

Description

@zerthimon

When keys are set with record_transformer filter, the plugin fails to parse correctly (all values go into sql key):

Config to reproduce:

<source>
  @type tail
  path /fluentd/mysql/mysql-slow.log
  format /^(?<message>.+)$/
  tag mysqld.slow
</source>

<filter mysqld.slow>
  @type record_transformer
  <record>
    docker_host docker.host.name
  </record>
  <record>
    env env_name
  </record>
  <record>
    region region_name
  </record>
  <record>
    geo_region geo_region_name
  </record>
</filter>

<match mysqld.slow>
  @type mysqlslowquerylog
  add_tag_prefix concatenated.
</match>

<match concatenated.**>
  @type file
  @id output1
  path /fluentd/log/data.*.log
  flush_interval 5s
  append true
  time_slice_format %Y%m%d
  time_slice_wait 10m
  time_format %Y%m%dT%H%M%S%z
</match>

Input:

# Time: 130107 11:36:21
# User@Host: root[root] @ localhost []
# Query_time: 0.000378  Lock_time: 0.000111 Rows_sent: 7  Rows_examined: 7
SET timestamp=1357526181;
select * from user;
# Time: 130107 11:38:47
# User@Host: root[root] @ localhost []
# Query_time: 0.002142  Lock_time: 0.000166 Rows_sent: 142  Rows_examined: 142
use information_schema;
SET timestamp=1357526327;
select * from INNODB_BUFFER_PAGE_LRU;

output:

2013-01-07T11:36:21+00:00	concatenated.mysqld.slow	{"user":null,"host":null,"query_time":0.0,"lock_time":0.0,"rows_sent":0,"rows_examined":0,"sql":"region_name geo_region_name # User@Host: root[root] @ localhost [] docker.host.name env_name region_name geo_region_name # Query_time: 0.000378  Lock_time: 0.000111 Rows_sent: 7  Rows_examined: 7 docker.host.name env_name region_name geo_region_name SET timestamp=1357526181; docker.host.name env_name region_name geo_region_name select * from user;"}
2013-01-07T11:38:47+00:00	concatenated.mysqld.slow	{"user":null,"host":null,"query_time":0.0,"lock_time":0.0,"rows_sent":0,"rows_examined":0,"sql":"region_name geo_region_name # User@Host: root[root] @ localhost [] docker.host.name env_name region_name geo_region_name # Query_time: 0.002142  Lock_time: 0.000166 Rows_sent: 142  Rows_examined: 142 docker.host.name env_name region_name geo_region_name use information_schema; docker.host.name env_name region_name geo_region_name SET timestamp=1357526327; docker.host.name env_name region_name geo_region_name select * from INNODB_BUFFER_PAGE_LRU;"}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions