Skip to content

CLI-3550: Confluent kafka link create is not providing feedback from config.txt that the tasks are in error and has no documentation on what is expected in the config.txt file. #3072

@kcarter-bread

Description

@kcarter-bread

Confluent CLI is lacking documentation on how to pass through arguments for syncing consumer offsets, ACL's and auto creating mirror topics.

confluent kafka link create $LINK_NAME \

--cluster $DEST_CLUSTER_ID \

--source-cluster $SOURCE_CLUSTER_ID \

--source-bootstrap-server $BOOTSTRAP_ENDPOINT \

--source-api-key $API_KEY \

--source-api-secret $API_SECRET \

--destination-cluster $DEST_CLUSTER_ID \

--config config.txt

# config.txt: 
auto.create.mirror.topics.enable=true
consumer.offset.sync.enable=true
acl.sync.enable=true

This creates the cluster link and reports no errors in the cli but when checking the tasks on the link they're reporting status in_error:

confluent kafka link task list --link [link-name]

---------------------+----------+------------------------------------
  AclSync            | IN_ERROR | MISCONFIGURATION_ERROR: "ACL
                     |          | migration is enabled but
                     |          | acl.filters is not set. Please
                     |          | set acl.filters to proceed
                     |          | with ACL migration."
  AutoCreateMirror   | IN_ERROR | MISCONFIGURATION_ERROR:
                     |          | "auto.create.mirror.topics.enable
                     |          | is true but no topic filters
                     |          | are specified. No topics will be
                     |          | mirrored."
  ConsumerOffsetSync | IN_ERROR | MISCONFIGURATION_ERROR:
                     |          | "consumer.offset.sync.enable
                     |          | is true but no consumer
                     |          | group filters are specified.
                     |          | No consumer offsets will be
                     |          | migrated."
  TopicConfigsSync   | ACTIVE   |

There are no hints in the cli for what is expected here, there is no documentation that I can find to get consumer offsets ACL's to sync or auto-create mirror topics via the cli

If I hadn't caught this before my DR failover, consumers wouldn't know where to begin with processing.

this is on confluent version v4.21.0

to temporarily resolve this I was able to create the link in the UI.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions