new s3 sink output format with less data for active-active multi cluster consistency checker #4279
Description
Is your feature request related to a problem?
multi-cluster-consistency-checker will detect data loss, data redundant, data inconsistency and lww violation, which only needs the record's PK, commit_ts, origin_ts and the consistency of value of other columns.
In order to reduce the data transfer, here request a new s3 sink output format which only contains PK, commit_ts, origin_ts and checksum of other columns.
Describe the feature you'd like
The file is in JSON format and contains multiple lines.
{"pk_data": {"pk_col_1": "0", "pk_col_2": 0, ...}, commit_ts: 0, origin_ts: 0, checksum: "0"}\r\n
Describe alternatives you've considered
No response
Teachability, Documentation, Adoption, Migration Strategy
No response