Add zstd dict trainer with ZDICT_optimizeTrainFromBuffer_cover (#806) (#806)

[Victor-C-Zhang]

Summary:
Add a dict training framework with an abstract DictTrainer base and a concrete ZstdDictTrainer implementation.

DictTrainer interface (base_dict_trainer.h):

• DictNodeInfo struct: nodeID, codecID, nodeName for discovered dict-requiring nodes.
• DictTrainer abstract class with findDictNodes() and trainDict() virtual methods.
• trainDictsForCandidate(): top-level orchestrator that discovers dict nodes across all registered trainers, collects per-node input samples via collectInputStreams, trains dicts, packs them into the ZL_Dict wire format, patches per-node dict_id and dict_bundle_id into the serialized compressor CBOR, and returns a TrainedCandidate.

trainDictsForCandidate implementation (base_dict_trainer.cpp):

• Creates all registered DictTrainer instances (currently just ZstdDictTrainer).
• For each trainer, calls findDictNodes() to discover nodes, then collectInputStreams() to capture input data flowing into those nodes.
• For each node with samples, calls trainDict() to get packed dict content, then wraps it in Dict_pack().
• Patches the serialized compressor CBOR with per-node dict_id values and sets the bundle ID.

ZstdDictTrainer (zstd_dict_trainer.h/cpp):

• findDictNodes(): discovers nodes named "zl.trainable.zstd" via ZL_Compressor_forEachNode.
• trainDict(): flattens MultiInput samples into byte spans, calls trainRawDict(), then packs the raw dict into the ZL_TrainedZstdContent envelope with the compression level from ZL_LocalParams.
• trainRawDict(): uses ZDICT_optimizeTrainFromBuffer_cover with fallback to ZDICT_trainFromBuffer.

test_dict_training.cpp:

• ZstdDictTrainer unit tests: TrainDictReturnsPackedContent, TrainDictUsesCompressionLevelFromLocalParams, FindDictNodesReturnsEmptyForDefaultCompressor, FindDictNodesFindsMultipleZstdNodes.
• TrainedCandidate helper tests: ReplaceBundleID, ReplaceDictID (single + batch + missing), PackFatBundleRoundTrip, PackFatBundleThrowsOnEmpty.

BUCK changes:

• New dict:base_dict_trainer library target.
• Add dict dep to :trainer target and test deps.

Differential Revision: D105006229

Pulled By: Victor-C-Zhang

[meta-codesync]

@Victor-C-Zhang has exported this pull request. If you are a Meta employee, you can view the originating Diff in D105006229.